Data processing method and device, equipment and storage medium

Abstract

The embodiment of the invention provides a data processing method and device, equipment and a storage medium, which are used for realizing rapidness and stability of distributed storage. The method is applied to the fields of block chains, big data, cloud technologies, artificial intelligence and the like, and comprises the steps that a data writing request sent by a client is received, and the data writing request is used for indicating data modification of N copies of target data; a consistency rule of the target storage pool is obtained, the consistency rule is used for indicating that the execution of the feedback data write request to the client is successful when response replies returned by M storage nodes are received, and M is a positive integer smaller than or equal to N; sending a data writing request to the N storage nodes to enable the N storage nodes to execute a writing operation of the target data; counting response replies fed back by the N storage nodes; when the response reply meets the consistency rule of the target storage pool, a response message is replied to the client side, and the response message is used for indicating that the data writing request is executed successfully.

Description

Technical Field

[0001] This application relates to the field of distributed storage technology, and particularly to a data processing method, apparatus, device, and storage medium. Background Art

[0002] Currently, in a distributed storage system, in order to ensure data security, strong consistency rules are usually adopted to ensure data consistency. For example, when the data is triple-replicated, a write request needs to be written to all three replica storage nodes (also called OSD nodes) before the client write operation can be successfully responded.

[0003] However, since the storage nodes of a distributed storage system can be built on servers with arbitrary performance, sometimes the disk performance of some servers is worse than that of other servers. Therefore, in the case of poor server performance (which may be due to slow disk performance, or may be due to insufficient central processing or memory, etc.), using strong consistency rules to ensure data consistency will cause a large number of full requests in the distributed storage cluster, resulting in slow data response.

[0004] Therefore, there is an urgent need for a distributed storage solution that can provide fast and stable data response currently. Summary of the Invention

[0005] Embodiments of this application provide a data processing method, apparatus, device, and storage medium for achieving fast and stable distributed storage.

[0006] In view of this, on the one hand, this application provides a data processing method, including: receiving a data write request sent by a client, where the data write request is used to indicate data modification to N replicas of target data, the N replicas are respectively stored in N storage nodes of a target storage pool, N is a positive integer, and the target storage pool is included in a distributed storage system; obtaining a consistency rule of the target storage pool, where the consistency rule is used to indicate that when M storage node response replies are received, a successful execution of the data write request is fed back to the client, and M is a positive integer less than or equal to N; sending the data write request to the N storage nodes so that the N storage nodes perform a write operation on the target data; counting the response replies fed back by the N storage nodes; and when the response replies meet the consistency rule of the target storage pool, replying a response message to the client, where the response message is used to indicate that the data write request is successfully executed.

[0007] On the other hand, this application provides a data processing apparatus, including: a receiving module, configured to receive a data write request sent by a client, where the data write request is used to indicate data modification to N replicas of target data, the N replicas are respectively stored in N storage nodes of a target storage pool, N is a positive integer, and the target storage pool is included in a distributed storage system;

[0008] A processing module, configured to obtain a consistency rule of the target storage pool, where the consistency rule is used to indicate that when response replies returned by M storage nodes are received, it is fed back to the client that the data write request is successfully executed, and M is a positive integer less than or equal to N;

[0009] A sending module, configured to send the data write request to the N storage nodes, so that the N storage nodes perform a write operation on the target data;

[0010] The processing module is configured to count response replies fed back by the N storage nodes;

[0011] The sending module is configured to, when the response reply meets the consistency rule of the target storage pool, send a response message to the client, where the response message is used to indicate that the data write request is successfully executed.

[0012] In a possible design, in another implementation manner of another aspect of the embodiments of the present application, a receiving module is configured to receive update information, where the update information carries flag information, and the flag information is used to indicate that the consistency rule of the target storage pool is that when response replies returned by M storage nodes are received, it is fed back to the client that the data write request is successfully executed, and M is a positive integer less than N;

[0013] The processing module is configured to set the flag information for the target storage pool to obtain a first storage cluster topology map after the distributed storage system is updated; synchronize the first storage cluster topology map to the storage nodes of the distributed storage system.

[0014] In a possible design, in another implementation manner of another aspect of the embodiments of the present application, a receiving module is configured to receive a command line command, where the command line command serves as the update information, and the command line command includes the pool name of the target storage pool and the flag information.

[0015] In a possible design, in another implementation manner of another aspect of the embodiments of the present application, a processing module is configured to obtain the first storage cluster topology map;

[0016] Determine a detection result of the target storage pool according to the first storage cluster topology map;

[0017] When the detection result indicates that the flag information is set for the target storage pool, determine that the consistency rule of the target storage pool is that when response replies returned by M storage nodes are received, it is fed back to the client that the data write request is successfully executed;

[0018] When the detection result indicates that the flag information is not set in the target storage pool, it is determined that the consistency rule of the target storage pool is to feedback to the client that the data write request is successfully executed when responses from N storage nodes are received.

[0019] In a possible design, in another implementation of another aspect of the embodiments of the present application, the processing module is configured to determine the target storage pool according to the first storage cluster topology map;

[0020] Detect the attribute information of the target storage pool to obtain the detection result of the target storage pool, where the attribute information is used to indicate the consistency rule of the target storage pool.

[0021] In a possible design, in another implementation of another aspect of the embodiments of the present application, the sending module sends the data write request to the primary storage node among the N storage nodes, so that the primary storage node performs the write operation of the target data, and the primary storage node is used to store the primary copy of the target data;

[0022] Invoke the primary storage node to send the data write request to N - 1 storage nodes, so that the N - 1 storage nodes perform the write operation of the target data, and the N - 1 storage nodes are used to store the secondary copies of the target data.

[0023] In a possible design, in another implementation of another aspect of the embodiments of the present application, M is greater than or equal to the minimum number of service - available storage nodes.

[0024] In a possible design, in another implementation of another aspect of the embodiments of the present application, the processing module is configured to, after a first failed storage node exists among the N storage nodes and the first failed storage node restarts, use at least one storage node among the M storage nodes to synchronize the data modification of the target data to the first failed storage node.

[0025] In a possible design, in another implementation of another aspect of the embodiments of the present application, the processing module is configured to discard the second failed storage node when the response reply does not meet the consistency rule of the target storage pool and there is a second failed storage node;

[0026] The sending module is configured to send the data write request to a first storage node, where the first storage node is a storage node other than the N storage nodes in the target storage pool.

[0027] Another aspect of the present application provides a computer device, including: a memory, a processor, and a bus system;

[0028] Wherein, the memory is used to store programs;

[0029] The processor is used to execute a program in the memory, and the processor is used to execute the methods of the above aspects according to the instructions in the program code;

[0030] The bus system is used to connect the memory and the processor to enable the memory and the processor to communicate.

[0031] Another aspect of the present application provides a computer-readable storage medium, in which instructions are stored, and when it runs on a computer, it causes the computer to execute the methods of the above aspects.

[0032] Another aspect of the present application provides a computer program product or a computer program. The computer program product or the computer program includes computer instructions, and the computer instructions are stored in a computer-readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the methods provided in the above aspects.

[0033] From the above technical solutions, it can be seen that the embodiments of the present application have the following advantages: Different consistency rules are respectively set for the storage pools in the distributed storage system, so that the storage pools including slow storage nodes can achieve fast data storage, and the storage pools without slow storage nodes can achieve strong consistency storage of data, thereby realizing the fast and stable storage function of the distributed storage system. Description of the Drawings

[0034] Figure 1 It is a schematic diagram of an architecture of a distributed storage system in an embodiment of the present application;

[0035] Figure 2 It is a schematic diagram of an architecture of a storage pool of a distributed storage system in an embodiment of the present application;

[0036] Figure 3 It is a schematic diagram of an embodiment of a data processing method in an embodiment of the present application;

[0037] Figure 4 It is a schematic flowchart of non-strong consistency data processing in an embodiment of the present application;

[0038] Figure 5 It is a schematic diagram of the update of a consistent heterogeneous pool in a distributed storage system in an embodiment of the present application;

[0039] Figure 6 It is a schematic flowchart of a data processing method in an embodiment of the present application;

[0040] Figure 7It is a schematic diagram of data synchronization when a failure occurs in the data processing method in the embodiment of the present application;

[0041] Figure 8 It is a schematic diagram of an embodiment of the data processing device in the embodiment of the present application;

[0042] Figure 9 It is a schematic diagram of another embodiment of the data processing device in the embodiment of the present application;

[0043] Figure 10 It is a schematic diagram of another embodiment of the data processing device in the embodiment of the present application. Detailed implementation manners

[0044] The embodiment of the present application provides a data processing method, device, equipment and storage medium for realizing fast and stable distributed storage.

[0045] Terms such as "first", "second", "third", "fourth", etc. (if any) in the specification, claims and above-mentioned drawings of the present application are used to distinguish similar objects and do not necessarily need to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged under appropriate circumstances so that the embodiments of the present application described here can be implemented in an order other than those illustrated or described here. In addition, the terms "comprising" and "corresponding to" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product or equipment comprising a series of steps or units does not necessarily need to be limited to those steps or units clearly listed, but may include other steps or units not clearly listed or inherent to these processes, methods, products or equipment.

[0046] Currently, in a distributed storage system, in order to ensure data security, strong consistency rules are usually adopted to ensure data consistency. For example, when the data is in three replicas, a write request needs to be written to all three replica storage nodes (also called OSD nodes) before the client write operation can be successfully responded. However, since the storage nodes of the distributed storage system can be built on servers with any performance, sometimes the disk performance of some servers is worse than that of other servers. Therefore, in the case of poor server performance (which may be due to slow disk performance, or may be due to insufficient central processing or memory, etc.), using strong consistency rules to ensure data consistency will cause a large number of full requests in the distributed storage cluster, resulting in slow data response. Therefore, there is an urgent need for a distributed storage solution that can provide fast and stable data response.

[0047] To solve the above technical problems, the present application provides the following technical solutions: receiving a data write request sent by a client, the data write request being used to indicate data modification to N copies of target data, the N copies being respectively stored in N storage nodes of a target storage pool, N being a positive integer, and the target storage pool being included in a distributed storage system; obtaining a consistency rule of the target storage pool, the consistency rule being used to indicate that when M storage nodes return response replies, a data write request execution success is fed back to the client, M being a positive integer less than or equal to N; sending the data write request to the N storage nodes so that the N storage nodes perform a write operation on the target data; counting the response replies fed back by the N storage nodes; when the response replies meet the consistency rule of the target storage pool, sending a response message to the client, the response message being used to indicate that the data write request execution is successful. In this way, different consistency rules are respectively set for storage pools in the distributed storage system, so that storage pools including slow storage nodes can achieve fast data storage, while storage pools not including slow storage nodes can achieve strong consistency data storage, thereby realizing the fast and stable storage function of the distributed storage system.

[0048] Please refer to Figure 1 , Figure 1 which is a schematic diagram of the architecture of a distributed storage system provided by an embodiment of the present application. The architecture of the distributed storage system includes a data storage node cluster 100a, a data management node cluster 200a, a distributed client cluster 300a, a configuration management client 400a, etc. Optionally, the distributed storage system may be a ceph system (a distributed storage system), which can provide object, file, and block storage services at the same time.

[0049] Among them, the data storage node cluster 100a may include at least one data storage node, and the data storage node is also called an OSD node. For example, the data storage node cluster 100a may include a data storage node 11a, a data storage node 12a,....... a data storage node 13a, etc. Here, the number of data storage nodes in the data storage node cluster 100a is not limited.

[0050] Among them, the data storage node cluster 100a is used for data storage. An object storage space (also called OSD) for data storage is deployed on each data storage node, and the number of object storage spaces deployed on each data storage node is one or more. It can be understood that the OSD is used to provide data storage services and manage the reading and writing of disk data. It can be understood that the object storage spaces deployed on the data storage node clusters included in the distributed storage system can be called the object storage spaces in the distributed storage system. It can be understood that in the object storage spaces in the distributed system, each file data is stored as an object. For example, a file can be divided into multiple objects, and then the multiple objects divided from the same file are respectively stored in the corresponding object storage spaces. It can be understood that the multiple objects divided from the same file can be stored in the same object storage space or in different object storage spaces, which specifically depends on the actual mapping result and is not limited here.

[0051] Among them, the data management node cluster 200a may include multiple data management nodes. The data management node is also called a mon node or a Monitor node. For example, the data management node cluster 200a may include data storage node 21a, data storage node 22a,....... data storage node 23a, etc. The number of data management nodes in the data management node cluster 200a is not limited here. Among them, the data management node cluster 200a is used to provide management services for the distributed storage system, used to store various cluster topology information, manage data distribution, such as it can be used to store OSD Map (a kind of information associated with the object storage space, also called space distribution information, also called the storage cluster topology map in this application), Monitor Map (a kind of information associated with the data management node cluster), PG Map (a kind of information associated with the placement group), and CRUSH Map (a kind of information about data distribution), etc. It can be understood that both the data storage node cluster 100a and the data management node cluster 200a belong to the distributed object storage system in the distributed storage system. For example, both the data storage node cluster 100a and the data management node cluster 200a belong to the RADOS (distributed object storage system) of the ceph distributed storage system. RADOS is the cornerstone of storing data in the ceph system. Data is stored as objects in the data storage nodes therein, and is specifically stored in the object storage spaces deployed on the data storage nodes.

[0052] Among them, the distributed client cluster 300a may include at least one client, which may also be referred to as a client. For example, the distributed client cluster 300a may include clients 31a, 32a,..., 33a, etc. Here, the number of clients in the distributed client cluster 300a is not limited. The distributed client cluster 300a is used to send read and write requests for data reading and writing, and cache file metadata and file data. For example, when business object A needs to view file 1 in data storage cluster 100a stored in the distributed storage system, it can obtain the space distribution information (OSD map) associated with the object storage space from the data management node in the distributed storage system through the distributed client corresponding to the business object A (such as distributed client 31a). Then, based on this space distribution information, it can determine the object storage space where file 1 is stored. Then, distributed client 31a sends a data query request to the data storage node where the determined object storage space is located to obtain the file data of file 1 from the data storage node in the distributed storage system.

[0053] Similarly, when business object A needs to modify file 2 in data storage cluster 100a stored in the distributed storage system, it can obtain the space distribution information (OSD map) associated with the object storage space from the data management node in the distributed storage system through the distributed client corresponding to the business object A (such as distributed client 31a). Then, based on this space distribution information, it can determine the object storage space where file 2 is stored. Then, distributed client 31a sends a data modification request to the data storage node where the determined object storage space is located to modify the file data of file 2 from the data storage node in the distributed storage system.

[0054] Among them, the configuration management client 400a can be a client used to manage the configuration data of service components in a distributed storage system. For example, the configuration management client can be used to publish and update the configuration data of service components. It can be understood that the service components in the distributed storage system are used to provide corresponding services. For example, the service component can be a service component for providing storage pool Qos services (storage pool service quality management function), a service component for providing MDS services (metadata management service), etc., which are not limited here. It can be understood that according to the functions corresponding to the service components, the locations where the service components run can also be different. For example, in the service component providing the storage pool Qos function (storage pool service quality function), it can include a Qos-agent component deployed on a distributed client and a Qos-manager component deployed on a management node in the distributed storage system for storage pool Qos management. Another example is that in the service component for providing MDS services (metadata management service), it can include an MDS component deployed on an MDS management node in the distributed storage system for providing MDS services.

[0055] It can be understood that a configuration publishing component for publishing configuration data and a configuration updating component for updating configuration data are running in the configuration management client 400a. Optionally, the configuration publishing component and the configuration updating component can be integrated in the same configuration management client. Optionally, the configuration publishing component and the configuration updating component can be deployed in different clients, and the different clients where they are deployed can be collectively referred to as the configuration management client. Among them, the configuration management client 400a can run on a data management node in a data management node cluster or on any computer device that establishes a data connection with the data management node, which is not limited here. It can be understood that the computer device on which the configuration management client runs can be a server or a terminal device. The server can be an independent physical server, a server cluster or a distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms. The terminal device includes but is not limited to mobile phones, computers, intelligent voice interaction devices, smart home appliances, vehicle-mounted terminals, aircraft, smart speakers, smart home appliances, etc.

[0056] Among them, a distributed storage system can be logically divided into multiple data pools, also known as storage pools. It can be understood that different storage pools (such as storage pool 1 and storage pool 2) in the distributed storage system are logically isolated and are mainly used to store RBD block devices. The reason for the division of storage pool 1 and storage pool 2 may be due to different user performance requirements. For example, storage pool 1 can be a pool composed of HDD (a type of disk) drives, while storage pool 2 is a pool composed of SSD (a type of disk) drives. Each storage pool can perform traffic management on the block devices in the storage pool through the corresponding storage pool QOS service.

[0057] It can be understood that there is a mapping relationship between the RBD (block device) in the client and the RBD (block device) in the distributed object storage system (such as RADOS in the ceph distributed storage system) in the distributed storage system. The RBD in the client is a logical block device provided for users and will have a drive letter in the client (similar to the C drive and D drive on a Windows machine), while the RBD (block device) in RADOS is the actual place where the data of this block device is stored. It can be understood that the RBD (block device) is a logical application, and the data of the RBD is ultimately stored in the object storage space (OSD) in the distributed storage system. For example, as Figure 2 shown, the block device 1 in client 1 corresponds to the block device 1 in data pool 1. Then, the data stored by client 1 in block device 1 is ultimately stored in the object storage space of data pool 1. Similarly, the block device 2 in client 2 corresponds to the block device 2 in data pool 2. Then, the data stored by client 2 in block device 2 is ultimately stored in the object storage space of data pool 2; the block device 3 in client 3 corresponds to the block device 3 in data pool 2. Then, the data stored by client 3 in block device 3 is ultimately stored in the object storage space of data pool 2. It can be understood that different data pools can be physically isolated, that is, the object storage spaces that different data pools can map to can be different, or they can also not be physically isolated, that is, the object storage spaces that different data pools can map to can be the same. This is not limited here and depends on actual requirements.

[0058] It can be understood that the embodiments of the present application can be applied to the field of cloud storage technology. Cloud storage is a new concept extended and developed from the concept of cloud computing. A distributed cloud storage system (hereinafter referred to as the storage system) refers to a storage system that combines a large number of different types of storage devices (storage devices are also called storage nodes) in the network through functions such as cluster applications, grid technology, and distributed storage file systems, and collaborates through application software or application interfaces to jointly provide data storage and service access functions to the outside world. For example, the distributed storage system in the embodiments of the present application can be a distributed cloud storage system. Currently, the storage method of the storage system is as follows: Create a logical volume. When creating a logical volume, physical storage space is allocated for each logical volume. This physical storage space may be composed of the disks of a certain storage device or several storage devices. The client stores data on a certain logical volume, that is, stores the data on the file system. The file system divides the data into many parts, and each part is an object. The object not only contains data but also additional information such as a data identifier (IDentity, ID). The file system writes each object into the physical storage space of the logical volume respectively, and the file system will record the storage location information of each object. Thus, when the client requests to access the data, the file system can enable the client to access the data according to the storage location information of each object. The process of the storage system allocating physical storage space for the logical volume is specifically as follows: According to the capacity estimation of the objects stored in the logical volume (this estimation often has a large margin relative to the actual capacity of the objects to be stored) and the group of the Redundant Array of Independent Disk (RAID), the physical storage space is pre-divided into stripes. A logical volume can be understood as a stripe, thereby allocating physical storage space for the logical volume.

[0059] It can be understood that the embodiments of the present application can also be applied to the field of autonomous driving technology. Autonomous driving technology generally includes technologies such as high-precision maps, environmental perception, behavior decision-making, path planning, and motion control. Autonomous driving technology has a wide range of application prospects. For example, the service components in the embodiments of the present application can be the service components involved in the field of autonomous driving, and the configuration data of the service components can be stored through the distributed storage system provided by the embodiments of the present application.

[0060] It can be understood that the above scenarios are only examples and do not constitute a limitation on the application scenarios of the technical solutions provided by the embodiments of the present application. The technical solutions of the present application can also be applied to other scenarios. For example, as is known to those of ordinary skill in the art, with the evolution of the system architecture and the emergence of new business scenarios, the technical solutions provided by the embodiments of the present application are equally applicable to similar technical problems.

[0061] It should be understood that in the specific embodiments of the present application, when it comes to related stored data such as target data, when the above embodiments of the present application are applied to specific products or technologies, user permission or consent is required, and the collection, use, and processing of relevant data need to comply with the relevant laws, regulations, and standards of relevant countries and regions.

[0062] Based on the above introduction, below, taking Figure 1 the distributed storage system shown as an example, the data processing method in the present application will be introduced. Please refer to Figure 3 One embodiment of the data processing method in the embodiments of the present application includes:

[0063] 301. Receive a data write request sent by a client. The data write request is used to indicate data modification to N copies of target data. The N copies are respectively stored in N storage nodes of a target storage pool. N is a positive integer, and the target storage pool is included in the distributed storage system.

[0064] In this embodiment, the client obtains the spatial distribution information (OSD map, that is, the first storage cluster topology diagram in the present application) associated with the storage space of the data to be modified (i.e., the target data) through the data management node in the distributed storage system, and thus sends the data write request to the storage node storing the main copy of the target data. The data write request may include the data modification information of the target data, such as the object data to be modified and the metadata to be modified, etc.

[0065] In another implementation, the client obtains the spatial distribution information associated with the storage space of the data to be modified (i.e., the target data) through the data management node in the distributed storage system, and thus sends the data write request to all storage nodes storing the target data. The data write request may include the data modification information of the target data, such as the object data to be modified and the metadata to be modified. For example, the target data is a mapping table, and its specific mapping relationship may be as follows: "The first user, the username of the first user is A, and the company name to which the first user belongs is Company A", and the data write request is used to indicate modifying Company A to which the first user belongs to Company B. Then, at this time, the data modification information of the target data is Company B, the metadata to be modified is "the first user", and the object data to be modified is "company name".

[0066] It should be understood that the N copies in this embodiment include the main copy of the target data and N - 1 secondary copies.

[0067] For example, as Figure 4As shown, the client can send the data write request to the master OSD. At this time, the master OSD will receive the data write request and transfer it to the OSD layer of the master OSD for corresponding logical status processing. For example, operations such as detecting the status of the master OSD and the slave OSD corresponding to the target data, detecting the permissions of the request object corresponding to the client, detecting whether the size of the data write request has an operation maximum value, and detecting the permissions of the data write request are performed.

[0068] In this embodiment, as Figure 4 shown, after receiving the data write request, the master OSD also needs to transfer the data write request to the PG layer to encapsulate the data write request (to prepare for forwarding the data write request to the slave OSD later), and then add the encapsulated data write request to the data processing queue.

[0069] 302. Obtain the consistency rule of the target storage pool. The consistency rule is used to indicate that when receiving response replies from M storage nodes, it is fed back to the client that the data write request is successfully executed. M is a positive integer less than or equal to N.

[0070] After the data processing device receives the data write request, it obtains the consistency rule of the target storage pool in response to the data write request. Specifically, after the data processing device receives the data write request, it obtains the first storage cluster topology map of the distributed storage system; detects the storage pool where the master OSD is located according to the first storage cluster topology map, that is, the target storage pool, and then obtains the detection result of the target storage pool; then according to when the detection result indicates that the target storage pool is set with flag information, it means that the target storage pool is a non-strong consistency storage pool, and its consistency rule is that when receiving response replies from M storage nodes, it can feed back a response message indicating that the data write request is successfully executed to the client. At this time, M is a positive integer less than N. If the detection result indicates that the target storage pool is not set with flag information, it means that the target storage pool is a strong consistency storage pool, and its consistency rule is that when receiving response replies from N storage nodes, it can feed back a response message indicating that the data write request is successfully executed to the client.

[0071] For example, as Figure 4 shown, the target data is stored in 4 storage nodes. Among them, the master OSD is used to store the main copy of the target, and the slave OSD1 to slave OSD3 are used to store three slave copies of the target data. In Figure 4In the process shown, if the master OSD detects the existence of the flag information (i.e., flag) in its corresponding storage pool, it determines that the storage pool where the master OSD is located is a non-strong consistency storage pool. At this time, the consistency rule of the storage pool where the master OSD is located can be that when receiving feedback from 3 storage nodes (including the feedback from the master OSD), it is determined that the write operation of the target data is completed.

[0072] Optionally, in this embodiment, the client can also obtain all its storage nodes according to the first storage cluster topology map, and then send the data write request to all the above storage nodes simultaneously. For example, in Figure 4 the scheme shown, the target data is stored in 4 storage nodes. Then when the client sends the data write request, it can obtain the master OSD of the target data and the slave OSD1 to slave OSD3 according to the first storage cluster topology map; then the client sends the data write request to the master OSD and the slave OSD1 to slave OSD3 respectively; finally, each OSD detects its consistency rule and feeds back the detected consistency rule to the master OSD; when the master OSD determines that the consistency rules are the same, it determines the waiting reply queue according to the consistency rule. For example, when the master OSD receives that the consistency rules fed back by the slave OSD1 to slave OSD3 are all non-strong consistency rules, it can set the consistency rule of the current storage pool to that when receiving feedback from 3 storage nodes (including the feedback from the master OSD), it is determined that the write operation of the target data is completed.

[0073] Based on the above description, it can be known that different consistency rules need to be set for each storage pool in the distributed storage system of the present application, and its specific operation can be as follows:

[0074] In one implementation scheme, after the data management cluster in the distributed storage system receives the update information, it determines the first storage pool indicated by the update information according to the original storage cluster topology map, and sets the flag information for the first storage pool according to the flag information carried in the update information; then updates the status information of the first storage pool; finally, updates the updated status information of the first storage pool in the original storage cluster topology map to obtain the updated storage cluster topology map, that is, obtains the first storage cluster topology map, and finally synchronizes the first storage cluster topology map to all storage nodes of the distributed storage. It should be understood that the first storage pool is only used to indicate the corresponding storage pool indicated in the update information and has no specific meaning.

[0075] In another implementation, the data management cluster in the distributed storage system receives update information. At this time, the update information carries a target indication value, where the target indication value indicates that the storage pool sets flag information, and the flag information is preset and used to indicate that the consistency rule of the storage pool is not strong consistency. Then, the distributed storage system determines the first storage pool indicated by the update information according to the original storage cluster topology map, and sets the flag information for the first storage pool. Then, the status information of the first storage pool is updated. Finally, the updated status information of the first storage pool is updated in the original storage cluster topology map to obtain an updated storage cluster topology map, that is, the first storage cluster topology map is obtained. Finally, the first storage cluster topology map is synchronized to all storage nodes of the distributed storage. Based on the above description, in an exemplary solution, Figure 5 The flowchart shown in

[0076] is used to illustrate the setting of the consistency rule of the storage pool in the distributed storage system: First, use the command line command to send the flag information for updating the consistency rule of the storage pool to the main monitor in the distributed storage system (for example, use TopN as the flag information. At this time, the TopN can be understood as waiting for N - 1 storage node responses in addition to the main OSD, and then the response message can be sent to the client). In an exemplary solution, the command line command can be set as follows: Ceph osd pool set <pool name> topn. For example, if the consistency rule of the storage pool is modified to a non-strong consistency rule, the command line command can be as follows: Ceph osd pool set <storage pool 1> topn.

[0077] In another implementable solution, the command line command can also be set as follows: Ceph osd pool set <storage pool 1> 1, where the 1 is not the flag information, but the 1 is used to indicate that the non-strong consistency rule is set for the storage pool 1, and the flag information for setting the non-strong consistency rule is "topn". Then, in response to the command line command, the flag information "topn" is set for the storage pool 1.

[0078] Optionally, the command line command can also be used to send another flag information for updating the consistency rule of the storage pool to the main monitor in the distributed storage system (for example, use N as the flag information. At this time, the N can be understood as after receiving the response replies of all storage nodes that have received the target data, the response message can be sent to the client). In an exemplary solution, the command line command can be set as follows: Ceph osd pool set <pool name> n. For example, if the consistency rule of the storage pool is modified to a strong consistency rule, the command line command can be as follows: Ceph osd pool set <storage pool 1> n.

[0079] Of course, it can also be directly set that when there is no flag information in the storage pool, it is defaulted to the strong consistency rule. At this time, the command-line command can also be used to send an update of the consistency rule of the storage pool to the main monitor in the distributed storage system. In an exemplary solution, the command-line command can be set as follows: Ceph osd pool set <pool name>. For example, to modify the consistency rule of the storage pool to the strong consistency rule, the command-line command can be as follows: Ceph osd pool set <storage pool 1>.

[0080] In another feasible implementation, the command-line command can also be set as follows: Ceph osd pool set <storage pool 1> 0, where the 0 is not flag information, but the 0 is used to indicate that the strong consistency rule is set for the storage pool 1, and the flag information for setting the strong consistency rule is "n"; then in response to the command-line command, the flag information "n" is set for the storage pool 1.

[0081] It should be understood that other methods can also be used to send the update information in the distributed storage system. For example, a management interface for the storage cluster topology map is set in the distributed storage system, and management options (such as options for setting consistency rules) are set for each storage pool. Then, in response to the selection operation of the management options for each storage pool, the consistency rules of each storage pool are directly modified. At the same time, it should be understood that when determining whether a storage pool needs to be set with a non-strong consistency rule, multiple condition judgments can be used. For example, the historical write operation feedback duration of the storage pool is statistically analyzed. If the feedback duration exceeds the threshold within a certain period of time, it is determined that the storage pool can be set with a non-strong consistency rule; or the hardware performance of each storage node in the storage pool is determined through hardware detection. If the hardware performance of at least one storage node does not meet the requirements, it is determined that the storage pool can be set with a non-strong consistency rule; or it is determined whether the data stored in the storage pool needs to maintain strong consistency. If not, it can be determined that the storage pool can be set with a non-strong consistency rule. Specifically, it is not limited here as long as it can meet the actual requirements.

[0082] Then the main monitor updates the consistency rule of the corresponding storage pool in the OSDMAP, that is, sets the TopN flag information for the storage pool, thereby updating the OSDMAP. For example, a corresponding flag information (storage pool 1 - TopN) will be newly added to the status information of the storage pool 1. At this time, the status information of the storage pool 1 stored in the OSDMAP also needs to be updated. Finally, the main Monitor updates the OSDMAP to the slave monitors through the Paxos protocol, and at the same time, the main monitor synchronously sends the updated OSDMAP to all storage nodes (i.e., OSDs).

[0083] Through the above operations, the distributed storage system can adjust the consistency rules of different storage pools during operation. For example, at the first moment, if it is determined that there is a storage node 1 in storage pool 1 whose hardware performance does not meet the requirements, the consistency rule of storage pool 1 can be set to non-strong consistency; at the second moment, the operation and maintenance personnel of the distributed storage system update the hardware of storage node 1 so that the hardware performance of storage node 1 meets the requirements. At this time, the consistency rule of storage pool 1 can be restored to strong consistency.

[0084] It should be understood that when setting the TopN, in order to ensure data availability, it can be set according to the minimum number of service-available storage nodes in the storage pool. For example, for triple replication, N can be set to 2; for quadruple replication, N can be set to 2 or 3; for quintuple replication, N can be set to 3 or 4. Specific settings are not limited here.

[0085] 303. Send the data write request to the N storage nodes so that the N storage nodes perform the write operation of the target data.

[0086] The data processing device sends the data write request to the N storage nodes, causing the N storage nodes to respectively perform the write operation of the target data.

[0087] In this embodiment, as Figure 4 shown, after the main OSD receives the data write request, it enters the storage engine layer to perform the write modification operation of the target data, including object data modification and object metadata modification.

[0088] At the same time, after the main OSD sends the data write request to the slave OSD1 to the slave OSD3, the slave OSD1, slave OSD2, and slave OSD3 also enter the local storage engine to perform the write modification operation of the target data; and after the write operation is completed, a response reply is sent to the main OSD.

[0089] 304. Count the response replies fed back by the N storage nodes.

[0090] The data processing device receives the response replies fed back by each storage node. At this time, the response reply request also includes the response reply of the main storage node.

[0091] In this embodiment, the data processing device can set the response waiting queue of the response reply according to the consistency rule of the target storage pool when receiving the data write request, and then count the response replies according to the response waiting queue.

[0092] The specific setting method can be random, that is, as long as the non-strong consistency rule is satisfied; it can also be set to specify specific storage nodes.

[0093] In an exemplary solution, as Figure 4 shown, after the main OSD receives the data write request, if it is determined that the consistency rule is strong consistency, the response waiting queue can be set to (primary replica, replica 1, replica 2, replica 3). If it is determined that the consistency rule is non-strong consistency, the response waiting queue can be set to (primary replica, replica 1 and replica 2). It should be understood that replica 1 and replica 2 here do not refer to specific replicas, but only indicate that responses from two slave replicas need to be received.

[0094] In another exemplary solution, as Figure 4 shown, after the main OSD receives the data write request, if it is determined that the consistency rule is strong consistency, the response waiting queue can be set to (primary replica, replica 1, replica 2, replica 3). If it is determined that the consistency rule is non-strong consistency and it is determined that the slave OSD2 is a storage node with slow response, the response waiting queue can be set to (primary replica, replica 1 and replica 3). At this time, replica 1 and replica 3 are used to refer to specific replicas.

[0095] In another exemplary solution, as Figure 4 shown, after the main OSD receives the data write request, if it is determined that the consistency rule is strong consistency, the response waiting queue can be set to (primary replica, replica 1, replica 2, replica 3). If it is determined that the consistency rule is non-strong consistency and it is determined that the main OSD is a storage node with slow response, the response waiting queue can be set to (replica 1, replica 2 and replica 3). At this time, replica 1, replica 2 and replica 3 are used to refer to specific replicas.

[0096] 305. When the response reply meets the consistency rule of the target storage pool, a response message is sent back to the client, and the response message is used to indicate that the data write request is successfully executed.

[0097] After the data processing device determines that the response reply meets the consistency rule of the target storage pool, a response message indicating that the data write request for the target data is successfully executed is sent back to the client.

[0098] In this embodiment, in order to ensure the stability of the distributed storage system, the fault recovery consistency of the distributed storage system can also be set. In an exemplary solution, when setting the TopN, it can be set according to the minimum number of storage nodes with available services in the storage pool. In this way, several situations can occur during the process of waiting for the TopN response:

[0099] In one case, if multiple storage nodes fail and stop serving while waiting for the TopN response (i.e., they do not feedback to the client that the data write operation is successful), at this time, since the number of storage nodes that have updated the data for the write operation does not reach the minimum number required for the service to be available, the updated data of the write operation of each storage node is unavailable. Therefore, the data processing device does not return a response message to the client. Then, after the storage nodes restart, the client cannot read the corresponding data either. In this case, the distributed storage system can discard the storage nodes of this data write request in this request and then resend the data write request.

[0100] In another case, if multiple storage nodes fail and stop serving after the TopN response is collected (i.e., they have feedback to the client that the data write operation is successful), because N storage nodes (including the primary OSD) have successfully updated the write operation to their respective local storages, after the failed storage nodes restart, N storage nodes can have the data written by the client last time. At this time, the client can read the data written by the client last time. At the same time, the storage nodes can also use the peering synchronization recovery mechanism of the PG, so that the storage nodes that did not have time to update the data before the failure can synchronize the modified data. For example, as Figure 6 shown, after the TopN response is collected, the primary OSD, the secondary OSD1, and the secondary OSD2 have completed the data modification; then assume that the secondary OSD1, the secondary OSD2, and the secondary OSD3 fail, resulting in the secondary OSD3 not having time to complete the data modification; after the secondary OSD1, the secondary OSD2, and the secondary OSD3 restart, the secondary OSD3 can obtain the modified data from the primary OSD.

[0101] The following uses a specific non-strong consistency data processing to illustrate the data processing in this embodiment. As Figure 7 shown:

[0102] The client initiates a data write request to the primary OSD (i.e., the OSD1); then the OSD1 initiates a copy request to the OSD2 and the OSD3, so that the OSD2 and the OSD3 also execute the data write request; since the OSD3 is a slow node, the non-strong consistency rule can be adopted in this data write request. Therefore, after receiving the response replies from the OSD1 and the OSD2, the OSD1 returns a response message to the client.

[0103] The following describes the data processing device in this application in detail. Please refer to Figure 8 , Figure 8 which is a schematic diagram of an embodiment of the data processing device in the embodiment of this application. The data processing device 20 includes:

[0104] A receiving module 201, configured to receive a data write request sent by a client, where the data write request is used to indicate data modification to a primary copy of target data and N secondary copies, the N secondary copies are respectively stored in N storage nodes of a target storage pool, N is a positive integer, and the target storage pool is included in a distributed storage system;

[0105] A processing module 202, configured to obtain a consistency rule of the target storage pool, where the consistency rule is used to indicate that when response replies are received from M storage nodes, a data write request execution success is fed back to the client, and M is a positive integer less than or equal to N;

[0106] A sending module 203, configured to send the data write request to the N storage nodes, so that the N storage nodes perform a write operation on the target data;

[0107] The processing module 202 is configured to count response replies fed back by the N storage nodes;

[0108] The sending module 203 is configured to, when the response reply meets the consistency rule of the target storage pool, send a response message to the client, where the response message is used to indicate that the data write request is successfully executed.

[0109] In an embodiment of the present application, a data processing device is provided. By using the above device, different consistency rules are respectively set for storage pools in a distributed storage system, so that a storage pool including slow storage nodes can achieve fast data storage, and a storage pool not including slow storage nodes can achieve strong consistency storage of data, thereby realizing the fast and stable storage function of the distributed storage system.

[0110] Optionally, based on the corresponding embodiment above, in another embodiment of the data processing device 20 provided in the embodiment of the present application, Figure 8 A receiving module 201 is configured to receive update information, where the update information carries flag information, and the flag information is used to indicate that the consistency rule of the target storage pool is that when response replies are received from M storage nodes, a data write request execution success is fed back to the client, and M is a positive integer less than N;

[0111] A processing module 202 is configured to set the flag information for the target storage pool to obtain a first storage cluster topology map after the distributed storage system is updated; and synchronize the first storage cluster topology map to storage nodes of the distributed storage system.

[0112]

[0113] ​In an embodiment of the present application, a data processing device is provided. By using the above device, the consistency rules of a flag information storage pool are set, and at the same time, the cluster topology map of the distributed storage system is utilized to enable each storage node to synchronously obtain the flag information, so that the storage node can know whether to perform weakly consistent storage, and further realize the fast and stable storage function of the distributed storage system.

[0114] Optionally, based on the corresponding embodiment above, in another embodiment of the data processing device 20 provided in the embodiment of the present application, a receiving module 201 is configured to receive a command line command, and the command line command serves as the update information, and the command line command includes the pool name of the target storage pool and the flag information. Figure 8 In an embodiment of the present application, a data processing device is provided. By using the above device, the consistency rules of the distributed storage system are set by using a command line command, which can achieve fast and accurate setting. At the same time, the feasibility of the solution is increased.

[0115] Optionally, based on the corresponding embodiment above, in another embodiment of the data processing device 20 provided in the embodiment of the present application,

[0116] Optionally, based on the corresponding embodiment above, Figure 8 In another embodiment of the data processing device 20 provided in the embodiment of the present application,

[0117] A processing module 202 is configured to obtain the first storage cluster topology map;

[0118] Determine the detection result of the target storage pool according to the first storage cluster topology map;

[0119] When the detection result indicates that the flag information is set in the target storage pool, determine that the consistency rule of the target storage pool is to feedback to the client that the data write request is successfully executed when receiving response replies from M storage nodes;

[0120] When the detection result indicates that the flag information is not set in the target storage pool, determine that the consistency rule of the target storage pool is to feedback to the client that the data write request is successfully executed when receiving response replies from N storage nodes.

[0121] In an embodiment of the present application, a data processing device is provided. By using the above device, the cluster topology map of the distributed storage system is utilized to enable each storage node to synchronously obtain the flag information, so that the storage node can know whether to perform weakly consistent storage, and further realize the fast and stable storage function of the distributed storage system.

[0122] Optionally, based on the corresponding embodiment above, Figure 8 In another embodiment of the data processing device 20 provided in the embodiment of the present application, the processing module 202 is configured to determine the target storage pool according to the first storage cluster topology map.

[0123] Detecting the attribute information of the target storage pool to obtain a detection result of the target storage pool, wherein the attribute information is used to indicate a consistency rule of the target storage pool.

[0124] In an embodiment of the present application, a data processing device is provided. The above device is used to determine the storage pool to which each OSD belongs using the spatial association information stored in the cluster topology diagram of the distributed storage system, and then detect whether the storage pool is set with the flag information to know whether weak consistency storage is performed, thereby realizing the fast and stable storage function of the distributed storage system.

[0125] Optionally, in the above Figure 8 On the basis of the corresponding embodiment, in another embodiment of the data processing device 20 provided in the embodiment of the present application, the sending module 203 sends the data write request to the primary storage node among the N storage nodes, so that the primary storage node performs the write operation of the target data, and the primary storage node is used to store the primary copy of the target data;

[0126] The master storage node is called to send the data write request to N-1 storage nodes, so that the N-1 storage nodes perform the write operation of the target data, and the N-1 storage nodes are used to store the slave copy of the target data.

[0127] In an embodiment of the present application, a data processing device is provided. Using the above device, the master storage node of the target data sends data synchronization information to the slave storage node of the target data, thereby reducing data interaction between the client and the distributed storage node, thereby saving network resources.

[0128] Optionally, in the above Figure 8 On the basis of the corresponding embodiment, in another embodiment of the data processing device 20 provided in the embodiment of the present application,

[0129] The M is greater than or equal to the minimum number of storage nodes available for service.

[0130] In an embodiment of the present application, a data processing device is provided. Using the above device, the M selects the minimum number of storage nodes whose services are available, which can effectively ensure that data failure recovery can be achieved under weak consistency rules.

[0131] Optionally, in the above Figure 8 On the basis of the corresponding embodiment, in another embodiment of the data processing device 20 provided in the embodiment of the present application, the processing module 202 is used to synchronize the data modification of the target data to the first faulty storage node using at least one storage node among the M storage nodes after there is a first faulty storage node among the N storage nodes and the first faulty storage node is restarted.

[0132] In an embodiment of the present application, a data processing device is provided. With the above device, since data modification is completed in the case of heterogeneous data storage pool consistency, when a failed storage node occurs and the failed storage node is restored, data recovery can be performed based on the stored data, thereby ensuring data consistency.

[0133] Optionally, in the above Figure 8 On the basis of the corresponding embodiment, in another embodiment of the data processing device 20 provided in the embodiment of the present application, the processing module 202 is used to discard the second faulty storage node when the response reply does not satisfy the consistency rule of the target storage pool and there is a second faulty storage node;

[0134] The sending module 203 is used to send the data write request to a first storage node, where the first storage node is a storage node other than the N storage nodes in the target storage pool.

[0135] In an embodiment of the present application, a data processing device is provided. With the above device, if a faulty storage node occurs when a data write operation is not completed, the faulty node can be directly discarded, and then a new storage node can be selected to implement data storage, which can effectively ensure business stability.

[0136] The data processing device provided in this application can be used in a server, see Figure 9 , Figure 9 : is a schematic diagram of a server structure provided in an embodiment of the present application. The server 300 may have relatively large differences due to different configurations or performances, and may include one or more central processing units (CPU) 322 (for example, one or more processors) and memory 332, and one or more storage media 330 (for example, one or more mass storage devices) storing application programs 342 or data 344. Among them, the memory 332 and the storage medium 330 can be short-term storage or permanent storage. The program stored in the storage medium 330 may include one or more modules (not shown in the figure), and each module may include a series of instruction operations on the server. Furthermore, the central processing unit 322 can be configured to communicate with the storage medium 330 to execute a series of instruction operations in the storage medium 330 on the server 300.

[0137] The server 300 may also include one or more power supplies 326, one or more wired or wireless network interfaces 350, one or more input and output interfaces 358, and / or one or more operating systems 341, such as Windows Server 2000. TM , Mac OS XTM , Unix TM , Linux TM , FreeBSD TM and so on.

[0138] The steps performed by the server in the above embodiments may be based on the Figure 9 server structure shown.

[0139] The data processing device provided in this application can be used in a terminal device. Please refer to Figure 10 . For the sake of convenience of description, only the parts related to the embodiments of this application are shown. For the specific technical details not disclosed, please refer to the method part of the embodiments of this application. In the embodiments of this application, a smart phone is taken as an example of the terminal device for illustration:

[0140] Figure 10 The block diagram of a part of the structure of a smart phone related to the terminal device provided in the embodiments of this application is shown. Referring to Figure 10 , the smart phone includes: a radio frequency (RF) circuit 410, a memory 420, an input unit 430, a display unit 440, a sensor 450, an audio circuit 460, a wireless fidelity (WiFi) module 470, a processor 480, and a power supply 490 and other components. Those skilled in the art can understand that Figure 10 the smart phone structure shown in [[ID]] does not limit the smart phone, and may include more or fewer components than shown in the figure, or combine some components, or arrange different components.

[0141] The following Figure 10 will specifically introduce each component of the smart phone:

[0142] The RF circuit 410 can be used for receiving and transmitting information or signals during communication. Specifically, after receiving the downlink information from the base station, it is sent to the processor 480 for processing. Additionally, the uplink data designed is sent to the base station. Generally, the RF circuit 410 includes, but is not limited to, antennas, at least one amplifier, a transceiver, a coupler, a low noise amplifier (LNA), a duplexer, etc. In addition, the RF circuit 410 can also communicate with the network and other devices through wireless communication. The above wireless communication can use any communication standard or protocol, including but not limited to the Global System of Mobile Communication (GSM), General Packet Radio Service (GPRS), Code Division Multiple Access (CDMA), Wideband Code Division Multiple Access (WCDMA), Long Term Evolution (LTE), email, Short Messaging Service (SMS), etc.

[0143] The memory 420 can be used to store software programs and modules. The processor 480 executes various functional applications and data processing of the smartphone by running the software programs and modules stored in the memory 420. The memory 420 mainly includes a program storage area and a data storage area. Among them, the program storage area can store the operating system, application programs required for at least one function (such as the sound playback function, image playback function, etc.); the data storage area can store data created according to the use of the smartphone (such as audio data, phone book, etc.). In addition, the memory 420 can include high-speed random access memory and can also include non-volatile memory, such as at least one magnetic disk storage device, a flash memory device, or other non-volatile solid-state storage devices.

[0144] The input unit 430 can be used to receive input numerical or character information and generate key signal inputs related to the user settings and function controls of the smart phone. Specifically, the input unit 430 can include a touch panel 431 and other input devices 432. The touch panel 431, also known as a touch screen, can collect touch operations of the user thereon or nearby (such as operations of the user using any suitable object or accessory such as a finger or a stylus on or near the touch panel 431), and drive corresponding connection devices according to a preset program. Optionally, the touch panel 431 can include two parts: a touch detection device and a touch controller. Among them, the touch detection device detects the touch orientation of the user, detects the signal brought by the touch operation, and transmits the signal to the touch controller; the touch controller receives the touch information from the touch detection device, converts it into contact coordinates, then sends it to the processor 480, and can receive and execute the commands sent by the processor 480. In addition, various types such as resistive, capacitive, infrared, and surface acoustic wave can be used to implement the touch panel 431. In addition to the touch panel 431, the input unit 430 can also include other input devices 432. Specifically, the other input devices 432 can include, but are not limited to, one or more of a physical keyboard, function keys (such as volume control keys, switch keys, etc.), a trackball, a mouse, a joystick, etc.

[0145] The display unit 440 can be used to display the information input by the user or the information provided to the user and various menus of the smart phone. The display unit 440 can include a display panel 441. Optionally, the display panel 441 can be configured in the form of a liquid crystal display (LCD), an organic light-emitting diode (OLED), etc. Further, the touch panel 431 can cover the display panel 441. When the touch panel 431 detects a touch operation thereon or nearby, it is transmitted to the processor 480 to determine the type of touch event. Subsequently, the processor 480 provides corresponding visual output on the display panel 441 according to the type of touch event. Although in Figure 10 it, the touch panel 431 and the display panel 441 are implemented as two independent components to realize the input and input functions of the smart phone, but in some embodiments, the touch panel 431 and the display panel 441 can be integrated to realize the input and output functions of the smart phone.

[0146] The smart phone may also include at least one sensor 450, such as a light sensor, a motion sensor, and other sensors. Specifically, the light sensor may include an ambient light sensor and a proximity sensor. Among them, the ambient light sensor can adjust the brightness of the display panel 441 according to the brightness of the ambient light, and the proximity sensor can turn off the display panel 441 and / or the backlight when the smart phone is moved to the ear. As a kind of motion sensor, the accelerometer sensor can detect the magnitude of acceleration in various directions (generally three axes), and can detect the magnitude and direction of gravity when stationary, and can be used for applications that identify the posture of the smart phone (such as horizontal and vertical screen switching, related games, magnetometer posture calibration), vibration recognition related functions (such as pedometer, tapping), etc.; as for other sensors such as gyroscopes, barometers, hygrometers, thermometers, and infrared sensors that the smart phone can also be configured with, they will not be elaborated here.

[0147] The audio circuit 460, the speaker 461, and the microphone 462 can provide an audio interface between the user and the smart phone. The audio circuit 460 can transmit the electrical signal converted from the received audio data to the speaker 461, and the speaker 461 converts it into a sound signal for output; on the other hand, the microphone 462 converts the collected sound signal into an electrical signal, which is received by the audio circuit 460 and then converted into audio data. After the audio data is output to the processor 480 for processing, it is sent through the RF circuit 410 to, for example, another smart phone, or the audio data is output to the memory 420 for further processing.

[0148] WiFi belongs to short - range wireless transmission technology. The smart phone can help users send and receive emails, browse the web, and access streaming media through the WiFi module 470, which provides users with wireless broadband Internet access. Although Figure 10 the WiFi module 470 is shown, it can be understood that it does not belong to an essential component of the smart phone and can be completely omitted within the scope of not changing the essence of the invention according to needs.

[0149] The processor 480 is the control center of the smart phone, connecting various parts of the entire smart phone using various interfaces and lines. By running or executing software programs and / or modules stored in the memory 420, and by calling the data stored in the memory 420, it executes various functions of the smart phone and processes data, thereby monitoring the smart phone as a whole. Optionally, the processor 480 may include one or more processing units; optionally, the processor 480 may integrate an application processor and a modem processor. Among them, the application processor mainly processes the operating system, user interface, and application programs, etc., and the modem processor mainly processes wireless communication. It can be understood that the above - mentioned modem processor may not be integrated into the processor 480 either.

[0150] The smart phone further includes a power supply 490 (such as a battery) for powering each component. Optionally, the power supply can be logically connected to the processor 480 through a power management system, so as to manage functions such as charging, discharging, and power consumption management through the power management system.

[0151] Although not shown, the smart phone may further include a camera, a Bluetooth module, etc., which will not be elaborated here.

[0152] In the above embodiments, the steps performed by the terminal device may be based on the Figure 10 shown terminal device structure.

[0153] An embodiment of the present application further provides a computer-readable storage medium, in which a computer program is stored. When it runs on a computer, it causes the computer to execute the methods described in the foregoing embodiments.

[0154] An embodiment of the present application further provides a computer program product including a program. When it runs on a computer, it causes the computer to execute the methods described in the foregoing embodiments.

[0155] Those skilled in the art can clearly understand that for the convenience and brevity of description, the specific working processes of the systems, devices, and units described above can refer to the corresponding processes in the foregoing method embodiments, which will not be elaborated here.

[0156] In several embodiments provided by the present application, it should be understood that the disclosed systems, devices, and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of the units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed mutual coupling or direct coupling or communication connection can be through some interfaces, and the indirect coupling or communication connection of the devices or units can be in electrical, mechanical or other forms.

[0157] The units described as separate components may or may not be physically separated. The components displayed as units may or may not be physical units, that is, they may be located in one place, or may be distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0158] In addition, in each embodiment of the present application, each functional unit may be integrated into a processing unit, or each unit may exist physically alone, or two or more units may be integrated into one unit. The above integrated unit may be implemented in the form of hardware or in the form of a software functional unit.

[0159] If the above integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it may be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, may be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in each embodiment of the present application. The foregoing storage medium includes: various media that can store program codes, such as USB flash drives, mobile hard disks, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical discs.

[0160] As described above, the above embodiments are only used to illustrate the technical solutions of the present application and are not intended to limit them. Although the present application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions described in the foregoing embodiments or perform equivalent replacements for some of the technical features. These modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of each embodiment of the present application.

Claims

1. A data processing method, characterized in that, it includes: Receiving a data write request sent by a client, the data write request being used to indicate data modification to N copies of target data, the N copies being respectively stored in N storage nodes of a target storage pool, N being a positive integer, and the target storage pool being included in a distributed storage system; Obtaining a consistency rule of the target storage pool, the consistency rule being used to indicate that when receiving response replies from M storage nodes, a successful execution of the data write request is fed back to the client, M being a positive integer less than or equal to N; Sending the data write request to the N storage nodes so that the N storage nodes perform a write operation on the target data; Counting the response replies fed back by the N storage nodes; When the response replies meet the consistency rule of the target storage pool, sending a response message to the client, the response message being used to indicate that the data write request is successfully executed.

2. The method according to claim 1, characterized in that, the method further includes: Receiving update information, the update information carrying flag information, the flag information being used to indicate that the consistency rule of the target storage pool is that when receiving response replies from M storage nodes, a successful execution of the data write request is fed back to the client, M being a positive integer less than N; Setting the flag information for the target storage pool to obtain a first storage cluster topology diagram after the distributed storage system is updated; Synchronizing the first storage cluster topology diagram to the storage nodes of the distributed storage system.

3. The method according to claim 2, characterized in that, the obtaining the consistency rule of the target storage pool includes: Obtaining the first storage cluster topology diagram; Determining a detection result of the target storage pool according to the first storage cluster topology diagram; When the detection result indicates that the flag information is set for the target storage pool, determining that the consistency rule of the target storage pool is that when receiving response replies from M storage nodes, a successful execution of the data write request is fed back to the client, M being a positive integer less than N; When the detection result indicates that the flag information is not set for the target storage pool, determining that the consistency rule of the target storage pool is that when receiving response replies from N storage nodes, a successful execution of the data write request is fed back to the client.

4. The method according to claim 3, characterized in that, determining a detection result of the target storage pool according to the first storage cluster topology diagram includes: Determining the target storage pool according to the first storage cluster topology diagram; Detecting attribute information of the target storage pool to obtain a detection result of the target storage pool, the attribute information being used to indicate the consistency rule of the target storage pool.

5. The method according to any one of claims 1 to 4, characterized in that, sending the data write request to the N storage nodes so that the N storage nodes perform a write operation on the target data includes: Send the data write request to the primary storage node among the N storage nodes, so that the primary storage node performs the write operation of the target data, and the primary storage node is used to store the primary copy of the target data; Call the primary storage node to send the data write request to N-1 storage nodes, so that the N-1 storage nodes perform the write operation of the target data, and the N-1 storage nodes are used to store the secondary copies of the target data.

6. The method according to any one of claims 1 to 4, characterized in that, when M is greater than or equal to the minimum number of storage nodes with available services, and when the response reply meets the consistency rules of the target storage pool, after replying to the client with a response message, the method further includes: After there is a first failed storage node among the N storage nodes and the first failed storage node is restarted, use at least one of the M storage nodes to synchronize the data modification of the target data to the first failed storage node.

7. The method according to any one of claims 1 to 4, characterized in that, after counting the response replies fed back by the N storage nodes, the method further includes: When the response reply does not meet the consistency rules of the target storage pool and there is a second failed storage node, discard the second failed storage node.

8. A data processing device, characterized in that, comprising: a receiving module, configured to receive a data write request sent by a client, where the data write request is used to indicate data modification to N copies of target data, and the N copies are respectively stored in N storage nodes of a target storage pool, N is a positive integer, and the target storage pool is included in a distributed storage system; a processing module, configured to obtain the consistency rules of the target storage pool, where the consistency rules are used to indicate that when response replies returned by M storage nodes are received, it is fed back to the client that the data write request is successfully executed, and M is a positive integer less than or equal to N; a sending module, configured to send the data write request to the N storage nodes, so that the N storage nodes perform the write operation of the target data; the processing module, configured to count the response replies fed back by the N storage nodes; the sending module, configured to reply to the client with a response message when the response reply meets the consistency rules of the target storage pool, and the response message is used to indicate that the data write request is successfully executed.

9. A computer device, characterized in that, comprising: a memory, a processor, and a bus system; wherein, the memory is used to store programs; the processor is configured to execute the programs in the memory, and the processor is configured to execute the method according to any one of claims 1 to 7 according to the instructions in the program code; the bus system is configured to connect the memory and the processor, so that the memory and the processor communicate with each other.

10. A computer-readable storage medium, including instructions, when running on a computer, causes the computer to execute the method according to any one of claims 1 to 7.

Images