Erasure code data reading system, method, device and equipment and storage medium

Abstract

The invention relates to the technical field of data persistence storage, and discloses an erasure code data reading system, method, device and equipment and a storage medium, and the method comprises the steps that a proxy server receives a read-write request, and obtains a penalty score of each data node; calculating an accumulated punishment score of each data node in a preset time period, and sorting the accumulated punishment scores of each data node; screening to obtain a set of a preset number of data nodes with the minimum accumulated punishment score, and initiating a read-write request to each data node; when the data nodes of which the return requests are overtime or failed exist in the set of the data nodes, initiating read-write requests to all the other data nodes, and updating penalty scores of the data nodes of which the return requests are overtime or failed; and a preset number of successfully returned data nodes exist. According to the method, the health state of the data node is comprehensively known, the sub-health data node is prevented from being accessed, the overall reading request speed is prevented from being slowed down, and the overall stability and performance of the system are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of data persistence storage, and in particular to an erasure code data reading system, method, device, equipment and storage medium. Background Art

[0002] In distributed storage systems, erasure codes are often used for data redundancy and fault tolerance. Erasure codes encode data into n data shards and m check shards, which are stored in different nodes. When no more than m nodes fail, the erasure codes can be restored through calculation. In the erasure code mode (n data shards and m check shards), as long as at least n shards successfully return data, the original data can be restored.

[0003] In the related technology, under normal circumstances, the nodes where each data shard is located can return data normally, and we only need one round of requests to complete the reading. However, when a data shard suddenly becomes very slow (not uncommon in distributed environments, there are too many requests for the data shard, network failure, program execution failure, etc.), in this abnormal situation, the method can only wait for the read request of this "sub-healthy" data node to time out or return failure, and then initiate a new request to the node where the verification shard is located. In this way, it is necessary to wait for two rounds of requests to return (the first round requests data nodes, the second round requests verification nodes), which will slow down the entire data reading request. It takes some time to replace the "sub-healthy" data node (involving data migration) or wait for the node to recover by itself. During this period of time, all data reading requests will be affected and slowed down. Summary of the invention

[0004] In view of this, the present invention provides an erasure code data reading system, method, apparatus, device and storage medium to solve the problem that data reading requests will be affected and slowed down under abnormal circumstances.

[0005] In a first aspect, the present invention provides an erasure code data reading system, the system comprising: a data node and a proxy server:

[0006] Data nodes, used to store data shards and verify shards;

[0007] A proxy server is used to receive read and write requests and obtain the penalty score of each data node; calculate the cumulative penalty score of each data node within a preset time period, and sort the cumulative penalty score of each data node; screen and obtain a set of first data nodes with the smallest cumulative penalty score of a first preset number, and initiate read and write requests to each first data node; when there is a first data node in the set of first data nodes that returns a request timeout or a request failure, initiate read and write requests to all other data nodes, and based on a preset penalty strategy, update the penalty score of the first data node that returns a request timeout or a request failure; until there are a first preset number of data nodes that successfully return, restore the complete data corresponding to the read and write request.

[0008] In the present invention, the actual performance of all data nodes is scored through the proxy server Proxy to fully understand the health status of the data nodes. Within a certain period of time, a dynamic penalty strategy is adopted for nodes whose requests time out or return failures, and corresponding penalty scores are added to them to reflect their sub-healthy status. Each time data is read, the top n nodes with the smallest penalty scores are selected according to the scores of the data nodes, so as to select relatively healthy data nodes for data reading operations as much as possible, prevent access to "sub-healthy" data nodes, avoid slowing down the overall read request speed, and improve the overall stability and performance of the system. It realizes instant adjustment according to the changes in the health status of the nodes, and can achieve smarter and more efficient data reading in the erasure code mode, which has obvious advantages and can be applied to various business scenarios, improving reading performance and system stability.

[0009] In an optional implementation, the proxy server is further configured to determine whether the penalty score of the current data node exceeds a preset period; when the penalty score of the current data node exceeds the preset period, the penalty score of the current data node is cleared to zero.

[0010] In this method, the proxy server clears the penalty score of the data node at regular intervals to prevent the need to perform erasure code calculations to restore data and consume CPU. If all data nodes return success, there is no need to perform erasure code calculations.

[0011] In an optional implementation, the proxy server is further configured to return a data read failure to the upper service request layer when there are a second preset number of data nodes whose return requests have timed out or whose requests have failed.

[0012] In this manner, when there are a second preset number of data nodes that return request timeouts or request failures, the proxy server determines that the data read has failed and returns the data read failure to the upload service request layer. There is no need to perform erasure code calculations on all data nodes, which reduces the amount of erasure code calculations and further improves the reading performance and system stability.

[0013] In a second aspect, the present invention provides a method for reading erasure coded data, which is applied to an erasure coded data reading system as described in any one of the first aspects, the system comprising: a data node and a proxy server, and the method comprising:

[0014] Receive read and write requests and obtain the penalty score of each data node;

[0015] Calculate the cumulative penalty score of each data node within a preset period of time, and sort the cumulative penalty score of each data node;

[0016] A set of first data nodes with the smallest cumulative penalty score is obtained by screening, and a read / write request is initiated to each first data node;

[0017] When there is a first data node in the set of first data nodes that returns a request timeout or a request failure, a read / write request is initiated to all other data nodes, and based on a preset penalty strategy, a penalty score of the first data node that returns a request timeout or a request failure is updated;

[0018] Until there are a first preset number of data nodes that have successfully returned, the complete data corresponding to the read / write request is restored.

[0019] In the present invention, the actual performance of all data nodes is scored to fully understand the health status of the data nodes. Within a certain period of time, a dynamic penalty strategy is adopted for nodes with request timeout or return failure, and the corresponding penalty score is added to reflect its sub-health status. Each time data is read, the top n nodes with the smallest penalty score are selected according to the score of the data node, so as to select relatively healthy data nodes for data reading operations as much as possible, prevent access to "sub-healthy" data nodes, avoid slowing down the overall read request speed, and improve the overall stability and performance of the system. It realizes instant adjustment according to the changes in the health status of the node, and can achieve smarter and more efficient data reading in the erasure code mode, which has obvious advantages and can be applied to various business scenarios, improving reading performance and system stability.

[0020] In an optional implementation, before calculating the cumulative penalty score of each data node within a preset period and sorting the cumulative penalty score of each data node, the method further includes:

[0021] Determine whether the penalty score of the current data node exceeds the preset period;

[0022] When the penalty score of the current data node exceeds a preset period, the penalty score of the current data node is cleared.

[0023] In this method, the proxy server clears the penalty score of the data node at regular intervals to prevent the need to perform erasure code calculations to restore data and consume CPU. If all data nodes return success, there is no need to perform erasure code calculations.

[0024] In an optional embodiment, the method further includes:

[0025] When there are a second preset number of data nodes whose return requests time out or whose requests fail, a data reading failure is returned to the upper service request layer.

[0026] In this method, when there are a second preset number of data nodes that return request timeouts or request failures, the proxy server determines that the data read failure has occurred, and returns the data read failure to the upload business request layer. There is no need to perform erasure code calculations on all data nodes, which reduces the amount of erasure code calculations and further improves the reading performance and system stability.

[0027] In a third aspect, the present invention provides an erasure code data reading device, the device comprising:

[0028] The request receiving module is used to receive read and write requests and obtain the penalty score of each data node;

[0029] The node sorting module is used to calculate the cumulative penalty score of each data node within a preset period of time and sort the cumulative penalty score of each data node;

[0030] A node screening module, used for screening a set of first data nodes with the smallest cumulative penalty score of a first preset number, and initiating a read and write request to each first data node;

[0031] A score updating module is used to initiate a read / write request to all other data nodes when there is a first data node in the set of first data nodes that returns a request timeout or a request failure, and update the penalty score of the first data node that returns a request timeout or a request failure based on a preset penalty strategy;

[0032] The data recovery module is used to recover the complete data corresponding to the read and write request until there are a first preset number of successfully returned data nodes.

[0033] In a fourth aspect, the present invention provides a computer device, comprising: a memory and a processor, the memory and the processor are communicatively connected to each other, computer instructions are stored in the memory, and the processor executes the erasure code data reading method of the second aspect or any corresponding embodiment thereof by executing the computer instructions.

[0034] In a fifth aspect, the present invention provides a computer-readable storage medium having computer instructions stored thereon, the computer instructions being used to enable a computer to execute the erasure code data reading method of the second aspect or any corresponding embodiment thereof.

[0035] In a sixth aspect, the present invention provides a computer program product, comprising computer instructions, wherein the computer instructions are used to enable a computer to execute the method for reading erasure code data according to the second aspect or any corresponding embodiment thereof. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0037] Figure 1 is a schematic diagram of a conventional erasure code data reading method according to an embodiment of the present invention.

[0038] Figure 2 is a structural schematic diagram of an erasure code data reading system according to an embodiment of the present invention.

[0039] Figure 3 1 is a schematic diagram of the architecture of an erasure code data reading system based on a node scoring mechanism according to an embodiment of the present invention.

[0040] Figure 4 4 is a flow chart of a method for reading erasure coded data according to an embodiment of the present invention.

[0041] Figure 5 It is a flowchart of a method and system for reading erasure code data based on a node scoring mechanism according to an embodiment of the present invention.

[0042] Figure 6 It is a schematic diagram of an average time consumption experimental comparison between an embodiment of the present invention and an existing erasure code data reading method.

[0043] Figure 7 4 is a structural block diagram of an erasure code data reading device according to an embodiment of the present invention.

[0044] Figure 8 It is a schematic diagram of the hardware structure of a computer device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0045] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0046] In the related art, in distributed storage systems, erasure codes are often used for data redundancy backup and fault tolerance. Erasure codes encode data into n data shards and m check shards, which are stored in different nodes. When no more than m nodes fail, the erasure codes can be restored by calculation. In the erasure code mode (n data shards and m check shards), as long as at least n shards successfully return data, the original data can be restored. Figure 1 is a schematic diagram of an existing erasure code data reading method according to an embodiment of the present invention. Figure 1 As shown in the figure, the implementation of erasure code data reading in the industry is mainly as follows:

[0047] 1. Request all data shards first. If all data shards are returned successfully, the complete data can be pieced together and returned.

[0048] 2. If the request for the data shard fails, a read request is initiated to the verification shard. After the verification shard returns successfully, the original data is restored through calculation.

[0049] 3. If less than n of all data shards and checksum shards successfully return data, the data cannot be recovered through calculation and the read failure is returned.

[0050] Under normal circumstances, the nodes where each data shard is located can return data normally, and we only need one round of requests to complete the reading. However, when a data shard suddenly becomes very slow (not uncommon in distributed environments, too many requests for the data shard, network failure, program execution failure, etc.), in this abnormal situation, this method can only wait for the read request of this "sub-healthy" data node to time out or return failure, and then initiate a new request to the node where the verification shard is located. In this way, it is necessary to wait for two rounds of requests to return (the first round requests data nodes, the second round requests verification nodes), which will slow down the entire data reading request. It takes some time to replace the "sub-healthy" data node (involving data migration) or wait for the node to recover by itself. During this period of time, all data reading requests will be affected and slowed down.

[0051] To solve the above problems, an erasure code data reading system is provided in an embodiment of the present invention. The erasure code data reading system in this embodiment is suitable for the use scenario of realizing data reading in the erasure code mode in a distributed storage system. The erasure code data reading system provided by the present invention scores the actual performance of all data nodes through the proxy server Proxy, and fully understands the health status of the data nodes. Within a certain period of time, a dynamic penalty strategy is adopted for nodes with request timeout or return failure, and the corresponding penalty score is increased to reflect its sub-healthy state. Each time data is read, the top n nodes with the smallest penalty score are selected according to the score of the data node, so as to select relatively healthy data nodes as much as possible for data reading operations, prevent access to "sub-healthy" data nodes, avoid slowing down the overall reading request speed, and improve the overall stability and performance of the system. It realizes instant adjustment according to the change of the health status of the node, and can realize smarter and more efficient data reading in the erasure code mode, which has obvious advantages and can be applied to various business scenarios, improving reading performance and system stability.

[0052] According to an embodiment of the present invention, a system for reading erasure code data is provided. Figure 2 is a schematic diagram of the structure of an erasure code data reading system according to an embodiment of the present invention. Figure 2 As shown, the system includes: data node 1 and proxy server 2: data node 1, used to store data shards and check shards; proxy server 2, used to receive read and write requests and obtain the penalty score of each data node 1; calculate the cumulative penalty score of each data node 1 within a preset time period, and sort the cumulative penalty score of each data node 1; screen to obtain a set of first data nodes with the smallest cumulative penalty score of a first preset number, and initiate a read and write request to each first data node; when there is a first data node in the set of first data nodes that returns a request timeout or a request failure, initiate a read and write request to all other data nodes 1, and based on a preset penalty strategy, update the penalty score of the first data node that returns a request timeout or a request failure; until there are a first preset number of data nodes 1 that successfully return, restore the complete data corresponding to the read and write request.

[0053] In an optional implementation, the proxy server 2 is further configured to determine whether the penalty score of the current data node 1 exceeds a preset period; when the penalty score of the current data node 1 exceeds the preset period, the penalty score of the current data node 1 is cleared.

[0054] In this method, the proxy server clears the penalty score of the data node at regular intervals to prevent the need to perform erasure code calculations to restore data and consume CPU. If all data nodes return success, there is no need to perform erasure code calculations.

[0055] In an optional implementation, the proxy server 2 is further configured to return a data read failure to the upper service request layer when there are a second preset number of data nodes 1 that return a request timeout or a request failure.

[0056] In this manner, when there are a second preset number of data nodes that return request timeouts or request failures, the proxy server determines that the data read has failed and returns the data read failure to the upload service request layer. There is no need to perform erasure code calculations on all data nodes, which reduces the amount of erasure code calculations and further improves the reading performance and system stability.

[0057] In one example, Figure 3 is a schematic diagram of the architecture of an erasure code data reading system based on a node scoring mechanism according to an embodiment of the present invention. Figure 3 As shown, the erasure code data reading system based on the node scoring mechanism includes: Proxy2: receiving upper-layer business read and write requests, and sending concurrent requests to data nodes; Data node 1: storing data shards and verification shards. When Proxy2 receives a read request from the upper-layer business, it will first sort the nodes according to the cumulative penalty scores during the current period, select the first n data nodes 1 with the lowest penalty scores and concurrently initiate read requests. If a data node 1 returns a request timeout or request failure, it will continue to initiate requests to the remaining data nodes 1 until n nodes have successfully returned and then the complete data is restored by calculation. Otherwise, the data is insufficient and cannot be restored by calculation, and only the reading failure can be returned. According to the pre-established penalty mechanism, the data node 1 whose request is not successfully returned is penalized and scored. The penalty points of data node 1 will be cleared at regular intervals. According to the actual situation of the business, the penalty mechanism can be 1 point for 3 consecutive request timeouts, 1 point for 4 consecutive request failures, etc. The penalty mechanism is not limited in the present invention.

[0058] The erasure code data reading system provided in this embodiment scores the actual performance of all data nodes through the proxy server Proxy to fully understand the health status of the data nodes. Within a certain period of time, a dynamic penalty strategy is adopted for nodes whose requests time out or return failures, and corresponding penalty scores are added to them to reflect their sub-healthy status. Each time data is read, the top n nodes with the smallest penalty scores are selected according to the scores of the data nodes, so as to select relatively healthy data nodes for data reading operations as much as possible, prevent access to "sub-healthy" data nodes, avoid slowing down the overall reading request speed, and improve the overall stability and performance of the system. It realizes instant adjustment according to the changes in the health status of the nodes, and can achieve smarter and more efficient data reading in the erasure code mode. It has obvious advantages and can be applied to various business scenarios, improving reading performance and system stability.

[0059] According to an embodiment of the present invention, an embodiment of a method for reading erasure coded data is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0060] In this embodiment, a method for reading erasure code data is provided, which can be used in the above-mentioned erasure code data reading system. Figure 4 is a flowchart of a method for reading erasure coded data according to an embodiment of the present invention. Figure 4 As shown, the process includes the following steps:

[0061] Step S401, receiving a read / write request and obtaining a penalty score of each data node.

[0062] Step S402, determining whether the penalty score of the current data node exceeds a preset period.

[0063] Step S403: when the penalty score of the current data node exceeds a preset period, the penalty score of the current data node is cleared.

[0064] In this method, the proxy server clears the penalty score of the data node at regular intervals to prevent the need to perform erasure code calculations to restore data and consume CPU. If all data nodes return success, there is no need to perform erasure code calculations.

[0065] Step S404, calculating the cumulative penalty score of each data node within a preset time period, and sorting the cumulative penalty score of each data node.

[0066] Step S405 , screening out a set of first data nodes with the smallest cumulative penalty scores of a first preset number, and initiating a read / write request to each first data node.

[0067] Step S406, when there is a first data node in the set of first data nodes that returns a request timeout or a request failure, initiate a read / write request to all other data nodes, and based on a preset penalty strategy, update the penalty score of the first data node that returns a request timeout or a request failure.

[0068] Step S407, until there are a first preset number of data nodes that have successfully returned, restore the complete data corresponding to the read / write request.

[0069] Step S408: When there are a second preset number of data nodes whose return requests have timed out or whose requests have failed, a data read failure is returned to the upper service request layer.

[0070] In this method, when there are a second preset number of data nodes that return request timeouts or request failures, the proxy server determines that the data read failure has occurred, and returns the data read failure to the upload business request layer. There is no need to perform erasure code calculations on all data nodes, which reduces the amount of erasure code calculations and further improves the reading performance and system stability.

[0071] In one example, Figure 5 is a flow chart of a method and system for reading erasure code data based on a node scoring mechanism according to an embodiment of the present invention. Figure 5 As shown, the erasure code data reading method based on the node scoring mechanism includes: 1. The proxy receives a business read request.

[0072] 2. The proxy determines whether the current penalty points for the node have expired, and clears them if expired.

[0073] 3. The proxy sorts the nodes in ascending order according to the current penalty scores, selects the first n nodes with the smallest penalty scores and initiates read requests concurrently.

[0074] 4. If a node request times out or returns a read failure, the proxy will continue to send read requests to the remaining nodes and update the score of the failed node according to the strategy.

[0075] 5. Repeat step 4 until n nodes have successfully read and returned data, and then restore the complete data after calculation.

[0076] 6. If more than m nodes return read failure or timeout, data read failure is returned.

[0077] In one implementation scenario, the proxy uses a global map (key: node IP, value: penalty score) to store node scores and record the map creation time t. When a business read request comes in, if the current map creation time t exceeds the set threshold (for example, 1 minute, 2 minutes, etc.), the current map data will be cleared and the time t will be refreshed. Otherwise, the node scores stored in the map will be sorted in ascending order, and read requests will be initiated concurrently to the first n nodes with the lowest penalty scores. If the first n nodes return successfully, the calculation is performed to restore the complete data and return it. If there is a failure or request timeout in the first n nodes (the penalty score of the node in the map is refreshed according to the penalty rule), read requests will continue to be initiated to the nodes behind the sort until the termination condition is reached: either n nodes return successfully, or more than m nodes return failures.

[0078] Some distributed storages will uniformly send read and write requests to the leader node, which will then forward them to other nodes. Here, the scoring of the nodes can be done on the leader node.

[0079] Assume that there are 10 time slices, and an average of 100 requests are processed in each time slice. From the 3rd to the 8th time slice, a data node has read timeouts due to its own reasons (such as slow disk or network card). The average time taken for a normal read request is between 7 and 10 ms, and the timeout is set to 20 ms. Figure 6 : is a schematic diagram of an average time consumption experiment comparing an embodiment of the present invention with an existing erasure code data reading method. The experimental results are shown in the attached Figure 6 As shown, the proposed erasure code data reading method and system based on the node scoring mechanism is quite stable compared with the original method in terms of average time consumption, and the entire reading request is not slowed down due to the "sub-health" of a node.

[0080] The erasure code data reading method provided in this embodiment scores the actual performance of all data nodes to fully understand the health status of the data nodes. Within a certain period of time, a dynamic penalty strategy is adopted for nodes whose requests time out or return failures, and corresponding penalty scores are added to them to reflect their sub-healthy status. Each time data is read, the top n nodes with the smallest penalty scores are selected according to the scores of the data nodes, so as to select relatively healthy data nodes for data reading operations as much as possible, prevent access to "sub-healthy" data nodes, avoid slowing down the overall reading request speed, and improve the overall stability and performance of the system. It realizes instant adjustment according to the changes in the health status of the nodes, and can achieve smarter and more efficient data reading in the erasure code mode, which has obvious advantages and can be applied to various business scenarios, improving reading performance and system stability.

[0081] In this embodiment, an erasure code data reading device is also provided, which is used to implement the above-mentioned embodiments and preferred implementation modes, and will not be repeated here. As used below, the term "module" can implement a combination of software and / or hardware for a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.

[0082] This embodiment provides an erasure code data reading device, such as Figure 7 As shown, including:

[0083] The request receiving module 701 is used to receive read and write requests and obtain the penalty score of each data node. Figure 4 Step S401 of the illustrated embodiment will not be described in detail here.

[0084] The node sorting module 702 is used to calculate the cumulative penalty score of each data node within a preset period of time and sort the cumulative penalty score of each data node. Figure 4 Step S404 of the illustrated embodiment will not be described in detail here.

[0085] The node screening module 703 is used to screen and obtain a set of first data nodes with the smallest cumulative penalty score of a first preset number, and initiate a read and write request to each first data node. Figure 4 Step S405 of the illustrated embodiment will not be described in detail here.

[0086] The score updating module 704 is used to initiate read and write requests to all other data nodes when there is a first data node in the set of first data nodes that returns a request timeout or a request failure, and update the penalty score of the first data node that returns a request timeout or a request failure based on a preset penalty strategy. Figure 4 Step S406 of the illustrated embodiment will not be described in detail here.

[0087] The data recovery module 705 is used to recover the complete data corresponding to the read and write request until there are a first preset number of successfully returned data nodes. Figure 4 Step S407 of the illustrated embodiment will not be described in detail here.

[0088] In some optional implementations, the erasure code data reading device further includes:

[0089] The scoring period judgment unit is used to judge whether the penalty score of the current data node exceeds a preset period.

[0090] The score clearing unit is used to clear the penalty score of the current data node to zero when the penalty score of the current data node exceeds a preset period.

[0091] In some optional implementations, the erasure code data reading device further includes:

[0092] The read failure unit is used to return data read failure to the upper layer business request layer when there are a second preset number of data nodes that return request timeout or request failure.

[0093] The further functional description of each of the above modules and units is the same as that of the above corresponding embodiments and will not be repeated here.

[0094] The erasure code data reading device in this embodiment is presented in the form of a functional unit, where the unit refers to an ASIC (Application Specific Integrated Circuit) circuit, a processor and memory that executes one or more software or fixed programs, and / or other devices that can provide the above functions.

[0095] The embodiment of the present invention also provides a computer device having the above Figure 7 The erasure code data reading device shown.

[0096] See also Figure 8 , Figure 8 is a schematic diagram of the structure of a computer device provided by an optional embodiment of the present invention, such as Figure 8 As shown, the computer device includes: one or more processors 10, a memory 20, and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. Various components are connected to each other using different buses for communication, and can be installed on a common mainboard or installed in other ways as needed. The processor can process the instructions executed in the computer device, including instructions stored in or on the memory to display the graphical information of the GUI on an external input / output device (such as, a display device coupled to the interface). In some optional embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories and multiple memories. Similarly, multiple computer devices can be connected, and each device provides some necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system). Figure 8 A processor 10 is taken as an example.

[0097] The processor 10 may be a central processing unit, a network processor or a combination thereof. The processor 10 may further include a hardware chip. The hardware chip may be a dedicated integrated circuit, a programmable logic device or a combination thereof. The programmable logic device may be a complex programmable logic device, a field programmable gate array, a general purpose array logic or any combination thereof.

[0098] The memory 20 stores instructions executable by at least one processor 10, so that the at least one processor 10 executes the method shown in the above embodiment.

[0099] The memory 20 may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created according to the use of the computer device, etc. In addition, the memory 20 may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some optional embodiments, the memory 20 may optionally include a memory remotely arranged relative to the processor 10, and these remote memories may be connected to the computer device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0100] The memory 20 may include a volatile memory, such as a random access memory; the memory may also include a non-volatile memory, such as a flash memory, a hard disk or a solid state drive; the memory 20 may also include a combination of the above types of memory.

[0101] The computer device also includes an input device 30 and an output device 40. The processor 10, the memory 20, the input device 30 and the output device 40 may be connected via a bus or other means. Figure 8 The example of connecting through bus is taken in the following.

[0102] The input device 30 can receive input digital or character information, and generate key signal input related to the user settings and function control of the computer device, such as a touch screen, a keypad, a mouse, a track pad, a touch pad, an indicator bar, one or more mouse buttons, a trackball, a joystick, etc. The output device 40 may include a display device, an auxiliary lighting device (e.g., an LED) and a tactile feedback device (e.g., a vibration motor), etc. The above-mentioned display device includes but is not limited to a liquid crystal display, a light emitting diode, a display and a plasma display. In some optional embodiments, the display device can be a touch screen.

[0103] The embodiment of the present invention also provides a computer-readable storage medium. The method according to the embodiment of the present invention can be implemented in hardware, firmware, or can be implemented as a computer code that can be recorded in a storage medium, or can be implemented as a computer code that is originally stored in a remote storage medium or a non-temporary machine-readable storage medium and will be stored in a local storage medium through a network download, so that the method described herein can be stored in such software processing on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. Among them, the storage medium can be a magnetic disk, an optical disk, a read-only storage memory, a random access memory, a flash memory, a hard disk or a solid-state hard disk, etc.; further, the storage medium can also include a combination of the above types of memories. It can be understood that a computer, a processor, a microprocessor controller, or programmable hardware includes a storage component that can store or receive software or computer code. When the software or computer code is accessed and executed by a computer, a processor, or hardware, the method shown in the above embodiment is implemented.

[0104] A part of the present invention may be applied as a computer program product, such as a computer program instruction, which, when executed by a computer, can call or provide the method and / or technical solution according to the present invention through the operation of the computer. Those skilled in the art should understand that the existence of the computer program instruction in a computer-readable medium includes, but is not limited to, a source file, an executable file, an installation package file, etc., and accordingly, the way in which the computer program instruction is executed by the computer includes, but is not limited to: the computer directly executes the instruction, or the computer compiles the instruction and then executes the corresponding compiled program, or the computer reads and executes the instruction, or the computer reads and installs the instruction and then executes the corresponding installed program. Here, the computer-readable medium may be any available computer-readable storage medium or communication medium accessible to the computer.

[0105] Although the embodiments of the present invention have been described in conjunction with the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present invention, and such modifications and variations are all within the scope defined by the appended claims.

Claims

1. A system for reading erasure code data, characterized in that: The system includes: a data node and a proxy server: The data node is used to store data slices and check slices; The proxy server is used to receive read and write requests and obtain the penalty score of each of the data nodes; calculate the cumulative penalty score of each of the data nodes within a preset time period, and sort the cumulative penalty scores of each of the data nodes; screen and obtain a set of first data nodes with the smallest cumulative penalty scores of a first preset number, and initiate the read and write requests to each of the first data nodes; when there is a first data node in the set of the first data nodes that returns a request timeout or a request failure, initiate the read and write requests to all other data nodes, and based on a preset penalty strategy, update the penalty score of the first data node that returns a request timeout or a request failure; until there are a first preset number of data nodes that successfully return, restore the complete data corresponding to the read and write request.

2. The system according to claim 1, characterized in that The proxy server is also used to determine whether the penalty score of the current data node exceeds a preset period; when the penalty score of the current data node exceeds the preset period, the penalty score of the current data node is cleared to zero.

3. The system according to claim 1, characterized in that The proxy server is further configured to return a data reading failure to the upper service request layer when there are a second preset number of data nodes that return a request timeout or a request failure.

4. A method for reading erasure code data, characterized in that: Applied to the erasure code data reading system according to any one of claims 1 to 3, the system comprises: a data node and a proxy server, and the method comprises: Receive read and write requests, and obtain penalty scores of each of the data nodes; Calculating the cumulative penalty score of each of the data nodes within a preset time period, and sorting the cumulative penalty scores of each of the data nodes; Screening and obtaining a set of first data nodes with the smallest cumulative penalty score in a first preset number, and initiating the read and write request to each of the first data nodes; When there is a first data node in the set of the first data nodes that returns a request timeout or a request failure, the read / write request is initiated to all other data nodes, and based on a preset penalty strategy, the penalty score of the first data node that returns a request timeout or a request failure is updated; Until there are a first preset number of data nodes that have successfully returned, the complete data corresponding to the read / write request is restored.

5. The method according to claim 4, characterized in that Before calculating the accumulated penalty scores of the data nodes within the preset time period and sorting the accumulated penalty scores of the data nodes, the method further includes: Determine whether the penalty score of the current data node exceeds the preset period; When the penalty score of the current data node exceeds a preset period, the penalty score of the current data node is cleared.

6. The method according to claim 4, characterized in that The method further comprises: When there are a second preset number of data nodes whose return requests time out or whose requests fail, a data reading failure is returned to the upper service request layer.

7. A device for reading erasure code data, characterized in that: The device comprises: The request receiving module is used to receive read and write requests and obtain the penalty score of each data node; A node sorting module, used to calculate the cumulative penalty score of each of the data nodes within a preset time period, and sort the cumulative penalty score of each of the data nodes; A node screening module, used for screening a set of first data nodes with the smallest cumulative penalty score of a first preset number, and initiating the read and write request to each of the first data nodes; A score updating module, configured to initiate the read / write request to all other data nodes when there is a first data node in the set of the first data nodes that returns a timeout or a request failure, and update the penalty score of the first data node that returns a timeout or a request failure based on a preset penalty strategy; The data recovery module is used to recover the complete data corresponding to the read and write request until a first preset number of data nodes return successfully.

8. A computer device, characterized in that: include: A memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the erasure code data reading method according to any one of claims 4 to 6 by executing the computer instructions.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a computer to execute the erasure code data reading method according to any one of claims 4 to 6.

10. A computer program product, characterized in that The method comprises computer instructions, wherein the computer instructions are used to enable a computer to execute the erasure code data reading method according to any one of claims 4 to 6.

Images