Encoding method, electronic device, and program product

Abstract

Embodiments of the present disclosure relate to an encoding method, an electronic device and a computer program product. The method comprises: receiving configuration data related to encoding using a predetermined encoding scheme; determining an encoding strategy based on the configuration data, the encoding strategy comprising parameters associated with encoding data on an entity; and causing the data to be encoded on the entity based on the encoding strategy. Using the technical solution of the present disclosure, higher flexibility and reliability can be provided when encoding data, so it is particularly suitable for the field of data protection, and the user experience of users with data encoding and data protection needs can be improved by encoding and protecting the data of users with lower overhead.

Description

TECHNICAL FIELD

[0001] Embodiments of the present disclosure generally relate to computer technology, and in particular, to an encoding method, an electronic device and a computer program product, which can be used in the field of data processing and the field of data protection, etc. BACKGROUND

[0002] Modern distributed storage systems have integrated erasure coding technology to achieve data protection purposes. Erasure coding technology divides data into multiple segments, uses redundant data segments to expand and encode data, and stores them in a set of different locations or storage media. Although the implementation of distributed storage systems varies, generally speaking, these distributed storage systems will set and use rules associated with erasure coding to protect data based on the unit of logical partitioning.

[0003] In a dynamic and resource-limited edge network environment, erasure coding technology plays an important role in edge storage systems due to its ability to achieve a balance between storage and computing overhead in storage systems. However, the storage system of the conventional technology cannot meet various requirements for efficient use of erasure coding technology, and in particular, cannot meet the requirements for efficient use of erasure coding technology in edge network solutions. For example, data generated by many users needs to be configured based on specific attributes of the data, rather than using a fixed database such as a directory. In addition, the scarcity of computing and storage resources on edge nodes also requires more flexible erasure coding strategies while maintaining data reliability. SUMMARY

[0004] Embodiments of the present disclosure provide an encoding method, an electronic device and a computer program product.

[0005] In a first aspect of the present disclosure, an encoding method is provided. The method comprises: receiving configuration data related to encoding using a predetermined encoding manner; determining an encoding strategy based on the configuration data, the encoding strategy comprising parameters associated with encoding data on an entity; and causing encoding of the data on the entity based on the encoding strategy.

[0006] In a second aspect of the present disclosure, an electronic device is provided. The electronic device comprises: at least one processing unit; and at least one memory coupled to the at least one processing unit and storing instructions for execution by the at least one processing unit, the instructions, when executed by the at least one processing unit, causing the device to perform actions comprising: receiving configuration data related to encoding using a predetermined encoding manner; determining an encoding strategy based on the configuration data, the encoding strategy comprising parameters associated with encoding data on an entity; and causing encoding of the data on the entity based on the encoding strategy.

[0007] In a third aspect of the present disclosure, a computer program product is provided. The computer program product is tangibly stored on a non-transitory computer readable medium and comprises machine executable instructions that, when executed, cause a machine to perform any of the steps of the method described according to the first aspect of the present disclosure.

[0008] The summary is provided to introduce a selection of concepts, in a simplified form, that are further described below in the detailed description. This summary is not intended to identify key features or essential features of embodiments of the disclosure, nor is it intended to limit the scope of embodiments of the disclosure. BRIEF DESCRIPTION OF DRAWINGS

[0009] The above and other objects, features and advantages of the present disclosure will become more apparent from the following detailed description when taken in conjunction with the accompanying drawings in which like reference characters refer to like parts throughout the figures.

[0010] Figure 1 A schematic diagram of an encoding environment 100 in which apparatus and / or methods according to embodiments of the present disclosure can be implemented is shown;

[0011] Figure 2 A flowchart of an encoding method 200 according to embodiments of the present disclosure is shown;

[0012] Figure 3 A flowchart of an encoding method 300 according to embodiments of the present disclosure is shown; and

[0013] Figure 4 A schematic block diagram of an example apparatus 400 that can be used to implement embodiments of the present disclosure is shown.

[0014] In the various drawings, like or corresponding elements are denoted by like or corresponding reference numbers. DETAILED DESCRIPTION

[0015] Preferred embodiments of the present disclosure will be described herein below with reference to the drawings. While the preferred embodiments of the present disclosure are shown in the drawings, it is understood that the present disclosure can be embodied in various forms without being limited by the embodiments set forth herein. Rather, these embodiments are provided so that the present disclosure will be thorough and complete, and will fully convey the scope of the present disclosure to those skilled in the art.

[0016] The term "includes," along with its variations, is used to mean "includes but is not limited to." The term "or" is used to mean "and / or" unless otherwise noted. The term "based on" means "based at least in part on." The term "one example embodiment" and "an embodiment" are used interchangeably. The term "another embodiment" means at least one additional embodiment. The terms "first," "second," and the like can refer to different or the same objects. Also, the terms "exemplary" and "example" mean "an example of" and are not necessarily to be construed as preferred or advantageous over other examples. The terms "plurality" and "a plurality" mean "two or more than two." The term "another" means at least one additional or

[0017] Encoding techniques, such as erasure coding techniques, are an effective method for data protection in storage systems. In the implementation of erasure coding techniques, data is broken into shards and expanded and encoded with redundant data shards for storage in a set of different locations or storage media.

[0018] Although data replication can be performed for the purpose of protecting data, modern distributed storage systems have integrated the implementation of erasure coding techniques into various systems. For example, the Hadoop Distributed File System can support erasure coding techniques such as RS erasure coding algorithm and XOR erasure coding algorithm based on directories, while the Ceph Distributed File System can statically support RS, Cauchy, and other erasure coding techniques based on logical partitioning of objects.

[0019] Erasure coding techniques are also important in edge network architecture. In edge network architecture, edge nodes with storage and computing functions control end users within their range and at the same time communicate with cloud services. In fact, due to the deployment scale, hardware functions, and different functions related to inter-system management and communication of edge nodes, edge nodes can be divided into multiple layers. Data generated by end users will be uploaded to the edge or cloud for storage and query, but some edge devices that save data are in a less restricted environment, which brings higher data unavailability compared to traditional distributed storage systems. Data replication can be a solution to this case, but the limited storage capacity of nodes in the edge network structure limits the utilization of data replication. Therefore, it is generally recognized that erasure coding, as a balance between storage and computing overhead in storage systems, is essential for data protection in edge scenarios.

[0020] However, current distributed storage systems mainly rely on system administrators to configure data protection strategies, so it is difficult to dynamically meet various requirements from the perspective of edge infrastructure and customers.

[0021] For example, in current distributed storage systems, erasure code algorithms are statically configured, so they cannot be conveniently changed at runtime. For example, in the Hadoop Distributed File System, management commands are provided to first add an erasure code policy based on an xml file, and then set the required erasure code policy to a certain directory. If the erasure code policy changes due to some business logic, the programmer must manipulate these management commands in the source code, and this privilege may not be granted in the system design of every application. When developing distributed applications, especially in a super large scale environment, this will be an obstacle.

[0022] For another example, in current distributed storage systems, erasure code policies are set based on directories or pools. This is not very detailed for use cases in edge scenarios. In such a highly dynamic environment, data stored in a logical unit can have various data protection requirements. There can be data generated from different sensors, or data from various third parties, which are used for the same big data processing purpose and have multiple erasure code policies. In addition, sometimes it is necessary to separate certain data from the same source into hot data and cold data directories, which formulate strategies from the perspective of data acquisition efficiency, but all these extreme cases increase the conflict between existing erasure code configuration methods and user needs.

[0023] For another example, in current distributed storage systems, since current distributed storage systems are deployed in relatively homogeneous and stable hardware, they do not take into account many factors related to hardware resource limitations. However, in edge architecture, nodes that make up the cluster are highly heterogeneous in terms of computing and storage capabilities, and their health status is unstable due to low-limit environments. In this case, there are new challenges for automatically adjusting erasure code policies to make the best effort to protect data.

[0024] To at least partially solve one or more of the above problems and other potential problems, embodiments of the present disclosure propose an encoding method that proposes a user-oriented erasure encoding solution on the basis of an edge network to solve the above highlighted problems. It should be noted that this method is not limited to edge networks, but can also be applied to data centers or cloud environments. Using the encoding method according to the embodiments of the present disclosure, higher flexibility and reliability can be provided when encoding data, so it is particularly suitable for use in the field of data protection, and can improve the user experience of users with data encoding and data protection needs by encoding and protecting the data of users with lower overhead.

[0025] Figure 1A schematic block diagram of an encoding environment 100 in which an encoding method in certain embodiments of the present disclosure can be implemented is shown. According to embodiments of the present disclosure, the encoding environment 100 can be a cloud environment.

[0026] As shown in Figure 1 According to embodiments of the present disclosure, the input data 120 may, for example, comprise configuration data related to encoding data using a predetermined encoding scheme such as a erasure code.

[0027] Further included in the encoding environment 100 are a computing device 110, an entity 140 and an entity 150, wherein the computing device 110 can also be considered as another entity, and the computing device 110, the entity 140 and the entity 150 respectively comprise an encoding executor 111, an encoding executor 141 and an encoding executor 151. It should be understood that the inclusion of two entities 140 and 150 in the encoding environment 100 is merely for example, and the scope of protection of embodiments of the present disclosure is not limited thereto. In fact, a large number of entities can be included in the encoding environment 100. As shown in Figure 1 As shown in

[0028] According to embodiments of the present disclosure, the input data 120 can also comprise to-be-encoded data, in which case the entity 140 can provide the obtained to-be-encoded data to the computing device 110, and the computing device 110 can provide the to-be-encoded data obtained directly or through the entity 140 to the entity 150.

[0029] According to embodiments of the present disclosure, since the computing device 110, the entity 140 and the entity 150 respectively comprise the encoding executor 111, the encoding executor 141 and the encoding executor 151, encoding the to-be-encoded data can be performed by at least one of the encoding executor 111, the encoding executor 141 and the encoding executor 151 on at least one of the computing device 110, the entity 140 and the entity 150.

[0030] According to an embodiment of the present disclosure, the to-be-encoded data can not be provided by the user 130, but can have been pre-stored in the computing device 110, the entity 140, the entity 150 or other storage locations in the encoding environment 100. At this time, when the to-be-encoded data needs to be encoded by at least one of the encoding executors 111, 141 and 151 on at least one of the computing device 110, the entity 140 and the entity 150, the to-be-encoded data stored on itself can be directly encoded by at least one of the computing device 110, the entity 140 and the entity 150 that will perform the encoding, or the to-be-encoded data can be obtained from other storage locations for encoding.

[0031] According to an embodiment of the present disclosure, when the computing device 110 receives the configuration data related to encoding data using a predetermined encoding mode such as erasure code, it can determine the encoding strategy based on the configuration data through the policy engine (not shown) included therein. In addition, the computing device 110 can further include a health state information monitor (not shown) for obtaining the health state of the computing device 110, the entity 140 and the entity 150, so that the encoding strategy can be further determined based on the health state.

[0032] According to an embodiment of the present disclosure, the computing device 110 and the entities 140 and 150 can be distinguished in that only the computing device 110 has the ability to determine the encoding strategy and obtain the health state. However, based on the design of different edge storage systems or the data scale in the architecture, the entities 140 and 150 can also have the above-mentioned capabilities, so that if the end user generates a large amount of data, each of the entities such as the entity 140 and the entity 150 that the end user can connect to can actually act as a computing device, thereby reducing the data storage and erasure code calculation burden of the system.

[0033] In Figure 1 In the encoding environment 100 shown, the data interaction between the user 130, the computing device 110, the entity 140, the entity 150 and other components that can exist in the computing environment 100 can be carried out through a network. In addition, in an embodiment of the present disclosure, the computing device 110 is taken as an example for illustration, and the computing device 110 can include any entity or virtual central processing unit, special processing unit and special accelerator, etc. However, the protection scope of the present disclosure is not limited thereto, but can be applied to various computing elements, units, modules or systems, etc. that can provide input and output processing capabilities.

[0034] It should be appreciated that the encoding environment 100 is merely exemplary and not limiting, and it is scalable or shrinkable. For example, more users 130 can be included in the encoding environment 100, more input data 120 provided by more users, more computing devices 110, and more entities, such that the needs of more users can be met simultaneously with more computing devices 110 and more entities to process more input data 120 simultaneously or non-simultaneously.

[0035] The encoding method 200 and the encoding method 300 shown in Figure 1 are described below with the computing devices 110 and the input data 120 included in Figures 2 to 4 as examples.

[0036] Figure 2 A flowchart of the encoding method 200 according to an embodiment of the present disclosure is shown. The method 200 can be implemented by the computing devices 110 shown in Figure 1 , or by other suitable devices. It should be appreciated that the encoding method 200 can also include additional steps not shown and / or the steps shown can be omitted, and the scope of the embodiments of the present disclosure is not limited in this respect.

[0037] At block 202, the computing devices 110 receive configuration data included in the input data 120 related to encoding using a predetermined encoding manner. According to an embodiment of the present disclosure, the predetermined encoding manner indicates encoding using erasure code.

[0038] According to an embodiment of the present disclosure, the configuration data can be static configuration data or dynamic configuration data. The static configuration data can indicate information related to data blocks and check blocks to be encoded using the predetermined encoding manner, and indicate entities. Specifically, the static configuration data can indicate on which entities the encoding is to be performed, for example, the encoding of data is to be performed on one or more of the computing devices 110, the entities 140, and the entities 150 shown in Figure 1 . In addition, the static data also directly indicates information related to data blocks and check blocks to be encoded using the predetermined encoding manner, for example, the number of data blocks and the number of check blocks to be encoded using the predetermined encoding manner. In addition, the static configuration data can indicate the kind of encoding to be used and the type of data to be encoded, such that it can be directly known from the static encoding data on which entities, in which encoding manner, and of which type of data the encoding is to be performed. Therefore, the static configuration data can be suitable for user critical data with strict erasure code requirements, such that the encoding system can always ensure that it meets the requirements.

[0039] The dynamic configuration data can indicate a priority of data protection, a ratio associated with a number of data blocks and a number of check blocks, and a limitation such as operating temperature of an entity on which data is to be encoded, so that an encoding strategy can be dynamically determined based on these instructive data, e.g., a number of data blocks and a number of check blocks to be encoded using a predetermined encoding scheme. Thus, the dynamic configuration data can be applicable to large data sets such as artificial intelligence training, and can support a infrastructure provider to charge different fees to users according to their configurations, similar to a “pay-as-you-go” approach in cloud services, so that users can decide their own configuration strategy according to their business needs.

[0040] At block 204, the computing device 110 determines an encoding strategy based on the configuration data received at block 402. According to embodiments of the present disclosure, the encoding strategy includes parameters associated with encoding data on an entity, and the parameters can indicate at least one of: a source of the data, a type of the data, a priority of protection of the data, a number of data blocks to be encoded using a predetermined encoding scheme, a number of check blocks to be encoded using the predetermined encoding scheme, a ratio associated with the number of data blocks and the number of check blocks, a kind of encoding, and the entity on which the data is to be encoded.

[0041] According to embodiments of the present disclosure, as previously mentioned, when the configuration data is static configuration data, the computing device 110 can read the parameters associated with encoding data on an entity from the static configuration data, e.g., in the form of direct reading, and form a corresponding encoding strategy. When the configuration data is dynamic configuration data, the computing device 110 can dynamically determine the encoding strategy based on the instructiveness of the dynamic configuration data. Determining the encoding strategy based on the configuration data can be specifically implemented by a strategy engine included in the computing device 110.

[0042] According to embodiments of the present disclosure, the computing device 110 determining the encoding strategy based on the configuration data can further include the computing device 110 obtaining health status information of a set of entities available for encoding data thereon, e.g., by a health status information monitor included therein, and then determining the encoding strategy based on the configuration data and the health status information. The set of entities available for encoding data thereon can include the computing device 110 itself and other entities, e.g., the entity 140 and the entity 150 shown in FIG. 1. Figure 1

[0043] ​According to embodiments of the present disclosure, the health status information of the set of entities available to the computing device 110 for encoding data thereon can indicate at least one of: environmental information of the available entities, such as the ambient temperature, humidity of the entities, which can affect the service life and failure rate of the entities; health status associated with the computing units in the available entities, such as the remaining service life, possibility of failure, etc.; health status associated with the storage units in the available entities, such as the remaining service life, whether there is a bad track, possibility of failure, etc.; utilization rate of the computing units in the available entities; and utilization rate of the storage units in the available entities.

[0044] According to embodiments of the present disclosure, after obtaining the health status information of the set of entities, the computing device 110 can perform the encoding operation using the entities with higher health as much as possible by determining the encoding strategy.

[0045] According to embodiments of the present disclosure, in order to avoid the oscillation of the state data constantly changing the strategy, after collecting the health status information about the computing units such as central processing units, memories, storage capacities or other hardware, the computing device 110 can process the health status information using a filtering or smoothing algorithm, and then determine the encoding strategy based on the configuration data and the health status information.

[0046] According to embodiments of the present disclosure, in determining the encoding strategy, the computing device can use various different algorithms, such as the algorithm shown in the following formula (1).

[0047]

[0048] In formula (1), the protection ratio can refer to the ratio associated with the number of data blocks and the number of check blocks. Formula (1) is directed to the highly dynamic and heterogeneous properties of the edge network structure, and the erasure code complexity finally determined according to formula (1) is proportional to the available storage space and the protection ratio, and inversely proportional to the working load of the set of entities and the data priority set by the user. In determining the encoding strategy, the erasure code complexity can be provided with an upper threshold, so that by adjusting the various parameters in the formula, the erasure code complexity can be prevented from exceeding the upper threshold, thereby avoiding excessive consumption of system resources.

[0049] According to embodiments of the present disclosure, when the input data 120 is first provided to the entity 140, if the computing resources of the computing device 100 are too limited to meet the requirement of encoding, or if it is very necessary to ensure the availability of data from the input data 120 is provided to the entity 140, the computing device can instruct the encoding executor in the entity 140 to encode the data to be encoded, which can be referred to as an online erasure code scheme. Otherwise, the data to be encoded can be encoded by the computing device 100, which can be referred to as an offline erasure code scheme. In addition, other factors in the erasure code or strategy algorithm can also be considered when determining the encoding strategy, such as the placement of the check block based on the state of the entity in the entity set, and whether it can be easily extended according to the industrial scene, etc.

[0050] At block 206, the computing device 110 causes the data to be encoded based on the encoding strategy on the entity on which the data encoding is to be performed as indicated by the encoding strategy. According to embodiments of the present disclosure, when the encoding strategy indicates that the entity on which the data encoding is to be performed is another entity other than the computing device 110, such as the entity 140 or the entity 150, the computing device 110 can send the encoding strategy to the entity on which the data encoding is to be performed, so that the entity receiving the encoding strategy can use the encoding executor included therein to encode the data to be encoded. In addition, when the data to be encoded indicated by the encoding strategy is not stored on the entity on which the data encoding is to be performed, the computing device 110 can instruct the entity to obtain the data to be encoded by sending instructions to receive the data to be encoded from the user 130 or read the data to be encoded from other data storage locations, etc.

[0051] According to embodiments of the present disclosure, after the computing device 110 determines the encoding strategy based on the configuration data or causes the data to be encoded on the entity on which the data encoding is to be performed based on the encoding strategy, the computing device 110 can store the parameters in the form of metadata, so that subsequent operations such as querying can be performed on the encoded data. For example, when a user needs to query some data later, the computing device 110 can first find the metadata corresponding to the data, and then find the location of each data block from the hash value of the metadata. If the reading of some data blocks fails, the computing device 110 can use the corresponding erasure code strategy and the check block to perform erasure code decoding and correction methods, so that the system can recover the data from the dynamic edge network environment.

[0052] Figure 3 A flowchart of an encoding method 300 according to embodiments of the present disclosure is shown. The method 300 can be performed by a computing device, such as the computing device 110, the computing device 120, the computing device 130, the computing device 140, the computing device 150, etc. Figure 1The computing device 110, and in particular the policy engine included in the computing device 110, shown in FIG. 1 can be implemented by other suitable devices. The encoding method 300 can be a particular description of the encoding method 200, and it can include additional steps not shown and / or can omit steps shown, the scope of embodiments of the present disclosure not being limited in this respect.

[0053] At block 302, the computing device 110 determines whether the configuration data indicates an entity on which data is to be encoded. If the configuration data indicates an entity on which data is to be encoded, the method 300 proceeds to block 304, otherwise the method 300 proceeds to block 308.

[0054] At block 304, the computing device 110 determines whether the entity on which data is to be encoded, indicated in the configuration data, is a non-computing device 110 entity, such as the entity 140 or the entity 150 shown in FIG. 1. If the entity on which data is to be encoded is a non-computing device 110 entity, the method 300 proceeds to block 306, otherwise the method 300 proceeds to block 314. Figure 1

[0055] At block 306, the computing device 110 sends an encoding policy to the entity on which data is to be encoded for the entity on which data is to be encoded to encode the data to be encoded based on the encoding policy.

[0056] At block 308, the computing device 110 obtains health status information for a set of entities on which data can be encoded, such as by a health status information monitor included therein.

[0057] At block 310, the computing device 110 determines an encoding policy based on the configuration data and the health status information.

[0058] At block 312, the computing device 112 determines whether the configuration data indicates that the data to be encoded needs to be encoded on a non-computing device 110 entity, such as the entity 140 or the entity 150 shown in FIG. 1. If the data to be encoded needs to be encoded on a non-computing device 110 entity, the method 300 proceeds to block 306, otherwise the method 300 proceeds to block 314. Figure 1

[0059] At block 314, the computing device 110 encodes the data to be encoded thereon. It will be appreciated that the method 300 is directed to the case where encoding will be performed on one of the computing device 110, the entity 140, and the entity 150. Thus, when the configuration data indicates that the data to be encoded needs to be encoded on a non-computing device 110 entity, then the data to be encoded can be encoded directly on that entity, otherwise the data to be encoded needs to be encoded on the computing device 110.​​

[0060] The above reference Figures 1 to 3 The relevant content of the encoding environment 100 in which the device and / or method according to embodiments of the present disclosure can be implemented, the encoding method 200 according to embodiments of the present disclosure, and the encoding method 300 according to embodiments of the present disclosure is described. It should be understood that the above description is to better demonstrate the content recorded in the embodiments of the present disclosure, and is not limited in any way.

[0061] It should be understood that the number of various elements and the size of physical quantities employed in the embodiments of the present disclosure and in the various drawings are only examples and are not a limitation on the scope of protection of the embodiments of the present disclosure. The above number and size can be arbitrarily set as needed without affecting the normal implementation of the embodiments of the present disclosure.

[0062] Through the above reference Figures 1 to 3 The technical solution according to embodiments of the present disclosure proposes an encoding scheme, which is a comprehensive, user-oriented and cluster-aware erasure code solution, and intelligently determines the encoding strategy and performs the erasure code codec to encode the data based on, for example, the configuration data provided by the user and the health state information of the cluster composed of, for example, multiple entities. At the same time, the encoding scheme can also help the encoding scheme to achieve more beneficial effects by providing a corresponding user interface, a sequence of strategies and metadata management. In addition, the encoding scheme can also ensure data availability and better performance by considering the resource state such as hardware resources of the edge node and the preferences of the user, so as to greatly improve the data service quality of the consumer in the edge network infrastructure. Specifically, the technical solution according to embodiments of the present disclosure has many advantages over the traditional scheme.

[0063] For example, using the technical solution of the present disclosure, the user can define the encoding strategy by providing configuration data at various granularities, from specific data to buckets or user units, so as to meet the special requirements of users in a super-large-scale edge storage system. In such a super-large-scale edge storage system, a large amount of data from different third parties requires different data protection strategies. Therefore, the technical solution of the present disclosure provides high flexibility in the erasure code configuration in the edge network case.

[0064] For another example, using the technical solution of the present disclosure, since the health state and storage capacity of the entity set composed of multiple entities, etc. are first considered in the encoding design, and then the specific encoding strategy is set, the balance between available hardware resources can be achieved, so as to ensure the system performance and user requirements for data availability in severe environments. Even if the edge storage environment is uncontrollable, the technical solution of the present disclosure can still ensure the availability of data.

[0065] For another example, using the technical solutions of the present disclosure, metadata associated with erasure code is designed, so that not only can it help the normal progress of encoding, but also can provide associated information for the purpose of user tracking and system data recovery.

[0066] In summary, using the technical solutions of the present disclosure, a comprehensive solution can be provided for the configuration and execution of erasure code redemption on an edge network system, so that a unified architecture of the edge network erasure code technical solution can be provided through the design of a user interface for configuration, the interaction between the server and the client for erasure code encoding execution, the basic strategy for intelligent decision-making, and the corresponding metadata management.

[0067] Figure 4 FIG. 1 illustrates a schematic block diagram of an example device 400 that can be used to implement embodiments of the present disclosure. According to embodiments of the present disclosure, Figure 1 The computing device 110 in FIG. 1 can be implemented by the device 400. As shown, the device 400 includes a central processing unit (CPU) 401, which can perform various appropriate actions and processes according to computer program instructions stored in a read-only memory (ROM) 402 or loaded from a storage unit 408 into a random access memory (RAM) 403. In the RAM 403, various programs and data required for the operation of the device 400 can also be stored. The CPU 401, the ROM 402, and the RAM 403 are connected to each other through a bus 404. An input / output (I / O) interface 405 is also connected to the bus 404.

[0068] A plurality of components in the device 400 are connected to the I / O interface 405, including: an input unit 406, such as a keyboard, a mouse, etc.; an output unit 407, such as various types of displays, speakers, etc.; a storage unit 408, such as a magnetic disk, an optical disk, etc.; and a communication unit 409, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 409 allows the device 400 to exchange information / data with other devices through a computer network, such as the Internet, and / or various telecommunications networks.

[0069] The various processes and processes described above, such as the method 200 and the method 300, can be performed by the processing unit 401. For example, in some embodiments, the method 200 and the method 300 can be implemented as a computer software program tangibly embodied in a machine-readable medium, such as the storage unit 408. In some embodiments, portions or all of the computer program can be loaded and / or installed onto the device 400 via the ROM 402 and / or the communication unit 409. When the computer program is loaded onto the RAM 403 and executed by the CPU 401, one or more acts of the above- described method 200 and the method 300 can be performed.

[0070] Embodiments of the disclosure can relate to methods, devices, systems and / or computer program products. Computer program products can include computer readable storage media having computer readable program instructions embodied therewith to carry out various aspects of embodiments of the disclosure.

[0071] The computer readable storage medium can be a tangible device that can retain and store instructions for use by an instruction execution device. The computer readable storage medium can be, for example, but is not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. More specific examples of the computer readable storage medium as a non-exhaustive list include the following: a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), a static random access memory (SRAM), a portable compact disc read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanically encoded device such as punch-cards or punched tape, a ROM, a magnetic track storage and any suitable combination of the foregoing. A computer readable storage medium, as used herein, is not to be construed as being transitory signals per se, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through a waveguide or other transmission media, or electrical signals through a wire, cable, or other transmission medium.

[0072] Computer readable program instructions described herein can be downloaded to respective computing / processing devices from a computer readable storage medium or to an external computer or external storage device via a network, for example, the Internet, a local area network, a wide area network and / or a wireless network. The network can comprise copper transmission cables, optical transmission fibers, wireless transmission, routers, firewalls, switches, gateway computers and / or edge servers. A network adapter card or network interface in each computing / processing device receives computer readable program instructions from the network and forwards the computer readable program instructions for storage in a computer readable storage medium within the respective computing / processing device.

[0073] Computer readable program instructions for carrying out operations of embodiments of the disclosure can be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine dependent instructions, microcode, firmware instructions, state-setting data, or source or object code, in any combination of one or more programming languages including an object oriented programming language such as Smalltalk, C++ or the like, and conventional procedural programming languages such as the "C" programming language or similar programming languages. The computer readable program instructions can execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computer, for example, through the Internet using an Internet Service Provider. In some embodiments, electronic circuitry including, for example, programmable logic circuitry, field-programmable gate array (FPGA), or programmable logic array (PLA) can execute the computer readable program instructions by utilizing state information of the computer readable program instructions to personalize the electronic circuitry, in order to perform aspects of embodiments of the present disclosure.

[0074] The computer readable program instructions can also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable apparatus or other device to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide processes for implementing the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0075] The computer readable program instructions can also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable apparatus or other device to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide processes for implementing the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0076] The computer readable program instructions can also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus or other device to produce a computer implemented process such that the instructions which execute on the computer, other programmable data processing apparatus, or other device implement the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0077] The flow diagrams and the block diagrams in the drawings are presented to illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flow diagrams and the block diagrams can represent a module, segment, or portion of instructions, which comprises one or more executable instructions for implementing the specified logic functions. In some alternative implementations, the functions noted in the blocks can occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks can sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and / or flow diagrams, and combinations thereof, can be implemented by special purpose hardware-based systems that perform the specified functions or acts, or combinations of special purpose hardware and

[0078] Embodiments of the present disclosure have been described above, and the description is intended to be illustrative of the embodiments and not restrictive. Many modifications and variations of the described embodiments are possible and are within the scope of the disclosure. The selection of the terms to be used in the description is not intended to limit the scope of the embodiments described herein, but rather to best describe the principles of the embodiments in the context of the specific application.

Claims

1. An encoding method, comprising: Receive configuration data related to encoding using a predetermined encoding method; An encoding strategy is determined based on the configuration data, the encoding strategy including parameters associated with encoding data on entities; as well as This enables the data to be encoded on the entity based on the encoding strategy; The configuration data indicates information related to a plurality of data blocks to be encoded using the predetermined encoding method and at least one corresponding check block, and the configuration data also indicates the entity; and The entity said entity includes a portion of at least one computing device.

2. The method of claim 1, wherein the parameter indicates at least one of the following: The source of the data, The type of data, The data protection priority, For the number of data blocks to be encoded using the predetermined encoding method. For the number of check blocks to be encoded using the predetermined encoding method, The ratio associated with the number of data blocks and the number of check blocks. The type of encoding, and The entity.

3. The method of claim 1, wherein determining the encoding strategy comprises: Obtain health status information of a set of available entities that can be used to encode the data thereon, the set of available entities including the entities; as well as The coding strategy is determined based on the configuration data and the health status information.

4. The method of claim 3, wherein the health status information indicates at least one of the following: Environmental information of available entities in the set of available entities; The health status associated with the computing units in the available entities; The health status associated with the storage units in the available entities; Utilization rate of the computing units in the available entities; as well as The utilization rate of the storage units in the available entities.

5. The method of claim 1, wherein encoding the data on the entity based on the encoding strategy comprises: The encoding strategy is sent to the entity.

6. The method of claim 1, wherein encoding the data on the entity based on the encoding strategy comprises: This enables the entity to obtain the data.

7. The method according to claim 1, further comprising: The parameters are stored in the form of metadata.

8. The method according to claim 1, wherein the predetermined encoding method indicates that erasure coding is used for encoding.

9. An electronic device, comprising: At least one processing unit; as well as At least one memory coupled to the at least one processing unit and storing instructions for execution by the at least one processing unit, the instructions, when executed by the at least one processing unit, causing the electronic device to perform an action, the action including: Receive configuration data related to encoding using a predetermined encoding method; An encoding strategy is determined based on the configuration data, the encoding strategy including parameters associated with encoding data on entities; and This enables the data to be encoded on the entity based on the encoding strategy; wherein the configuration data indicates information related to a number of data blocks and at least one corresponding check block to be encoded using the predetermined encoding scheme, and the configuration data further indicates the entity; and wherein the entity comprises a portion of at least one computing device.

10. The electronic device of claim 9, wherein the parameter indicates at least one of: an origin of the data, a type of the data, a protection priority of the data, a number of data blocks to be encoded using the predetermined encoding scheme, a number of check blocks to be encoded using the predetermined encoding scheme, a ratio associated with the number of data blocks and the number of check blocks, a kind of the encoding, and the entity.

11. The electronic device of claim 9, wherein determining the encoding strategy comprises: obtaining health status information of a set of available entities available to encode the data thereon, the set of available entities including the entity; and determining the encoding strategy based on the configuration data and the health status information.

12. The electronic device of claim 11, wherein the health status information indicates at least one of: environmental information of an available entity in the set of available entities; a health status associated with a computing unit in the available entity; a health status associated with a storage unit in the available entity; a utilization of the computing unit in the available entity; and a utilization of the storage unit in the available entity.

13. The electronic device of claim 9, wherein causing the data to be encoded on the entity based on the encoding strategy comprises: sending the encoding strategy to the entity.

14. The electronic device of claim 9, wherein causing the data to be encoded on the entity based on the encoding strategy comprises: causing the entity to obtain the data.

15. The electronic device of claim 9, the action further comprising: storing the parameter in the form of metadata.

16. The electronic device of claim 9, wherein the predetermined encoding scheme indicates encoding with erasure code.

17. A computer program product tangibly stored on a non-transitory computer readable medium and comprising machine executable instructions that, when executed, cause a machine to perform the steps of the method of any one of claims 1 to 8. ​ ​

Images