Data storage method and device, electronic equipment and storage medium

Abstract

The invention discloses a data storage method and device, electronic equipment and a storage medium, and relates to the technical field of computers, and the method comprises the following steps: obtaining load description information and resource description information corresponding to virtual storage units matched with a target terminal; determining respective priority coefficients of the at least one virtual storage unit according to the load description information and the resource description information; under the condition that a first virtual storage unit is determined from the at least one virtual storage unit, at least one second virtual storage unit in routing connection with the first virtual storage unit and the at least one first virtual storage unit are added to a storage unit group; and determining a target storage unit from all the storage unit groups. According to the technical scheme, the priority is comprehensively obtained through the load description information and the resource description information, and then the target storage unit is obtained based on the priority, so that the accuracy of selecting the virtual storage unit is improved, and the problem that the accuracy of selecting the virtual storage unit is low in the related technology is solved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer, and particularly relates to a data storage method and device, electronic equipment and storage medium. BACKGROUND

[0002] In the current virtual storage system application scenario, with the explosive growth of data and the continuous improvement of business requirements for data read-write response speed, the reasonable selection and allocation of virtual storage units become the key link to ensure the stable operation of the system.

[0003] In the related art, the selection of the virtual storage unit mostly only considers part of the factors, for example, only determines whether to store data according to the remaining storage space size of the virtual storage unit, or only simply refers to the historical read-write speed data to make a decision, which lacks comprehensive consideration of the actual running state of the virtual storage unit, resulting in low accuracy of selecting the virtual storage unit. SUMMARY

[0004] The present application provides a data storage method and device, electronic equipment and storage medium to at least solve the problem of low accuracy of selecting a virtual storage unit in the related art.

[0005] The present application provides a data storage method, comprising: obtaining load description information and resource description information corresponding to each of at least one virtual storage unit matched with a target terminal, wherein the load description information is used to indicate the load condition of the virtual storage unit when performing a data read-write task, and the resource description information is used to indicate the read-write efficiency of the virtual storage unit when performing a data read-write task; determining the priority coefficient of each of the at least one virtual storage unit according to the load description information and the resource description information; in the case of determining at least one first virtual storage unit from the at least one virtual storage unit based on the priority coefficient, adding at least one second virtual storage unit having a routing connection with the first virtual storage unit and the at least one first virtual storage unit to a storage unit group; determining a target storage unit from all the storage unit groups, wherein the target storage unit is used to store the currently obtained object data.

[0006] The application further provides a data storage device, comprising: a first acquisition unit, configured to acquire load description information and resource description information corresponding to each of at least one virtual storage unit matched with a target terminal, wherein the load description information is used to indicate a load condition of the virtual storage unit when performing a data read / write task, and the resource description information is used to indicate a read / write efficiency of the virtual storage unit when performing the data read / write task; a first determination unit, configured to determine a priority coefficient of each of the at least one virtual storage unit according to the load description information and the resource description information; an adding unit, configured to add at least one second virtual storage unit having a routing connection with the first virtual storage unit and the at least one first virtual storage unit to a storage unit group in a case that the at least one first virtual storage unit is determined from the at least one virtual storage unit based on the priority coefficient; and a second determination unit, configured to determine a target storage unit from all the storage unit group, wherein the target storage unit is used to store currently acquired object data.

[0007] The application further provides an electronic device, comprising: a memory, configured to store a computer program; and a processor, configured to execute the computer program to implement steps of any of the data storage methods.

[0008] The application further provides a computer readable storage medium, wherein the computer readable storage medium stores a computer program, and the computer program is executed by a processor to implement steps of any of the data storage methods.

[0009] The application further provides a computer program product, comprising a computer program, and the computer program is executed by a processor to implement steps of any of the data storage methods.

[0010] By the present application, firstly, the load description information and the resource description information corresponding to each of the at least one virtual storage unit matched with the target terminal are acquired, breaking the limitation of the related art of judging only according to the remaining storage space or a single historical read-write speed, combining the load condition and the read-write efficiency of the virtual storage unit, two types of key running state information, realizing multi-dimensional comprehensive consideration of the actual running state, providing complete data support for subsequent selection to avoid evaluation deviation caused by one-sided information; then, the priority coefficient of each virtual storage unit is determined according to the above two types of description information, compared with the subjective judgment of the related art in a single dimension, the accuracy of the priority determination is improved, laying a foundation for selecting suitable storage units; subsequently, in the case of determining the first virtual storage unit, the second virtual storage unit having a routing connection with the first virtual storage unit and the first virtual storage unit are added to the storage unit group together, solving the problem of the related art of only selecting a single dimension unit and missing high-quality associated units, expanding the range of effective candidate storage units, and further improving the selection flexibility; finally, the target storage unit for storing the current object data is determined from the entire storage unit group, since the storage unit group has covered the high-priority units evaluated comprehensively and the associated units with data transmission feasibility, the target storage unit finally selected can not only meet the storage requirements of the current object data, but also has the optimal running state of low load and high read-write efficiency, thereby solving the technical problem of low accuracy of selecting a virtual storage unit in the related art, and improving the accuracy of selecting a virtual storage unit. BRIEF DESCRIPTION OF DRAWINGS

[0011] In order to more clearly illustrate the embodiments of the present application, the drawings needed in the embodiments will be briefly introduced as follows. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0012] Figure 1 A hardware structure block diagram of a server device of a data storage method according to an embodiment of the present application is provided.

[0013] Figure 2 A schematic diagram of an optional data storage method according to an embodiment of the present application is provided.

[0014] Figure 3 A structure block diagram of a data storage device according to an embodiment of the present application is provided.

[0015] Figure 4 A schematic diagram of an electronic device according to an embodiment of the present application is provided. DETAILED DESCRIPTION

[0016] With reference to the drawings and the specific embodiments described below, the technical solutions in the embodiments of the present application will be better understood.

[0017] It should be noted that, in the description of the present application, the terms "comprising", "containing" or any other variants thereof are intended to cover non-exclusive inclusion, so that the process, method, article or equipment including a series of elements not only includes those elements, but also includes other elements not explicitly listed or inherent to such process, method, article or equipment. The terms "first", "second" and the like in the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence.

[0018] In order for those skilled in the art to better understand the technical solutions of the present application, the present application will be further described in detail below with reference to the drawings and specific embodiments.

[0019] The method embodiments provided in the embodiments of the present application can be executed in a server device or similar computing device. Taking the case of running on a server device, Figure 1 is a hardware structure block diagram of a computer device of a data storage method according to an embodiment of the present application. As shown in Figure 1 , the server device can include one or more (only one is shown in Figure 1 ) processor 102 (the processor 102 can include but not limited to a central processing unit CPU, a microprocessor MCU or a programmable logic device FPGA processing device) and a memory 104 for storing data, wherein the above-mentioned server device can also include a transmission device 106 for communication function and an input and output device 108. Those skilled in the art can understand that Figure 1 The structure shown is only schematic, which does not limit the structure of the above-mentioned server device. For example, the server device can also include more or less components than Figure 1 shown, or have a different configuration from Figure 1 shown.

[0020] The memory 104 can be used to store computer programs, such as software programs of application software and modules, such as a computer program corresponding to the data storage method in the embodiments of the present application. The processor 102 can execute various functional applications and data processing, i.e., implement the above method, by running the computer programs stored in the memory 104. The memory 104 can include a high-speed random access memory, and can further include a non-volatile memory, such as one or more magnetic storage devices, flash memories, or other non-volatile solid-state memories. In some examples, the memory 104 can further include a memory remotely arranged with respect to the processor 102, which can be connected to a server device through a network. Examples of the above network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and a combination thereof.

[0021] The transmission device 106 is configured to receive or send data via a network. A specific example of the above network can include a wireless network provided by a communication provider of a server device. In one example, the transmission device 106 includes a network adapter (NIC), which can be connected to other network devices through a base station so as to be able to communicate with the Internet. In one example, the transmission device 106 can be a radio frequency (RF) module, which is configured to communicate with the Internet in a wireless manner.

[0022] In the embodiments of the present application, a data storage method is provided, Figure 2 is a flowchart of a data storage method according to the embodiments of the present application, as shown in Figure 2 The flowchart includes the following steps:

[0023] S202, obtaining load description information and resource description information corresponding to each of at least one virtual storage unit matched with a target terminal, wherein the load description information is used to indicate a load condition of the virtual storage unit when performing a data read-write task, and the resource description information is used to indicate a read-write efficiency of the virtual storage unit when performing the data read-write task;

[0024] S204, determining a priority coefficient of each of the at least one virtual storage unit according to the load description information and the resource description information;

[0025] S206, in a case where at least one first virtual storage unit is determined from the at least one virtual storage unit based on the priority coefficient, adding at least one second virtual storage unit having a routing connection with the first virtual storage unit and the at least one first virtual storage unit to a storage unit group;

[0026] S208, determining a target storage unit from all storage unit groups, wherein the target storage unit is used to store the currently acquired object data.

[0027] Optionally, in the embodiment, the target terminal can be, but is not limited to, an operation device or an edge storage control device directly facing a user or a business system, initiating an object data storage request, receiving a user operation instruction or business data, and triggering a storage process.

[0028] Optionally, in the embodiment, the virtual storage unit can be, but is not limited to, a minimum storage management unit formed by uniformly abstracting hardware layer heterogeneous storage media (such as SSD, HDD, NVMe, etc.) and different protocol storage cluster systems (such as NFS, SMB, S3) by an application layer through a high-speed network, having independent load characteristics and resource performance parameters, and can be dynamically allocated to meet the storage needs of terminal layer heterogeneous devices.

[0029] Optionally, in the embodiment, the load description information can be, but is not limited to, used to indicate the load condition of the virtual storage unit when performing data read / write tasks, and is a core index set for measuring the current working pressure of the virtual storage unit, including central processor load, memory load, storage IO load and other dimensions. The load description information reflects the current busy degree of the virtual storage unit. Through the load description information, the object data can be avoided to be stored in the virtual storage unit with too high load, and problems such as storage response delay and data read / write lag can be prevented, so as to ensure the rationality of storage resource scheduling.

[0030] Optionally, in the embodiment, the resource description information can be, but is not limited to, used to indicate the read / write efficiency of the virtual storage unit when performing data read / write tasks, and is a key index for measuring the storage performance of the virtual storage unit, which can include, but is not limited to, effective processing capacity of unit central processor resource, effective utilization rate of unit memory resource, data transmission rate and the like, and determines the efficiency of the virtual storage unit in processing data read / write tasks. The resource description information reflects the storage performance level of the virtual storage unit, and provides a performance dimension basis for priority coefficient calculation, so as to ensure that the virtual storage unit with high read / write efficiency is considered first when the virtual storage unit is selected, the overall speed of object data storage is improved, and the demand for storage efficiency in a high-performance heterogeneous computing scenario is met.

[0031] Optionally, in the embodiment, the priority coefficient can be, but is not limited to, a value obtained by a preset algorithm according to the load description information and the resource description information of the virtual storage unit, for quantifying the priority of the virtual storage unit, and the larger the value, the more suitable the unit is for receiving the object data storage task of the target terminal in the current scenario. The priority coefficient provides a unified quantitative standard for the screening of the virtual storage unit, solves the priority ranking problem among multiple virtual storage units, ensures that the target terminal can quickly identify the optimal storage unit, and improves the accuracy and efficiency of the selection of the storage resource.

[0032] Optionally, in the embodiment, the first virtual storage unit can be, but is not limited to, a virtual storage unit with a relatively high priority selected based on the priority coefficient from at least one virtual storage unit matched with the target terminal, and is a component of the storage unit group.

[0033] For further illustration, it is assumed that the virtual storage units matched with the target terminal are VS1, VS2, VS3 and VS4, the priority coefficients of which are 0.74, 0.82, 0.44 and 0.56 respectively, and if the virtual storage unit with a priority coefficient greater than 0.6 is set as the first virtual storage unit, then VS1 and VS2 will be selected as the first virtual storage unit and become components of the storage unit group.

[0034] Optionally, in the embodiment, the routing connection can be, but is not limited to, a communication link established between the virtual storage units through a network routing device, which can ensure that different virtual storage units can realize data transmission and state interaction.

[0035] Optionally, in the embodiment, the second virtual storage unit can be, but is not limited to, a virtual storage unit having a storage capacity and having a routing connection with the first virtual storage unit, and is a supplement to the first virtual storage unit.

[0036] Optionally, in the embodiment, the storage unit group can be, but is not limited to, a virtual storage unit collection composed of at least one first virtual storage unit and at least one second virtual storage unit having a routing connection with the first virtual storage unit, and is a screening range of the target storage unit, and the virtual storage units in the collection have different priorities and storage capacities, and jointly provide candidate resources for the object data storage of the target terminal.

[0037] Optionally, in the embodiment, the object data can be, but is not limited to, specific data currently needed to be stored by the target terminal, and is a processing object of the storage task.

[0038] Optionally, in the embodiment, the target storage unit can be but is not limited to a virtual storage unit selected from the group of storage units, finally used to store the currently acquired object data, meeting the conditions of idle state, performance adaptation to object data demand, etc., and is an execution carrier of the storage task.

[0039] Optionally, in the embodiment, first, all virtual storage units directly connected with the target terminal initiating the storage request are determined, and then the load description information and resource description information of the matching virtual storage units are acquired through the virtualization platform of the application layer. By accurately selecting the virtual storage units matching the target terminal, irrelevant data collection of the virtual storage units is avoided, and the data processing amount is reduced; at the same time, the two types of key information of load and resource are acquired, ensuring that the subsequent priority calculation has comprehensive and accurate data basis, laying a foundation for selecting the adaptive storage unit, and preventing priority judgment deviation caused by information loss.

[0040] Next, based on the acquired load description information and resource description information, priority value calculation is performed on each virtual storage unit to obtain the priority coefficient of each unit, and the coefficient size directly reflects the adaptation degree of the unit in receiving the storage task in the current scenario. By converting the load and resource characteristics of the virtual storage unit into the priority coefficient, the problem of difficulty in directly comparing the adaptability between different virtual storage units is solved.

[0041] Then, according to the priority coefficient, a screening rule is set to screen out a first virtual storage unit with higher priority from the matching virtual storage units; then, through the network routing topology information, a second virtual storage unit directly connected with each first virtual storage unit is identified; finally, the first virtual storage unit and the second virtual storage unit are integrated to form an ordered storage unit group, and the screening range of the subsequent target storage unit is determined.

[0042] Finally, based on the real-time state of each virtual storage unit, a virtual storage unit most suitable for storing the current object data is selected from the group of all storage units as the target storage unit, and the unit is determined as the storage carrier of the current object data.

[0043] It can be understood that, by matching the virtual storage unit first and collecting information, then calculating the priority coefficient quantifying the adaptability, then constructing the storage unit group containing the core and the expansion unit, and finally screening the target storage unit, the problem of unordered selection of virtual storage units in a heterogeneous storage environment can be solved. Through the matching mechanism and the priority quantification, it is ensured that the selected storage unit is accurately adapted to the target terminal and the object data demand, avoiding storage delay or performance waste caused by improper selection. By constructing the storage unit group to define the screening range, unordered search in the entire virtual storage resource pool is avoided, and the storage task blocking probability is reduced by combining the routing connection to expand the candidate range. Through the abstraction of the virtual storage unit, the target terminal does not need to pay attention to the difference between the underlying storage medium and the protocol, simplifying the storage resource calling process, reducing the adaptation cost of multi-device collaboration, and providing efficient and stable storage support for high-performance heterogeneous computing.

[0044] Through the embodiments provided in the present application, firstly, the load description information and the resource description information corresponding to each of the at least one virtual storage unit matched with the target terminal are acquired, breaking the limitation of related art that only relies on the remaining storage space or a single historical read-write speed for judgment. The load status and read-write efficiency of the virtual storage unit are combined, two types of key running state information are combined, the actual running state is comprehensively considered in multiple dimensions, complete data support is provided for subsequent selection to avoid evaluation deviation caused by one-sided information; then, the priority coefficient of each virtual storage unit is determined according to the above two types of description information, compared with the subjective judgment of a single dimension in related art, the accuracy of priority determination is improved, laying a foundation for screening suitable storage units; subsequently, in the case of determining the first virtual storage unit, the second virtual storage unit having a routing connection with the first virtual storage unit and the first virtual storage unit are added to the storage unit group together, solving the problem that related art only screens units that meet a single dimension and misses high-quality associated units, expanding the range of effective candidate storage units, and further improving the selection flexibility; finally, the target storage unit for storing the current object data is determined from the entire storage unit group. Since the storage unit group has covered high-priority units that are comprehensively evaluated and associated units that have data transmission feasibility, the finally selected target storage unit can meet the storage demand of the current object data and has the optimal running state of low load and high read-write efficiency, thereby solving the technical problem of low accuracy of selecting virtual storage units in related art and improving the accuracy of selecting virtual storage units.

[0045] As an optional solution, adding the at least one second virtual storage unit having a routing connection with the first virtual storage unit and the at least one first virtual storage unit to the storage unit group comprises:

[0046] According to the priority order indicated by the priority coefficient, at least one virtual storage unit is traversed, and the following operations are performed:

[0047] S1, obtaining a first virtual storage unit from at least one virtual storage unit;

[0048] S2, obtaining a neighbor unit set corresponding to the first virtual storage unit, wherein the neighbor unit set comprises at least one neighbor storage unit having a routing connection with the first virtual storage unit;

[0049] S3, determining a neighbor storage unit having a maximum priority indicated by a priority coefficient as a current neighbor storage unit from the at least one neighbor storage unit;

[0050] S4, in a case where a second priority indicated by the priority coefficient of the current neighbor storage unit is greater than a first priority indicated by the priority coefficient of the first virtual storage unit, determining the current neighbor storage unit as a second virtual storage unit, and adding the first virtual storage unit and the second virtual storage unit to the storage unit group.

[0051] Optionally, in the embodiment, the neighbor unit set may, but is not limited to, be a set of all neighbor storage units having a direct routing connection with the first virtual storage unit, and the neighbor storage units have data interaction capability with the first virtual storage unit and are potential candidate ranges for expansion of the storage unit group.

[0052] Optionally, in the embodiment, the neighbor storage unit may, but is not limited to, be a virtual storage unit having a direct routing connection with the first virtual storage unit, having a priority coefficient, a load, and energy efficiency characteristics, and being a candidate unit for expansion of the storage unit group. The neighbor storage unit may be a potential expansion unit for the storage unit group, and when the neighbor storage unit has a higher priority than the first virtual storage unit, the neighbor storage unit may be included in the storage unit group to enhance overall performance adaptability of the storage unit group and ensure that the terminal can select a better storage unit.

[0053] Optionally, in the embodiment, the second virtual storage unit may, but is not limited to, be a virtual storage unit determined by the current neighbor storage unit when the second priority of the current neighbor storage unit is greater than the first priority of the first virtual storage unit, and be an expansion member of the storage unit group.

[0054] Optionally, in the embodiment, first, the at least one virtual storage unit is prioritized from high to low according to a priority coefficient of the virtual storage unit, and then each virtual storage unit is accessed in order according to the prioritization, and in the process of accessing each virtual storage unit, the currently traversed virtual storage unit is taken as the first virtual storage unit.

[0055] Then, by querying the network routing topology information of the application layer, all neighbor storage units having a direct routing connection with the first virtual storage unit are found, and these neighbor storage units are integrated to form a neighbor unit set, thereby explicitly defining the potential candidate range of the storage unit group expansion. By enabling the storage unit group to expand from a single first virtual storage unit to a set containing multiple neighbor storage units, the candidate objects of the storage unit group are enriched.

[0056] Then, for all neighbor storage units in the neighbor unit set, the respective priority coefficients are extracted, and by comparing the priority coefficients, the neighbor storage unit having the largest priority coefficient is screened out and is determined as the current neighbor storage unit. This reduces the workload of the priority comparison, and there is no need to compare all units in the neighbor unit set with the first virtual storage unit one by one. Instead, only the current neighbor storage unit having the highest priority needs to be locked, and whether the neighbor unit set has an expansion unit that can be included in the storage unit group can be determined, thereby improving the expansion efficiency.

[0057] Finally, the priority coefficient of the current neighbor storage unit and the priority coefficient of the first virtual storage unit are compared in size. If the second priority is greater than the first priority, it indicates that the current neighbor storage unit has higher adaptability, and the current neighbor storage unit is determined as the second virtual storage unit. Then, the first virtual storage unit and the second virtual storage unit are jointly added to the storage unit group, thereby completing the expansion of the storage unit group. If the second priority is not greater than the first priority, the storage unit group is not expanded.

[0058] It can be understood that by expanding the storage unit group through routing connection, the limitation of a single unit is broken, and the candidate objects are enriched. Even if the first virtual storage unit is busy, the terminal can select an idle second virtual storage unit from the storage unit group, thereby improving the success rate of completing the storage task.

[0059] According to the embodiments provided in the present application, a first virtual storage unit is obtained from at least one virtual storage unit; a neighbor unit set corresponding to the first virtual storage unit is obtained, wherein the neighbor unit set includes at least one neighbor storage unit having a routing connection with the first virtual storage unit; a neighbor storage unit having the largest priority indicated by a priority coefficient in the at least one neighbor storage unit is determined as a current neighbor storage unit; in a case where a second priority indicated by a priority coefficient of the current neighbor storage unit is greater than a first priority indicated by a priority coefficient of the first virtual storage unit, the current neighbor storage unit is determined as a second virtual storage unit, and the first virtual storage unit and the second virtual storage unit are added to a storage unit group. By expanding the storage unit group through routing connection, the limitation of a single unit is broken, and the candidate objects are enriched. Even if the first virtual storage unit is busy, the terminal can select an idle second virtual storage unit from the storage unit group, thereby improving the success rate of completing the storage task.

[0060] As an optional solution, after determining the current neighbor storage unit as the second virtual storage unit, the method further comprises:

[0061] S1, determining the second virtual storage unit as the current virtual storage unit;

[0062] S2, obtaining a neighbor unit set corresponding to the current virtual storage unit, wherein the neighbor unit set comprises at least one neighbor storage unit having a routing connection with the current virtual storage unit;

[0063] S3, determining, as the current neighbor storage unit, a neighbor storage unit having a maximum priority indicated by a priority coefficient among the at least one neighbor storage unit;

[0064] S4, in a case where a third priority indicated by a priority coefficient of the current neighbor storage unit is greater than a fourth priority indicated by a priority coefficient of the current virtual storage unit, determining the current neighbor storage unit as a third virtual storage unit, and adding the third virtual storage unit to the storage unit group.

[0065] Optionally, in the embodiment, the third virtual storage unit can be, but is not limited to, a virtual storage unit determined by the current neighbor storage unit when the third priority of the current neighbor storage unit is greater than the fourth priority of the current virtual storage unit, and is an added expansion member of the storage unit group. By further expanding the size of the storage unit group and improving the overall adaptability, the storage unit group contains more high-adaptability units, and the probability of storage task blocking caused by a single high-adaptability unit being busy is reduced.

[0066] Optionally, in the embodiment, the second virtual storage unit determined in the process of constructing the storage unit group is defined as the current virtual storage unit, so that the unit becomes an object of the recursive expansion process of the storage unit group.

[0067] Next, by querying the network routing topology data recorded by the application layer, all neighbor storage units having a direct routing connection with the current virtual storage unit are filtered out, and these neighbor storage units are integrated to form a set, i.e., a neighbor unit set.

[0068] Then, for each neighbor storage unit in the neighbor unit set, a corresponding priority coefficient is extracted, and the neighbor storage unit having the maximum priority coefficient is found out through numerical comparison, and the unit is determined as the current neighbor storage unit.

[0069] Finally, the priority coefficient of the current neighbor storage unit and the priority coefficient of the current virtual storage unit are extracted respectively, and the two priorities are compared in value; if the third priority is greater than the fourth priority, the third virtual storage unit is determined, and then the third virtual storage unit is added to the existing storage unit group; if the third priority is not greater than the fourth priority, the expansion operation is not performed.

[0070] Further, assuming that there are two virtual storage units with priorities of 3 and 2, when the storage unit group is established, the two virtual storage units with priorities of 3 and 2 are respectively taken as the first element to construct two storage unit groups, the first storage unit group is {3}, and the second storage unit group is {2}.

[0071] For the first storage unit group, starting from the last virtual storage unit with a priority of 3, the virtual storage units 4, 5 and 6 with priorities higher than 3 and not added to the first storage unit group are found from the neighbor set of the virtual storage unit, the virtual storage unit corresponding to the highest priority 6 is selected and added to the first storage unit group, and the first storage unit group is {3, 6}; then starting from the last virtual storage unit with a priority of 6, the virtual storage unit with a priority higher than 6 and not added to the first storage unit group is found in the neighbor set of the virtual storage unit, until there is no virtual storage unit with a higher priority and not added to the first storage unit group in the neighbor set of the last virtual storage unit in the first storage unit group, and the first storage unit group is updated.

[0072] Then, for the second storage unit group, repeating from the last virtual storage unit with a priority of 2, the virtual storage units with priorities higher than 2 and not added to the second storage unit group are found in the neighbor set of 2, the virtual storage unit corresponding to the highest priority 7 is selected and added to the second storage unit group, and the second storage unit group is {2, 7}; then starting from the last virtual storage unit with a priority of 7, the virtual storage unit with a priority higher than 7 and not added to the second storage unit group is found in the neighbor set of the virtual storage unit, until there is no virtual storage unit with a higher priority and not added to the second storage unit group in the neighbor set of the last element in the second storage unit group, and the second storage unit group is updated.

[0073] It can be understood that, with the expansion of the storage unit group scale, the terminal has more high-adaptability options when selecting a target storage unit, and can quickly find a high-adaptability idle unit even if some units are busy, thereby reducing the probability of storage task blocking. Meanwhile, the third virtual storage unit in the storage unit group has a direct routing connection with the current virtual storage unit, so that data can be transmitted efficiently by using the existing routing link when storing data in the third virtual storage unit, thereby avoiding the increase of transmission delay caused by complex routing links.

[0074] In the embodiments provided in the present application, a second virtual storage unit is determined as a current virtual storage unit, a neighbor unit set corresponding to the current virtual storage unit is obtained, the neighbor unit set includes at least one neighbor storage unit having a routing connection with the current virtual storage unit, a neighbor storage unit having the maximum priority indicated by a priority coefficient in the at least one neighbor storage unit is determined as a current neighbor storage unit, and the current neighbor storage unit is determined as a third virtual storage unit and added to the storage unit group in a case where a third priority indicated by the priority coefficient of the current neighbor storage unit is greater than a fourth priority indicated by the priority coefficient of the current virtual storage unit.

[0075] As an optional solution, the load description information and the resource description information corresponding to each of the at least one virtual storage unit matched with the target terminal are obtained, including:

[0076] Based on the central processor load information, the memory load information and the read-write load information of the fourth virtual storage unit in the at least one storage unit, a load coefficient corresponding to the fourth virtual storage unit is obtained, wherein the central processor load information is used to indicate the average utilization rate of the central processor of the fourth virtual storage unit in a unit time, the memory load information is used to indicate the remaining available memory of the fourth virtual storage unit, the read-write load information is used to indicate the read-write data volume of the fourth virtual storage unit in a unit time, and the load description information includes the load coefficient.

[0077] Optionally, in the present embodiment, the fourth virtual storage unit may, but is not limited to, be a specific virtual storage unit selected from the at least one storage unit and subjected to load coefficient calculation.

[0078] Optionally, in the present embodiment, the central processor load information may, but is not limited to, be used to quantitatively indicate the average utilization rate of the central processor of the fourth virtual storage unit in a unit time.

[0079] Optionally, in the present embodiment, the memory load information may, but is not limited to, be used to indicate the remaining available memory capacity of the fourth virtual storage unit, and is index information reflecting the idle degree of the memory resource in the unit.

[0080] Optionally, in the embodiment, the read-write load information can be, but is not limited to, the total amount of read-write data of the fourth virtual storage unit in a unit time, which is an index reflecting the IO busy degree of the unit, and can be obtained by summing the read data amount and the write data amount in a unit time through an IO monitoring tool.

[0081] Optionally, in the embodiment, the load coefficient is a quantitative value calculated based on the central processor load information, the memory load information and the read-write load information of the fourth virtual storage unit, and is used to comprehensively reflect the overall load condition of the fourth virtual storage unit. The larger the value is, the higher the overall load of the unit is, and the load coefficient is a core component of the load description information.

[0082] It can be understood that, first, the fourth virtual storage unit is selected as a target object from at least one storage unit of a system application layer; then the central processor load information, the memory load information and the read-write load information of the unit are collected through a monitoring module; and then the load coefficient comprehensively reflecting the overall load of the unit is calculated. Since the load description information contains not only the load coefficient for quick comparison, but also the original load information, the scheduling module can efficiently screen low-load units and realize accurate scheduling, thereby avoiding waste or overload of storage resources.

[0083] According to the embodiments provided in the application, the load coefficient corresponding to the fourth virtual storage unit is obtained based on the central processor load information, the memory load information and the read-write load information of the fourth virtual storage unit in at least one storage unit, wherein the central processor load information is used to indicate the average utilization rate of the central processor of the fourth virtual storage unit in a unit time, the memory load information is used to indicate the remaining available memory of the fourth virtual storage unit, and the read-write load information is used to indicate the read-write data amount of the fourth virtual storage unit in a unit time. The load description information includes the load coefficient. First, the fourth virtual storage unit is selected as a target object from at least one storage unit of a system application layer; then the central processor load information, the memory load information and the read-write load information of the unit are collected through a monitoring module; and then the load coefficient comprehensively reflecting the overall load of the unit is calculated. The load description information contains not only the load coefficient for quick comparison, but also the original load information, so that the scheduling module can efficiently screen low-load units and realize accurate scheduling, thereby avoiding waste or overload of storage resources.

[0084] As an optional solution, the load coefficient corresponding to the fourth virtual storage unit is obtained based on the central processor load information, the memory load information and the read-write load information of the fourth virtual storage unit in at least one storage unit, and includes the following steps:

[0085] S1, based on the central processing unit type corresponding to the fourth virtual storage unit, obtaining a first load weight, a second load weight and a third load weight, wherein the first load weight is used to indicate the importance of the central processing unit load information, the second load weight is used to indicate the importance of the memory load information, and the third load weight is used to indicate the importance of the read-write load information, and the sum of the first load weight, the second load weight and the third load weight satisfies a preset threshold condition;

[0086] S2, based on the first load weight, the second load weight and the third load weight, the central processing unit load information, the memory load information and the read-write load information are weighted and summed to obtain a load coefficient corresponding to the fourth virtual storage unit.

[0087] Optionally, in the embodiment, the central processing unit type may be, but is not limited to, a specific model or architecture category of the central processing unit carried by the fourth virtual storage unit, and different central processing unit types have differences in processing computing tasks, memory interaction, IO scheduling and the like, which are the basis for determining the weight distribution of different load information.

[0088] Optionally, in the embodiment, the first load weight is determined based on the central processing unit type of the fourth virtual storage unit, and is used to quantify the weight value of the importance of the central processing unit load information in the load coefficient calculation. The greater the influence of the central processing unit type on the processing capability of the central processing unit, the higher the first load weight, which is one of the parameters for weighted summation calculation.

[0089] Optionally, in the embodiment, the second load weight is determined based on the central processing unit type of the fourth virtual storage unit, and is used to quantify the weight value of the importance of the memory load information in the load coefficient calculation. The compatibility of the central processing unit type with the memory controller and the memory access speed are directly related, and the better the compatibility and the faster the access speed, the higher the second load weight.

[0090] Optionally, in the embodiment, the third load weight is determined based on the central processing unit type of the fourth virtual storage unit, and is used to quantify the weight value of the importance of the read-write load information in the load coefficient calculation. The influence of the central processing unit type on the IO scheduling capability directly determines the third load weight. The stronger the IO scheduling capability of the central processing unit, the higher the third load weight.

[0091] Optionally, in the embodiment, the preset threshold condition is a numerical condition preset by the system for constraining the sum of the first, second and third load weights, to ensure that the weight distribution of the three types of load information can completely cover all dimensions of the load coefficient calculation.

[0092] Optionally, in the embodiment, the fourth virtual storage unit load coefficient weighted calculation process, around the hardware characteristics determine the weight, the weight guide coefficient calculation, first according to the central processing unit type of the fourth virtual storage unit, from the preset mapping table to obtain the matching first, second, third load weight; then the three kinds of load information are normalized, and then weighted summation is carried out according to the weight, and finally the load coefficient which can accurately reflect the actual load condition of the unit is obtained.

[0093] It can be understood that, since the load weight is dynamically allocated based on the central processing unit type, the load coefficient calculation can be adapted to the ability characteristics of different central processing units, such as high-performance computing central processing units focusing on central processing unit load weight, IO optimized central processing units focusing on read-write load weight, avoiding load evaluation distortion caused by uniform weight, and ensuring that the load coefficient can truly reflect the actual load pressure of the unit.

[0094] Through the embodiments provided in the application, based on the central processing unit type corresponding to the fourth virtual storage unit, the first load weight, the second load weight and the third load weight are obtained, wherein the first load weight is used to indicate the importance of the central processing unit load information, the second load weight is used to indicate the importance of the memory load information, and the third load weight is used to indicate the importance of the read-write load information, and the sum of the first load weight, the second load weight and the third load weight satisfies a preset threshold condition; based on the first load weight, the second load weight and the third load weight, the central processing unit load information, the memory load information and the read-write load information are weighted summed to obtain the load coefficient corresponding to the fourth virtual storage unit. Since the load weight is dynamically allocated based on the central processing unit type, the load coefficient calculation can be adapted to the ability characteristics of different central processing units, avoiding load evaluation distortion caused by uniform weight, and ensuring that the load coefficient can truly reflect the actual load pressure of the unit.

[0095] As an optional solution, the load description information and resource description information corresponding to each of the at least one virtual storage unit matched with the target terminal are obtained, including:

[0096] Based on the central processing unit energy efficiency information and the memory energy efficiency information of the fourth virtual storage unit, the energy efficiency coefficient corresponding to the fourth virtual storage unit is obtained, wherein the central processing unit energy efficiency information is used to indicate the processing capacity of the fourth virtual storage unit on a unit central processing unit, and the memory energy efficiency information is used to indicate the utilization rate of the fourth virtual storage unit on a unit memory resource, and the resource description information includes the energy efficiency coefficient.

[0097] Optionally, in this embodiment, the central processor energy efficiency information is used to quantitatively indicate the effective processing capacity of the fourth virtual storage unit per unit of central processor resource, which can be but is not limited to the amount of data processed per unit of central processor resource divided by the number of completed tasks, and can be but is not limited to obtained by real-time monitoring of the central processor resource input and actual processing results by the virtualization platform. The central processor energy efficiency information reflects the utilization efficiency of the central processor resource of the fourth virtual storage unit, and the stronger the processing capacity per unit of central processor resource, the higher the central processor energy efficiency, thereby providing key data in the central processor dimension for energy efficiency coefficient calculation and ensuring that the energy efficiency coefficient can reflect the advantages or shortcomings of the unit in the utilization of computing resources.

[0098] Optionally, in this embodiment, the memory energy efficiency information is used to quantitatively indicate the effective utilization rate of the fourth virtual storage unit per unit of memory resource, which can be but is not limited to obtained by calculating the amount of active data per unit of memory resource per unit of time, wherein the active data refers to data that is read, written, or modified more than a threshold number of times per unit of time.

[0099] Optionally, in this embodiment, the energy efficiency coefficient is a quantitative value obtained based on the central processor energy efficiency information and the memory energy efficiency information of the fourth virtual storage unit, and is used to comprehensively reflect the overall energy efficiency level of the fourth virtual storage unit, wherein the larger the value, the higher the efficiency of the unit in the utilization of central processor and memory resources.

[0100] Optionally, in this embodiment, after determining the fourth virtual storage unit whose energy efficiency needs to be evaluated, the central processor energy efficiency information and the memory energy efficiency information of the unit are collected through the application layer virtualization platform; then, the energy efficiency coefficient that comprehensively reflects the overall energy efficiency of the unit is calculated.

[0101] It can be understood that by integrating the multi-dimensional information of central processor energy efficiency and memory energy efficiency into a single energy efficiency coefficient, the problem of difficulty in directly comparing traditional multi-dimensional efficiency is solved, and the efficiency difference of different virtual storage units can be quickly judged by the storage resource scheduling module, thereby improving the scheduling decision efficiency.

[0102] Through the embodiments provided in this application, based on the central processor energy efficiency information and the memory energy efficiency information of the fourth virtual storage unit, the energy efficiency coefficient corresponding to the fourth virtual storage unit is obtained, wherein the central processor energy efficiency information is used to indicate the processing capacity of the fourth virtual storage unit per unit of central processor, the memory energy efficiency information is used to indicate the utilization rate of the fourth virtual storage unit per unit of memory resource, and the resource description information includes the energy efficiency coefficient. By integrating the multi-dimensional information of central processor energy efficiency and memory energy efficiency into a single energy efficiency coefficient, the problem of difficulty in directly comparing traditional multi-dimensional efficiency is solved, and the efficiency difference of different virtual storage units can be quickly judged by the storage resource scheduling module, thereby improving the scheduling decision efficiency.

[0103] As an optional approach, before obtaining the energy efficiency coefficient corresponding to the fourth virtual memory unit based on the CPU energy efficiency information and memory energy efficiency information of the fourth virtual memory unit, the following steps are included:

[0104] S1 determines the resource size value by multiplying the number of CPU cores allocated to the fourth virtual memory unit by the preset time.

[0105] S2, the ratio of the number of tasks completed by the central processing unit in the fourth virtual storage unit within a preset time to the resource scale value is determined as the central processing unit energy efficiency information;

[0106] S3 determines the value of the temporary storage scale by multiplying the memory capacity allocated to the fourth virtual storage unit by the preset time.

[0107] S4, the ratio of the amount of active data in the fourth virtual storage unit to the value of the temporary storage size within a preset time is determined as memory energy efficiency information, wherein the amount of active data is the actual storage capacity occupied by data whose number of read and write operations meets the threshold condition.

[0108] Optionally, in this embodiment, the number of CPU cores refers to the number of CPU cores allocated to the fourth virtual storage unit. It is an indicator that measures the scale of CPU computing resources that the unit can call upon. Different virtual storage units are allocated different numbers of cores according to the complexity of the storage tasks they undertake, which directly affects the upper limit of the CPU processing capability of the unit.

[0109] Optionally, in this embodiment, the preset time may be, but is not limited to, a time period pre-set by the system for uniformly calculating energy efficiency information, so as to ensure that the energy efficiency information of different fourth virtual storage units is calculated based on the same time dimension, thereby eliminating the impact of time differences on the energy efficiency evaluation results.

[0110] Optionally, in this embodiment, the resource scale value can be, but is not limited to, a quantized value calculated by multiplying the number of CPU cores of the fourth virtual storage unit by a preset time, and is used to quantify the total scale of CPU resources that the fourth virtual storage unit can call within a preset time.

[0111] Optionally, in this embodiment, the number of tasks called to complete by the central processing unit may refer to, but is not limited to, the total number of storage-related tasks successfully completed by the fourth virtual storage unit within a preset time by calling the allocated central processing unit resources, such as data block writing tasks, data verification tasks, cache synchronization tasks, etc. The more tasks there are, the better the processing results of the central processing unit.

[0112] Optionally, in the embodiment, the temporary storage scale value may be, but is not limited to, a total scale of memory resources that the fourth virtual storage unit can call within a preset time, so as to integrate the resource input of the memory capacity and the time dimension into a single value.

[0113] Optionally, in the embodiment, the active data amount may be, but is not limited to, an actual storage capacity occupied by data whose read-write operation frequency reaches a system preset frequency threshold within a preset time of the fourth virtual storage unit. The greater the active data amount is, the more high-value data in the memory cache.

[0114] Optionally, in the embodiment, the actual storage occupation capacity may be, but is not limited to, a physical storage space size actually occupied in the memory.

[0115] Optionally, in the embodiment, the number of central processor cores allocated for the fourth virtual storage unit is first acquired, and then a system preset uniform time period is extracted. Then, the number of central processor cores and the preset time are used to calculate a value quantifying the total input scale of central processor resources, and the value is determined as the resource scale value. Next, the total amount of storage-related tasks successfully completed by the fourth virtual storage unit within the preset time by calling the allocated central processor resources is counted, and then the ratio of the task amount to the resource scale value is obtained, and the obtained ratio is determined as the central processor energy efficiency information.

[0116] Further, the memory capacity allocated for the fourth virtual storage unit is acquired, and then a system preset time period is extracted. Then, the memory capacity and the preset time are used to obtain the temporary storage scale value. Then, the ratio of the active data amount of the fourth virtual storage unit within the preset time to the temporary storage scale value is determined as the memory energy efficiency information.

[0117] It can be understood that, since the central processor task amount and the active data amount are both based on the real-time running data of the fourth virtual storage unit, evaluation distortion caused by the dependence on static parameters when acquiring the parameters of the fourth virtual storage unit can be avoided, so that the evaluation result of the fourth virtual storage unit can match the actual resource utilization state of the fourth virtual storage unit.

[0118] The product of the number of central processor cores allocated to the fourth virtual storage unit and the preset time is determined as a resource size value; the ratio of the number of tasks completed by the central processor in the preset time to the resource size value is determined as the central processor energy efficiency information; the product of the memory capacity allocated to the fourth virtual storage unit and the preset time is determined as a temporary storage size value; and the ratio of the active data amount of the fourth virtual storage unit in the preset time to the temporary storage size value is determined as the memory energy efficiency information, where the active data amount is the actual storage occupation capacity of the data whose read-write operation frequency meets the frequency threshold condition. Since the number of central processor tasks and the active data amount are based on the real-time running data of the fourth virtual storage unit, the evaluation distortion caused by the dependence on static parameters when obtaining the parameters of the fourth virtual storage unit can be avoided, so that the evaluation result of the fourth virtual storage unit can match the actual resource utilization state of the fourth virtual storage unit.

[0119] As an optional solution, based on the central processor energy efficiency information and the memory energy efficiency information of the fourth virtual storage unit, the energy efficiency coefficient corresponding to the fourth virtual storage unit is obtained, including:

[0120] S1, based on the central processor type corresponding to the fourth virtual storage unit, the first energy efficiency weight and the second energy efficiency weight are obtained, where the first energy efficiency weight is used to indicate the importance of the central processor energy efficiency information, and the second energy efficiency weight is used to indicate the importance of the memory energy efficiency information;

[0121] S2, the central processor energy efficiency information and the memory energy efficiency information are weighted and summed based on the first energy efficiency weight and the second energy efficiency weight to obtain the energy efficiency coefficient.

[0122] Optionally, in this embodiment, the first energy efficiency weight can be but is not limited to a quantitative weight determined based on the central processor type of the fourth virtual storage unit, which is used to clearly indicate the importance proportion of the central processor energy efficiency information in the energy efficiency coefficient calculation.

[0123] Optionally, in this embodiment, the second energy efficiency weight can be but is not limited to a quantitative weight determined based on the central processor type of the fourth virtual storage unit, which is used to clearly indicate the importance proportion of the memory energy efficiency information in the energy efficiency coefficient calculation.

[0124] Optionally, in this embodiment, the specific type of the central processor carried by the fourth virtual storage unit is determined first, and the first energy efficiency weight and the second energy efficiency weight corresponding to the central processor type are determined.

[0125] Then, the central processor energy efficiency information and the memory energy efficiency information are normalized and calculated, and the central processor energy efficiency information and the memory energy efficiency information are weighted and summed based on the first energy efficiency weight and the second energy efficiency weight, and the weighted sum result is determined as the energy efficiency coefficient of the fourth virtual storage unit, and the larger the value is, the higher the resource utilization efficiency of the fourth virtual storage unit is.

[0126] It can be understood that, since the energy efficiency weight is dynamically allocated based on the central processor type, the energy efficiency coefficient calculation can be adapted to the capability characteristics of different central processors, avoiding the reduction of evaluation accuracy caused by uniform weight, so as to ensure that the energy efficiency coefficient obtained by weighting and summing the central processor energy efficiency information and the memory energy efficiency information can truly reflect the actual resource utilization efficiency of the unit.

[0127] According to the embodiments provided in the present application, the first energy efficiency weight and the second energy efficiency weight are obtained based on the central processor type corresponding to the fourth virtual storage unit, wherein the first energy efficiency weight is used to indicate the importance of the central processor energy efficiency information, and the second energy efficiency weight is used to indicate the importance of the memory energy efficiency information; the central processor energy efficiency information and the memory energy efficiency information are weighted and summed based on the first energy efficiency weight and the second energy efficiency weight, and the energy efficiency coefficient is obtained. Since the energy efficiency weight is dynamically allocated based on the central processor type, the energy efficiency coefficient calculation can be adapted to the capability characteristics of different central processors, avoiding the reduction of evaluation accuracy caused by uniform weight, so as to ensure that the energy efficiency coefficient obtained by weighting and summing the central processor energy efficiency information and the memory energy efficiency information can truly reflect the actual resource utilization efficiency of the unit.

[0128] As an optional solution, before obtaining the load description information and the resource description information corresponding to each of the at least one virtual storage unit matched with the target terminal, the method further comprises:

[0129] S1, determining a storage device having a remaining storage capacity satisfying a remaining capacity condition and located in a target space as a candidate device, wherein the target space is a region having a device density satisfying a density threshold condition, and the device density is used to indicate the number of storage devices in a unit space;

[0130] S2, determining a master device and at least one slave device from the current candidate device currently being traversed and other candidate devices located in adjacent spaces, wherein the current candidate device can communicate with other storage devices located in adjacent spaces;

[0131] S3, constructing a storage cluster corresponding to the current candidate device based on the master device and the at least one slave device, wherein the storage cluster is used to determine at least one virtual storage unit.

[0132] Optionally, in the embodiment, the remaining storage capacity may, but is not limited to, refer to the size of the physical storage space that is currently not occupied by the storage device and can be used to store data, and may, but is not limited to, be obtained in real time by a capacity monitoring module built in the storage device.

[0133] Optionally, in the embodiment, the remaining capacity condition may, but is not limited to, be a capacity standard preset by the system for screening candidate devices, for example, the remaining storage capacity is greater than or equal to twice the target terminal single storage data amount, and the like, to ensure that the candidate device has sufficient remaining storage capacity.

[0134] Optionally, in the embodiment, the target space may, but is not limited to, be a specific physical space in the deployment area of the storage device, in which the device density satisfies a system preset density threshold condition, and is a space screening range of the candidate device, to ensure that the candidate device is screened from the physical space in which the device density satisfies a specific condition.

[0135] Optionally, in the embodiment, the device density may, but is not limited to, be used to quantitatively indicate the number of storage devices deployed in a unit physical space, and may be calculated by a ratio of the total number of storage devices in the target area to the physical space area of the area.

[0136] Optionally, in the embodiment, the adjacent space may, but is not limited to, be a sub-area of the target space that is directly adjacent to the physical space in which the current candidate device is located and in which the device density also satisfies the density threshold condition.

[0137] Optionally, in the embodiment, the master device may, but is not limited to, be a candidate device selected from the current candidate device and other candidate devices in the adjacent space, which is responsible for data transmission with the virtual storage unit and stores data of the virtual storage unit, and undertakes tasks such as data distribution, load balancing, and fault detection in the storage cluster.

[0138] Optionally, in the embodiment, the slave device may, but is not limited to, be a candidate device other than the master device in the storage cluster, which undertakes specific data storage tasks under the coordination of the master device, accepts state monitoring and task scheduling of the master device, and forms a data redundancy backup relationship with the master device and other slave devices.

[0139] Optionally, in the embodiment, the storage cluster may, but is not limited to, be a set of storage devices composed of one master device and at least one slave device, the devices in the cluster are interconnected through a high-speed network, and the storage resources are collaboratively scheduled, data redundancy is backed up, and load balancing is achieved under the unified management of the master device, which is a hardware basis for generating a virtual storage unit.

[0140] Optionally, in the embodiment, the device in the physical space where the device density meets the specific condition is determined as the candidate device. Then, each candidate device is traversed, and the master device and at least one slave device are determined from the current candidate device currently traversed. And the storage cluster corresponding to the current candidate device is constructed based on the master device and at least one slave device.

[0141] It can be understood that, in the candidate device screening process, the devices with insufficient capacity and physical dispersion are excluded, the high-performance master device and sufficient slave devices are selected in the master-slave device screening link, so that the storage cluster meets the conditions of sufficient storage capacity, efficient processing capability and centralized physical deployment, and the delay of data transmission in the cluster is reduced; at the same time, the master-slave device enables the cluster to have load balancing and fault redundancy capability, the master device can be switched in the event of failure, and other slave devices are determined as new master devices, the slave device failure can be recovered through other slave devices, and the risk of storage task failure is reduced.

[0142] Through the embodiments provided in the application, the storage device with the remaining storage capacity meeting the remaining capacity condition and located in the target space is determined as the candidate device, wherein the target space is a region where the device density meets the density threshold condition, and the device density is used to indicate the number of storage devices in a unit space; each candidate device is traversed, and the master device and at least one slave device are determined from the current candidate device currently traversed and other candidate devices located in the adjacent space, wherein the current candidate device can communicate with other storage devices located in the adjacent space; and the storage cluster corresponding to the current candidate device is constructed based on the master device and at least one slave device, wherein the storage cluster is used to determine at least one virtual storage unit. In the candidate device screening process, the devices with insufficient capacity and physical dispersion are excluded, the high-performance master device and sufficient slave devices are selected in the master-slave device screening link, so that the storage cluster meets the conditions of sufficient storage capacity, efficient processing capability and centralized physical deployment, and the delay of data transmission in the cluster is reduced; at the same time, the master-slave device enables the cluster to have load balancing and fault redundancy capability, the master device can be switched in the event of failure, and other slave devices are determined as new master devices, the slave device failure can be recovered through other slave devices, and the risk of storage task failure is reduced.

[0143] As an optional solution, the master device and at least one slave device are determined from the current candidate device currently traversed and other candidate devices located in the adjacent space, including at least one of the following:

[0144] S1, the master device is determined based on the current candidate device and the candidate broadcast issued by other candidate devices in the adjacent space, and other storage devices except the master device are determined as slave devices, wherein the candidate broadcast is used to indicate the time of becoming a candidate device;

[0145] S2, determining the master device based on the network bandwidth of the current candidate device and other candidate devices in the adjacent space, and determining other storage devices except the master device as slave devices;

[0146] S3, determining the master device based on the failure rate of the current candidate device and other candidate devices in the adjacent space, and determining other storage devices except the master device as slave devices.

[0147] Optionally, in the embodiment, the candidate broadcast can be, but is not limited to, a communication signal sent by the current candidate device and other candidate devices in the adjacent space to the surrounding devices when participating in the master device competition, containing the time when the candidate device becomes the candidate device.

[0148] Further, it is assumed that the candidate device m and the candidate device n compete for the master node, and the candidate device m and the candidate device n will broadcast in the adjacent area, but due to the coverage radius limitation, the candidate device m can receive the broadcast information of the candidate device n, and the candidate device n cannot receive the broadcast information of the candidate device m. When the candidate device m receives the broadcast information of the candidate device n, the broadcast information sending time of the candidate device n is compared with the broadcast information sending time of the candidate device m.

[0149] If the broadcast information sending time of the candidate device n is earlier than the broadcast information sending time of the candidate device m, the candidate device n is received as the master node, and the candidate device m, the candidate device n and the storage devices in the adjacent area of the candidate device n constitute a storage cluster system.

[0150] If the broadcast information sending time of the candidate device n is later than the broadcast information sending time of the candidate device m, the candidate device m considers itself as the master node; the candidate device n cannot receive the broadcast information of the candidate device m and also considers itself as the master node; the candidate device m and the candidate device n become the master nodes in their respective storage cluster systems.

[0151] Optionally, in the embodiment, the network bandwidth can be, but is not limited to, the maximum data transmission rate when the current candidate device and other candidate devices in the adjacent space transmit data with external devices, which is a core index for measuring the data transmission capacity of the device.

[0152] Further, it is assumed that there are a current candidate device X and candidate devices Y and Z in the adjacent space; wherein, the network bandwidth of the current candidate device X is the largest, the current candidate device X is determined as the master device; the candidate device Y and the candidate device Z are determined as slave devices because their bandwidths are lower than that of the current candidate device X.

[0153] Optionally, in the embodiment, the failure rate can be, but is not limited to, a probability of failure of the current candidate device and other candidate devices in the adjacent space within a preset statistical period, which is usually obtained by a ratio of the number of failures to the total running time or a ratio of the failure time to the total running time, and can reflect the stability level of the device, and can be obtained by statistical data such as device failure time and failure times recorded by the system.

[0154] It can be understood that the determination manners of the three master devices correspond to scenarios of efficiency priority, performance priority, and stability priority respectively, for example, in a high-frequency storage scenario, the network bandwidth dimension can be selected to ensure fast data transmission, in a key business storage scenario, the failure rate dimension can be selected to ensure cluster stability, and in an ordinary data storage scenario, the candidate broadcast time dimension can be selected to ensure fast cluster construction, so that different storage requirements in a heterogeneous computing environment can be met.

[0155] According to the embodiments provided in the application, the master device is determined based on the candidate broadcast issued by the current candidate device and other candidate devices in the adjacent space, and the other storage devices except the master device are determined as slave devices, wherein the candidate broadcast is used to indicate the time of becoming a candidate device; the master device is determined based on the network bandwidth of the current candidate device and other candidate devices in the adjacent space, and the other storage devices except the master device are determined as slave devices; the master device is determined based on the failure rate of the current candidate device and other candidate devices in the adjacent space, and the other storage devices except the master device are determined as slave devices. Different master device determination manners are selected in different scenarios, so that different storage requirements in a heterogeneous computing environment can be met.

[0156] As an optional solution, before the load description information and the resource description information corresponding to each of the at least one virtual storage unit matched with the target terminal are acquired, the method further includes:

[0157] S1, acquiring a data transmission rate corresponding to each of the storage clusters for a fifth virtual storage unit in the at least one virtual storage unit, wherein the data transmission rate corresponding to the storage cluster for the fifth virtual storage unit is used to indicate a time required for the fifth virtual storage unit to perform unit capacity data transmission with the storage cluster;

[0158] S2, establishing a mapping relationship between the fifth virtual storage unit and the storage cluster corresponding to the maximum data transmission rate.

[0159] Optionally, in the embodiment, the fifth virtual storage unit can be, but is not limited to, a specific virtual storage unit in the at least one virtual storage unit, which is to be established a mapping relationship with the storage cluster.

[0160] Optionally, in this embodiment, the data transmission rate can be, but is not limited to, an efficiency indicator when the fifth virtual storage unit transmits data with a certain storage cluster, and can be, but is not limited to, used to indicate the time required for the two to transmit a unit of capacity data.

[0161] Optionally, in this embodiment, the mapping relationship can be, but is not limited to, a logical association between the fifth virtual storage unit and a specific storage cluster established by the protocol layer. Through this relationship, the storage task of the fifth virtual storage unit can be directly routed to the corresponding storage cluster, and the storage cluster can also feed back the data read-write state to the fifth virtual storage unit, thereby providing a communication link between the application layer virtual storage unit and the hardware layer storage cluster, and enabling the storage requirement of the fifth virtual storage unit to be converted into a storage operation of the storage device in the storage cluster.

[0162] Optionally, in this embodiment, first, a unit of test data is respectively sent between the fifth virtual storage unit and each storage cluster through a transmission rate test tool; and the time required for each transmission of a unit of data is recorded, which is converted into a data transmission rate, and finally the transmission rate data corresponding to the fifth virtual storage unit and each storage cluster is obtained.

[0163] Then, the maximum transmission rate is selected through numerical comparison from the obtained data transmission rates of the fifth virtual storage unit and each storage cluster; then the specific storage cluster corresponding to the maximum rate is determined; and finally, a logical association is established between the fifth virtual storage unit and the storage cluster through the mapping management module of the protocol layer, so as to ensure that the subsequent storage task of the fifth virtual storage unit can be automatically routed to the storage cluster, and efficient storage of data is realized.

[0164] It can be understood that, through rate testing and optimal selection, the fifth virtual storage unit is matched to the storage cluster with the fastest data transmission, the data transmission delay is maximally reduced, the storage task is prevented from being stalled due to low transmission efficiency, and the overall response speed of the storage system is improved.

[0165] Through the embodiments provided in this application, the data transmission rate corresponding to each storage cluster in at least one virtual storage unit is obtained, wherein the data transmission rate corresponding to the fifth virtual storage unit and the storage cluster is used to indicate the time required for the fifth virtual storage unit and the storage cluster to transmit a unit of capacity data; and the storage cluster corresponding to the maximum data transmission rate is established with the fifth virtual storage unit. Through rate testing and optimal selection, the fifth virtual storage unit is matched to the storage cluster with the fastest data transmission, the data transmission delay is maximally reduced, the storage task is prevented from being stalled due to low transmission efficiency, and the overall response speed of the storage system is improved.

[0166] As an optional solution, the target storage unit is determined from the whole set of storage units, comprising:

[0167] According to the priority coefficient of each virtual storage unit, the whole set of storage units is traversed to perform the following operations:

[0168] S1, in the case that there is a free storage unit in the current set of storage units, the free storage unit is determined as the target storage unit, wherein the free storage unit is a virtual storage unit not in working state;

[0169] S2, in the case that there is no free storage unit in the current set of storage units, the next set of storage units is determined as the current set of storage units.

[0170] Optionally, in the embodiment, the free storage unit can be, but is not limited to, a virtual storage unit not in working state in the set of storage units, which can be, but is not limited to, understood as a unit not currently undertaking any data read / write task, and the central processing unit load, memory load and IO load are all at a low level, and the unit has the ability to immediately undertake a new storage task.

[0171] Optionally, in the embodiment, the priority coefficient of each virtual storage unit in the whole set of storage units is first extracted, and the whole set of storage units is sorted; then, according to the sorting result, each set of storage units is accessed in turn from the set of storage units with the highest priority; and it is judged whether there is a free storage unit not in working state; if there is, the storage unit with the highest priority coefficient is selected from the free storage units and determined as the target storage unit, and the screening process is completed.

[0172] If it is found that all virtual storage units in the current set of storage units are in working state, the next set of storage units of the current set of storage units, which can be, but is not limited to, understood as the set with the second highest priority, is updated as the new current set of storage units, and it is rechecked whether there is a free storage unit in the new current set.

[0173] It can be understood that according to the priority coefficient of the virtual storage unit, the whole set of storage units is sorted in descending order of adaptability, and then each set of storage units is traversed in this order; if there is a free storage unit not in working state in the current set, it is determined as the target storage unit; if there is no free unit in the current set, the current set is updated as the next set of storage units in order, and the checking operation is repeated until the target storage unit is found, by always preferentially screening the free unit from the high-priority set of storage units, it is ensured that the determined target storage unit is in idle state and has high adaptability, and the execution efficiency and stability of the storage task are improved.

[0174] In the case that there is an idle storage unit in the current storage unit group, the idle storage unit is determined as the target storage unit, wherein the idle storage unit is a virtual storage unit not in a working state; in the case that there is no idle storage unit in the current storage unit group, the next storage unit group is determined as the current storage unit group.

[0175] As an optional solution, in order to better understand the process of the above data storage method, the following describes the flow of the data storage method in combination with optional embodiments, but is not used to limit the technical solutions of the embodiments of the present application.

[0176] The embodiment provides a storage device cooperative management method and system based on heterogeneous computing. The embodiment solves the problems of hardware interconnection, protocol compatibility, resource virtual integration and efficient scheduling of a storage system in a heterogeneous computing environment by constructing a four-layer architecture including a terminal layer, an application layer, a protocol layer and a hardware layer, realizes deep cooperation of storage resources and computing resources, and provides bottom support for high-performance heterogeneous computing.

[0177] Step 1: Construct a four-layer architecture of the storage device management system: terminal layer, application layer, protocol layer and hardware layer;

[0178] The terminal layer is composed of operation devices and edge storage devices directly facing users, such as display screens, keyboards, mice and user local storage controllers; the user local storage controller is mainly a lightweight storage controller responsible for preprocessing and caching local data;

[0179] The application layer is responsible for interconnecting the storage devices in the lower layer through a high-speed network to form a virtual storage resource pool to meet the needs of the heterogeneous devices in the terminal layer; the smallest management unit of the virtual storage resource pool is a virtual storage unit; for example, virtual storage units are dynamically allocated according to the storage needs of the terminal layer to realize efficient use of storage resources;

[0180] The protocol layer supports protocol compatibility and is responsible for mapping the storage cluster systems with different protocols from the hardware layer to the virtual storage units;

[0181] The hardware layer supports the interconnection of heterogeneous storage medium interfaces and supports CXL3.0 and PCIe6.0 hybrid interfaces; it is responsible for constructing storage devices with heterogeneous storage medium interfaces into a storage cluster system for management.

[0182] Step 2: Set priorities for the virtual storage units in the application layer according to load and energy efficiency;

[0183] The load of the virtual storage unit is related to CPU load, memory load and storage IO load; the energy efficiency of the virtual storage unit is related to CPU energy efficiency and memory energy efficiency;

[0184] The specific calculation method is as follows: In the application layer, the virtualization platform monitors the CPU load, memory load and storage I / O load of the virtual storage unit in real time. The load index of the virtual storage unit is expressed as shown in formula (1):

[0185] (1);

[0186] in, Represents virtual storage unit The load index, Indicates CPU load factor; Represents virtual storage unit The CPU load is obtained by calculating the average CPU utilization per unit time. Indicates the memory load factor. Represents virtual storage unit The memory load is obtained by calculating the proportion of used memory to the total allocated memory. Indicates the storage I / O load factor. Represents virtual storage unit The storage I / O load is obtained by calculating the amount of data read and written per unit time; the CPU load coefficient, memory load coefficient, and storage I / O load coefficient satisfy... ;

[0187] In the application layer, the virtualization platform monitors the CPU and memory energy efficiency of the virtual storage unit in real time. The energy efficiency index of the virtual storage unit is expressed as shown in formula (2):

[0188] (2);

[0189] in, Represents virtual storage unit Energy efficiency indicators Indicates the CPU energy efficiency coefficient. Represents virtual storage unit CPU energy efficiency is obtained by calculating the effective processing capacity of a unit of CPU resources within a virtual memory unit. Among these... The method for obtaining it is shown in formula (3):

[0190] (3);

[0191] Indicates the memory energy efficiency coefficient. Represents virtual storage unit The memory efficiency is obtained by calculating the effective utilization rate of virtual memory unit memory resources. The above coefficients are specified according to the CPU model, and can also be considered as determined by experiments. The acquisition mode is shown as formula (4):

[0192] (4);

[0193] It should be noted that, within a specified time, or said running time, the data in the memory which is read, written, modified more than the threshold value is called active data; calculating the data amount (such as how many b, how many kb of data) of this part of active data according to the storage space is the active data amount.

[0194] To the virtual storage unit , based on the load index and energy efficiency index, the priority function is calculated, as shown in formula (5):

[0195] (5);

[0196] Among them, represents the priority function of the virtual storage unit , represents the minimum value of the load index of the virtual storage unit, represents the maximum value of the load index of the virtual storage unit, represents the minimum value of the energy efficiency index of the virtual storage unit, represents the maximum value of the energy efficiency index of the virtual storage unit;

[0197] and are the load index and energy efficiency index calculated according to the current virtual unit.

[0198] Through the above calculation method, the priority function of all virtual storage units is obtained.

[0199] The third step: when the terminal has data to be stored, the storage is carried out according to the priority of the virtual storage unit, as follows:

[0200] When the terminal has data to be stored, the virtual storage units directly connected with the terminal are sorted in descending order according to the priority function, and an ordered set is constructed, wherein, represents the virtual storage units directly connected with the terminal sorted in descending order according to the priority function, represents the number of virtual storage units directly connected with the terminal;

[0201] Taking each virtual storage unit directly connected with the terminal as the starting point, a group sequence is generated.

[0202] When the terminal has data to be stored, the terminal selects a group sequence Begin by examining each group sequence in turn:

[0203] S1 terminal pair group Check each element in sequence. ;

[0204] S2 if If available, select Store the data and mark it as busy;

[0205] S3 if If busy, continue with the inspection team. The next element inside;

[0206] If the current group If all elements within the sequence are busy, then proceed to the next priority group sequence. Repeat step S1;

[0207] If all elements of all groups are busy, the terminal enters a waiting state until a virtual memory unit becomes free.

[0208] Among them, group sequence The generation method is as follows: group The first element is the virtual storage unit directly connected to the terminal. Then, a recursive method is used to find virtual memory units with higher priority for the group. To expand;

[0209] Among them, group The specific methods for expansion are as follows:

[0210] S11: From group The last element Depart, gather at its neighbors In the middle, find all those with higher priority than And not joined the group Virtual storage units;

[0211] S12: If multiple virtual storage units meet the criteria, select the virtual storage unit with the highest priority and add it to the group. The end;

[0212] S13: Repeat step S11 until the group... The last element Of all the neighbors in the set, there is no higher priority neighbor that is not in the group. Up to the point of virtual storage units;

[0213] in, Neighbor set It refers to and a set of virtual storage units connected by direct routing;

[0214] For further illustration, assume that there are three virtual storage units with priorities of 3, 2 and 1 in the set, and that the three virtual storage units with priorities of 3, 2 and 1 are respectively taken as the first elements to form three groups when the groups are formed. Now, group 1 is {3}, group 2 is {2}, and group 3 is {1}.

[0215] Then, for group 1, starting from the last element 3, find the virtual storage units 4, 5 and 6 in the neighbor set of 3 which have higher priorities than 3 and have not been added to group 1, select the one with the highest priority 6 to add to group 1, and group 1 is {3, 6}. Then, starting from the last element 6, find the virtual storage units in the neighbor set of 6 which have higher priorities than 6 and have not been added to group 1, until there is no virtual storage unit in the neighbor set of the last element of group 1 which has a higher priority and has not been added to group 1, and group 1 is updated.

[0216] Then, for group 2, repeat the above process, starting from the last element 2, find the virtual storage units 4 and 7 in the neighbor set of 2 which have higher priorities than 2 and have not been added to group 1 (the neighbor set may have duplicates, and the virtual storage units meeting the condition can be added to multiple groups), select the one with the highest priority 7 to add to group 2, and group 2 is {2, 7}. Then, starting from the last element 7, find the virtual storage units in the neighbor set of 7 which have higher priorities than 7 and have not been added to group 2, until there is no virtual storage unit in the neighbor set of the last element of group 2 which has a higher priority and has not been added to group 2, and group 2 is updated.

[0217] Then, for group 3, repeat the above process, starting from the last element 1, find the virtual storage unit 6 in the neighbor set of 1 which has a higher priority than 1 and has not been added to group 3 (the neighbor set may have duplicates, and the virtual storage units meeting the condition can be added to multiple groups), select the one with the highest priority 6 to add to group 3, and group 3 is {1, 6}. Then, starting from the last element 6, find the virtual storage unit in the neighbor set of 6 which has a higher priority than 6 and has not been added to group 3, until there is no virtual storage unit in the neighbor set of the last element of group 3 which has a higher priority and has not been added to group 3, and group 3 is updated.

[0218] Fourth step: hardware layer, build a storage cluster system based on storage devices;

[0219] One storage device in each storage cluster system is selected as a master node, and other storage devices are slave nodes;

[0220] The master node is responsible for data transmission with the virtual storage unit and stores data of the virtual storage unit;

[0221] The master node sends data of the virtual storage unit to a slave node inside the storage cluster system, and the slave node stores the data;

[0222] Master node determination process in the storage cluster system:

[0223] First, the candidate nodes in the storage cluster system are determined according to the storage device density and the remaining storage capacity of the surrounding storage devices;

[0224] For the storage device , a threshold function is set, and when , it is shown as formula (6):

[0225] (6);

[0226] For the storage device , a threshold function is set, and when , it is shown as formula (7):

[0227] (7);

[0228] wherein and represent the storage device density coefficient and the surrounding storage device remaining storage capacity coefficient respectively, represents the device density at the location of the storage device , represents the maximum device density at the location of the storage device, represents the remaining storage capacity of the storage device , and represents the average remaining storage capacity of the surrounding storage devices of the storage device . The device density is calculated according to how many devices are in a fixed physical space, that is, the device density. A random number between and

[0229] is assigned to the storage device ; if the random number is less than , the storage device is marked as a candidate node; or the greater the density of the storage device, the greater the remaining storage capacity, the greater the threshold value, and the higher the probability of being a candidate node.

[0230] ​Then, each candidate node competes with candidate nodes in the adjacent area for the master node;

[0231] When a certain storage device is selected as a candidate node, a broadcast is performed in the adjacent area;

[0232] If other storage devices receive the broadcast message of the candidate node, the candidate node is confirmed as the cluster master node, and the other storage devices do not convert into candidate nodes;

[0233] If other candidate nodes receive the broadcast message of the candidate node, the sending time is compared; the candidate node with the earliest sending time is selected as the cluster master node; the remaining candidate nodes exit the competition and do not perform the broadcast.

[0234] candidate node communication link length of the candidate node

[0235] (8);

[0236] The candidate node is considered to be in the adjacent area with the candidate node ; wherein, denotes the coverage radius of the candidate node , denotes the coverage radius of the candidate node , denotes the maximum value, denotes the communication link length of the candidate node and the candidate node ;

[0237] wherein, the coverage radius of the candidate node is calculated as shown in formula (9):

[0238] (9);

[0239] wherein, denotes the maximum communication link length between the candidate nodes, denotes the device density at the location of the candidate node , denotes the remaining storage capacity of the candidate node , denotes the maximum value of the remaining storage capacity of the candidate node, denotes the minimum value of the remaining storage capacity of the candidate node, denotes the maximum coverage radius, , , ​respectively represent the candidate node communication distance coefficient, the device density coefficient and the storage capacity coefficient;

[0240] According to the number of devices in the fixed physical space, the device density is calculated, that is, the placement density of the devices; the maximum residual storage capacity is the maximum residual storage capacity in the storage devices corresponding to the candidate nodes; the minimum residual storage capacity is the minimum residual storage capacity in the storage devices corresponding to the candidate nodes;

[0241] The plurality of storage devices are physically connected, and then the storage cluster management software is used to group the connected plurality of storage devices into a storage cluster system. There is one master node in a storage cluster system, and other storage devices are slave nodes. The storage cluster system communicates externally through the master node.

[0242] A plurality of storage devices are pre-placed, and then a plurality of candidate nodes are determined from the storage devices. The plurality of candidate nodes compete for the master node, and the storage device that competes for the master node broadcasts the information that it is the master node to the adjacent area. The storage device corresponding to the master node and all storage devices in the adjacent area jointly form a storage cluster system. There are a plurality of storage cluster systems.

[0243] The master node is at the hardware layer, and the group is seen at the application layer. The virtual storage unit stored in the group is given a mapping relationship based on the routing information between the storage cluster system and the virtual storage unit at the fifth step protocol layer, that is, the master node and the virtual storage unit are bound.

[0244] Step 5: The protocol layer establishes a mapping between the storage cluster system with different protocols from the hardware layer and the virtual storage unit, and the content of the virtual storage unit is stored by the corresponding storage cluster system;

[0245] According to the speed of the virtual storage unit transmitting data to the storage cluster system, the storage cluster system with the maximum data transmission speed is selected for each virtual storage unit.

[0246] A mapping function is established based on the routing information (link capacity, link length) between the storage cluster system and the virtual storage unit, as shown in formula (10):

[0247] (10);

[0248] wherein, the mapping function between the storage cluster system and the virtual storage unit, the minimum value of the function, the node pair composed of the virtual storage unit and the storage cluster system, the node pair set, a rate at which data from the virtual storage unit is stored into the storage cluster system in a node pair, a storage link of the virtual storage unit and the storage cluster system, a set of storage links, a storage link a storage link capacity of the storage link, a packet data, all data to be stored, a packet data an arrival rate of the packet data, an indicator function is denoted;

[0249] a constraint condition is set, as shown in expression (11):

[0250] (11)

[0251] a mapping of the virtual storage unit and the storage device cluster is determined by optimizing .

[0252] When the terminal has data to be stored, the data is stored according to the priority of the virtual storage unit;

[0253] The design experiment proves that the storage method proposed in the embodiment is superior to other methods; the other methods include a traditional method, a breadth-first method, and a random method.

[0254] In the traditional method, the terminal only accesses the directly connected virtual storage unit, selects according to the priority from high to low, and does not search for indirectly connected virtual storage units through routing expansion. In the breadth-first algorithm, the priority is not considered, and all reachable virtual storage units are searched according to the level (breadth-first) from the virtual storage unit directly connected to the terminal, and the idle virtual storage unit found first is selected preferentially. In the random method, a virtual storage unit is randomly selected from all reachable virtual storage units of the terminal, and the priority and distance are not considered.

[0255] In addition, the parameters include the number of virtual storage units, the priority, the connection relationship, the busy state, and the number of storage requests, wherein the number of virtual storage units is 50, which is divided into virtual storage units directly connected to the terminal and virtual storage units indirectly connected to the terminal; the priority is randomly generated for the virtual storage unit; the connection relationship is that the average connection degree of each virtual unit is 3, forming an undirected connected graph; the busy state is that each virtual storage unit is in a busy state with a probability of 0.6 when a storage request arrives; and the number of storage requests is 1000 times of Monte Carlo simulation, each time being independent.

[0256] The evaluation indexes adopt response time and invalid search rate; wherein, the response time represents the search step number from the terminal initiating the storage request to finding the idle virtual storage unit; the invalid search rate represents the proportion of the busy virtual storage unit checked in the search process;

[0257] In the obtained data, the response time of the method of the embodiment is 8, the invalid search rate is 20%, the response time of the traditional method is 14, the invalid search rate is 48%, the response time of the breadth priority method is 15, the invalid search rate is 46%, the response time of the random method is 21, and the invalid search rate is 63%.

[0258] It is shown from the experiment that the response time required by the method of the embodiment is the shortest, and the invalid search rate is the lowest. It is shown that the method of the embodiment can help the terminal to quickly find the idle virtual storage unit for storage.

[0259] Through the embodiments of the present application, for the complex scene of storage device cooperative management in a heterogeneous computing environment, through the innovative design of the four-layer architecture and the intelligent scheduling mechanism, efficient integration and cooperative optimization of storage resources are realized; through the construction of a virtual storage resource pool in the application layer, heterogeneous storage media (such as SSD, HDD, NVMe, etc.) and storage cluster systems (such as NFS, SMB, S3) of different protocols in the hardware layer are uniformly abstracted as virtual storage units, and the compatibility problem of heterogeneous devices in the terminal layer is completely solved. Compared with the traditional heterogeneous management scheme, the terminal device access time is greatly shortened, and the adaptation cost of multi-device cooperation is significantly reduced; the application layer dynamically sets the priority of the virtual storage unit based on the load pressure and the energy efficiency ratio, and the terminal preferentially selects a high-performance unit (taking into account the priority and energy consumption) for storage, avoiding the problem of hotspot overload or low-efficiency resource idling caused by the traditional fixed priority strategy. The actual measurement shows that in the peak load scene, the median of the storage request response time is reduced from 80ms to 35ms, the resource utilization rate is increased by 65%, and the system performance and energy consumption are effectively balanced. The protocol layer establishes a dynamic mapping relationship between the storage cluster system and the virtual storage unit, and optimizes the data transmission path in real time based on the network routing state, which significantly reduces the data interaction overhead across protocols and media. Compared with the traditional fixed mapping scheme, the data transmission time delay is reduced, the average network traffic of a single node is reduced, and the total data transmission cost is reduced. The hardware layer constructs heterogeneous storage media into a high-reliability cluster management system, and realizes safe data storage without affecting normal business through a multi-copy redundancy mechanism. Compared with the traditional homogeneous backup scheme, the data backup time of the cluster management system of the present application is greatly reduced.

[0260] Through the description of the above embodiments, those skilled in the art can clearly understand that the method according to the above embodiments can be realized by means of software and the necessary general hardware platform, of course, it can also be realized by hardware, but in many cases, the former is a better embodiment.

[0261] Based on such understanding, the technical solutions of the present application, essentially or in other words, the part that contributes to the related art, can be embodied in the form of a software product. The computer software product is stored in a storage medium (such as a ROM / RAM, a magnetic disk, or an optical disc) and includes a plurality of instructions that cause a terminal device (which can be a mobile phone, a computer, a server, or a network device) to execute the methods of various embodiments of the present application.

[0262] In this embodiment, a data storage device is also provided, which is used to implement the above-mentioned embodiments and preferred embodiments, and will not be described again. As used below, the term "module" can be a combination of software and / or hardware that implements a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, hardware or a combination of software and hardware is also possible and is contemplated.

[0263] Figure 3 is a structural block diagram of a data storage device according to an embodiment of the present application; as shown in Figure 3 comprises:

[0264] The first acquisition unit 302 is configured to acquire load description information and resource description information corresponding to each of the at least one virtual storage unit matched with the target terminal, wherein the load description information is used to indicate the load condition of the virtual storage unit when performing a data read / write task, and the resource description information is used to indicate the read / write efficiency of the virtual storage unit when performing the data read / write task.

[0265] The first determination unit 304 is configured to determine the priority coefficient of each of the at least one virtual storage unit according to the load description information and the resource description information.

[0266] The adding unit 306 is configured to, in a case where at least one first virtual storage unit is determined from the at least one virtual storage unit based on the priority coefficient, add at least one second virtual storage unit having a routing connection with the first virtual storage unit and the at least one first virtual storage unit to the storage unit group.

[0267] The second determination unit 308 is configured to determine a target storage unit from the entire storage unit group, wherein the target storage unit is used to store the currently acquired object data.

[0268] As an optional solution, the adding unit 306 comprises: a first traversing module, configured to traverse the at least one virtual storage unit according to the priority order indicated by the priority coefficient; a first obtaining module, configured to obtain a first virtual storage unit from the at least one virtual storage unit; a second obtaining module, configured to obtain a neighbor unit set corresponding to the first virtual storage unit, wherein the neighbor unit set comprises at least one neighbor storage unit having a routing connection with the first virtual storage unit; a first determining module, configured to determine, from the at least one neighbor storage unit, a neighbor storage unit having a maximum priority indicated by the priority coefficient as a current neighbor storage unit; and a second determining module, configured to determine the current neighbor storage unit as a second virtual storage unit in a case where a second priority indicated by the priority coefficient of the current neighbor storage unit is greater than a first priority indicated by the priority coefficient of the first virtual storage unit, and add the first virtual storage unit and the second virtual storage unit to the storage unit group.

[0269] As an optional solution, the second determining module comprises: a first determining submodule, configured to determine the second virtual storage unit as a current virtual storage unit; a first obtaining submodule, configured to obtain a neighbor unit set corresponding to the current virtual storage unit, wherein the neighbor unit set comprises at least one neighbor storage unit having a routing connection with the current virtual storage unit; a first determining submodule, configured to determine, from the at least one neighbor storage unit, a neighbor storage unit having a maximum priority indicated by the priority coefficient as a current neighbor storage unit; and a second determining submodule, configured to determine the current neighbor storage unit as a third virtual storage unit in a case where a third priority indicated by the priority coefficient of the current neighbor storage unit is greater than a fourth priority indicated by the priority coefficient of the current virtual storage unit, and add the third virtual storage unit to the storage unit group.

[0270] As an optional solution, the first obtaining unit 302 comprises: a third obtaining module, configured to obtain, based on central processor load information, memory load information and read-write load information of a fourth virtual storage unit in the at least one storage unit, a load coefficient corresponding to the fourth virtual storage unit, wherein the central processor load information is used to indicate an average utilization rate of a central processor of the fourth virtual storage unit in a unit time, the memory load information is used to indicate a remaining available memory of the fourth virtual storage unit, the read-write load information is used to indicate a read-write data volume of the fourth virtual storage unit in a unit time, and the load description information comprises the load coefficient.

[0271] As an optional solution, the third obtaining module comprises: a second obtaining submodule, configured to obtain a first load weight, a second load weight and a third load weight based on the central processor type corresponding to the fourth virtual storage unit, wherein the first load weight is used to indicate the importance of the central processor load information, the second load weight is used to indicate the importance of the memory load information, and the third load weight is used to indicate the importance of the read-write load information, and the sum of the first load weight, the second load weight and the third load weight satisfies a preset threshold condition; and a first summation submodule, configured to perform weighted summation based on the first load weight, the second load weight and the third load weight, the central processor load information, the memory load information and the read-write load information, to obtain the load coefficient corresponding to the fourth virtual storage unit.

[0272] As an optional solution, the first obtaining unit 302 comprises: a fourth obtaining module, configured to obtain the energy efficiency coefficient corresponding to the fourth virtual storage unit based on the central processor energy efficiency information and the memory energy efficiency information of the fourth virtual storage unit, wherein the central processor energy efficiency information is used to indicate the processing capability of the fourth virtual storage unit on a unit central processor, and the memory energy efficiency information is used to indicate the utilization rate of the fourth virtual storage unit on a unit memory resource, and the resource description information comprises the energy efficiency coefficient.

[0273] As an optional solution, the fourth obtaining module comprises: a third determination submodule, configured to determine the product of the number of central processor cores allocated for the fourth virtual storage unit and a preset time as the resource size value; a fourth determination submodule, configured to determine the ratio of the number of tasks completed by the central processor for the fourth virtual storage unit in a preset time to the resource size value as the central processor energy efficiency information; a fifth determination submodule, configured to determine the product of the memory capacity allocated for the fourth virtual storage unit and the preset time as the temporary storage size value; and a sixth determination submodule, configured to determine the ratio of the active data amount of the fourth virtual storage unit in the preset time to the temporary storage size value as the memory energy efficiency information, wherein the active data amount is the actual storage occupation capacity of the data whose execution read-write operation times satisfy a times threshold condition.

[0274] As an optional solution, the fourth obtaining module comprises: a third obtaining submodule, configured to obtain a first energy efficiency weight and a second energy efficiency weight based on the central processor type corresponding to the fourth virtual storage unit, wherein the first energy efficiency weight is used to indicate the importance of the central processor energy efficiency information, and the second energy efficiency weight is used to indicate the importance of the memory energy efficiency information; and a second summation submodule, configured to perform weighted summation on the central processor energy efficiency information and the memory energy efficiency information based on the first energy efficiency weight and the second energy efficiency weight, to obtain the energy efficiency coefficient.

[0275] As an optional solution, the apparatus further comprises: a third determining unit, configured to determine, as a candidate device, a storage device that satisfies a residual capacity condition and is located in a target space, wherein the target space is a region in which a device density satisfies a density threshold condition, and the device density is used to indicate a number of storage devices in a unit space; a fourth determining unit, configured to determine, from a current candidate device currently being traversed and other candidate devices located in adjacent spaces, a master device and at least one slave device; and a constructing unit, configured to construct, based on the master device and the at least one slave device, a storage cluster corresponding to the current candidate device, wherein the storage cluster is used to determine at least one virtual storage unit.

[0276] As an optional solution, the fourth determining unit comprises: a third determining module, configured to determine, based on a candidate broadcast sent by the current candidate device and other candidate devices in the adjacent spaces, the master device, and determine, as the slave devices, the other storage devices except the master device, wherein the candidate broadcast is used to indicate a time of becoming a candidate device; a fourth determining module, configured to determine, based on network bandwidths of the current candidate device and other candidate devices in the adjacent spaces, the master device, and determine, as the slave devices, the other storage devices except the master device; and a fifth determining module, configured to determine, based on failure rates of the current candidate device and other candidate devices in the adjacent spaces, the master device, and determine, as the slave devices, the other storage devices except the master device.

[0277] As an optional solution, the apparatus further comprises: a second obtaining unit, configured to obtain a data transmission rate corresponding to each storage cluster respectively for a fifth virtual storage unit in the at least one virtual storage unit, wherein the data transmission rate corresponding to the storage cluster for the fifth virtual storage unit is used to indicate a time required for the fifth virtual storage unit and the storage cluster to perform unit capacity data transmission; and a establishing unit, configured to establish a mapping relationship between the storage cluster corresponding to the maximum data transmission rate and the fifth virtual storage unit.

[0278] As an optional solution, the second determining unit 308 comprises: a second traversing module, configured to traverse all storage unit groups according to respective priority coefficients of the at least one virtual storage unit; a sixth determining module, configured to, in a case where there is an idle storage unit in a current storage unit group, determine the idle storage unit as a target storage unit, wherein the idle storage unit is a virtual storage unit that is not in a working state; and a seventh determining module, configured to, in a case where there is no idle storage unit in the current storage unit group, determine a next storage unit group as the current storage unit group.

[0279] The features of the embodiments of the data storage apparatus can be referred to the related descriptions of the embodiments of the data storage method, which will not be repeated here.

[0280] An embodiment of the present application further provides an electronic device, Figure 4 is a schematic diagram of the electronic device according to the embodiment of the present application, as Figure 4 shown, the electronic device comprises a memory and a processor, the memory stores a computer program, and the processor is configured to execute the computer program to perform the steps in any of the above data storage method embodiments.

[0281] In an example embodiment, the above electronic device can further comprise a transmission device connected to the processor and an input / output device connected to the processor.

[0282] The specific examples in the embodiment can refer to the examples described in the above embodiments and example embodiments, which will not be described here again.

[0283] An embodiment of the present application further provides a computer readable storage medium, which stores a computer program, wherein the computer program is configured to execute the steps in any of the above data storage method embodiments when executed.

[0284] In an example embodiment, the above computer readable storage medium can include, but is not limited to, a U disk, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk, and various media that can store computer programs.

[0285] An embodiment of the present application further provides a computer program product comprising a computer program, wherein the computer program is executed by a processor to implement the steps of the method in the embodiments of the present application; the computer program product further comprises a non-volatile computer readable storage medium, the non-volatile computer readable storage medium stores the computer program, and the computer program is executed by the processor to implement the steps of the data storage method in the embodiments of the present application.

[0286] The skilled person can further realize that the units and algorithm steps of the examples described in conjunction with the embodiments disclosed herein can be realized in electronic hardware, computer software or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the components and steps of the examples have been described in the above description in general terms. Whether the functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. The skilled person can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0287] The above describes in detail the data storage method provided by the present application. The principles and implementation manners of the present application are described by using specific examples, and the above description of the embodiments is only used to help understand the method of the present application and its core idea. It should be pointed out that, for those skilled in the art, some improvements and modifications can be made to the present application without departing from the principles of the present application, and these improvements and modifications also fall within the protection scope of the claims of the present application.

Claims

1. A data storage method, characterized by, The method comprises the following steps: obtaining load description information and resource description information corresponding to each of at least one virtual storage unit matched with a target terminal, wherein the load description information is used to indicate a load condition of the virtual storage unit when performing a data read / write task, and the resource description information is used to indicate a read / write efficiency of the virtual storage unit when performing the data read / write task; determining a priority coefficient of each of the at least one virtual storage unit according to the load description information and the resource description information; in a case where at least one first virtual storage unit is determined from the at least one virtual storage unit based on the priority coefficient, adding at least one second virtual storage unit having a routing connection with the first virtual storage unit and the at least one first virtual storage unit to a storage unit group; determining a target storage unit from the entire storage unit group based on the priority coefficient, wherein the target storage unit is used to store currently acquired object data.

2. The method of claim 1, wherein, The step of adding at least one second virtual storage unit having a routing connection with the first virtual storage unit and the at least one first virtual storage unit to a storage unit group comprises the following steps: traversing the at least one virtual storage unit according to a priority order indicated by the priority coefficient, and performing the following operations: acquiring a first virtual storage unit from the at least one virtual storage unit; acquiring a neighbor unit set corresponding to the first virtual storage unit, wherein the neighbor unit set comprises at least one neighbor storage unit having a routing connection with the first virtual storage unit; determining a neighbor storage unit having a maximum priority indicated by the priority coefficient from the at least one neighbor storage unit as a current neighbor storage unit; in a case where a second priority indicated by the priority coefficient of the current neighbor storage unit is greater than a first priority indicated by the priority coefficient of the first virtual storage unit, determining the current neighbor storage unit as the second virtual storage unit, and adding the first virtual storage unit and the second virtual storage unit to the storage unit group.

3. The method of claim 2, wherein, After the current neighbor storage unit is determined as the second virtual storage unit, the method further comprises the following steps: determining the second virtual storage unit as a current virtual storage unit; acquiring a neighbor unit set corresponding to the current virtual storage unit, wherein the neighbor unit set comprises at least one neighbor storage unit having a routing connection with the current virtual storage unit; determining a neighbor storage unit having a maximum priority indicated by the priority coefficient from the at least one neighbor storage unit as a current neighbor storage unit; in a case where a third priority indicated by the priority coefficient of the current neighbor storage unit is greater than a fourth priority indicated by the priority coefficient of the current virtual storage unit, determining the current neighbor storage unit as a third virtual storage unit, and adding the third virtual storage unit to the storage unit group.

4. The method of claim 1, wherein, The step of obtaining load description information and resource description information corresponding to each of at least one virtual storage unit matched with a target terminal comprises the following steps: The load coefficient corresponding to the fourth virtual storage unit is obtained based on central processor load information, memory load information and read-write load information of the fourth virtual storage unit in the at least one storage unit, wherein the central processor load information is used to indicate the average utilization rate of the central processor of the fourth virtual storage unit in a unit time, the memory load information is used to indicate the remaining available memory of the fourth virtual storage unit, and the read-write load information is used to indicate the read-write data volume of the fourth virtual storage unit in a unit time, and the load description information comprises the load coefficient.

5. The method of claim 4, wherein, The load coefficient corresponding to the fourth virtual storage unit is obtained based on central processor load information, memory load information and read-write load information of the fourth virtual storage unit in the at least one storage unit, wherein the central processor load information is used to indicate the average utilization rate of the central processor of the fourth virtual storage unit in a unit time, the memory load information is used to indicate the remaining available memory of the fourth virtual storage unit, and the read-write load information is used to indicate the read-write data volume of the fourth virtual storage unit in a unit time, and the load description information comprises the load coefficient. The first load weight, the second load weight and the third load weight are obtained based on the central processor type corresponding to the fourth virtual storage unit, wherein the first load weight is used to indicate the importance of the central processor load information, the second load weight is used to indicate the importance of the memory load information, and the third load weight is used to indicate the importance of the read-write load information, and the sum of the first load weight, the second load weight and the third load weight satisfies a preset threshold condition; The first load weight, the second load weight and the third load weight are obtained based on the central processor type corresponding to the fourth virtual storage unit, wherein the first load weight is used to indicate the importance of the central processor load information, the second load weight is used to indicate the importance of the memory load information, and the third load weight is used to indicate the importance of the read-write load information, and the sum of the first load weight, the second load weight and the third load weight satisfies a preset threshold condition; 6. The method of claim 4, wherein, The load description information and the resource description information corresponding to at least one virtual storage unit matched with the target terminal are obtained, comprising: The energy efficiency coefficient corresponding to the fourth virtual storage unit is obtained based on central processor energy efficiency information and memory energy efficiency information of the fourth virtual storage unit, wherein the central processor energy efficiency information is used to indicate the processing capacity of the fourth virtual storage unit per unit central processor, and the memory energy efficiency information is used to indicate the utilization rate of the fourth virtual storage unit per unit memory resource, and the resource description information comprises the energy efficiency coefficient.

7. The method of claim 6, wherein, Before the energy efficiency coefficient corresponding to the fourth virtual storage unit is obtained based on the central processor energy efficiency information and the memory energy efficiency information of the fourth virtual storage unit, comprising: The product of the number of central processor cores allocated for the fourth virtual storage unit and a preset time is determined as a resource size value; The ratio of the number of tasks completed by the central processor of the fourth virtual storage unit in the preset time to the resource size value is determined as the central processor energy efficiency information; The product of the memory capacity allocated for the fourth virtual storage unit and the preset time is determined as a temporary storage size value; The ratio of the active data volume of the fourth virtual storage unit in the preset time to the temporary storage size value is determined as the memory energy efficiency information, wherein the active data volume is the actual storage occupied capacity of the data whose execution read-write operation times satisfy a times threshold condition.

8. The method of claim 6, wherein, The central processor energy efficiency information and the memory energy efficiency information of the fourth virtual storage unit are used to obtain an energy efficiency coefficient corresponding to the fourth virtual storage unit, including: The central processor type corresponding to the fourth virtual storage unit is used to obtain a first energy efficiency weight and a second energy efficiency weight, wherein the first energy efficiency weight is used to indicate the importance of the central processor energy efficiency information, and the second energy efficiency weight is used to indicate the importance of the memory energy efficiency information; The central processor energy efficiency information and the memory energy efficiency information are weighted and summed based on the first energy efficiency weight and the second energy efficiency weight to obtain the energy efficiency coefficient.

9. The method of claim 1, wherein, Before obtaining the load description information and the resource description information corresponding to each of the at least one virtual storage unit matched with the target terminal, the method further includes: A storage device with a remaining storage capacity satisfying a remaining capacity condition and located in a target space is determined as a candidate device, wherein the target space is a region with a device density satisfying a density threshold condition, and the device density is used to indicate the number of storage devices in a unit space; A master device and at least one slave device are determined from a currently traversed current candidate device and other candidate devices located in a neighboring space, wherein the current candidate device can communicate with other storage devices located in the neighboring space; A storage cluster corresponding to the current candidate device is constructed based on the master device and at least one slave device, wherein the storage cluster is used to determine at least one virtual storage unit.

10. The method of claim 9, wherein, Determining a master device and at least one slave device from a currently traversed current candidate device and other candidate devices located in a neighboring space includes at least one of the following: The master device is determined based on a candidate broadcast issued by the current candidate device and other candidate devices in the neighboring space, and other storage devices except the master device are determined as the slave devices, wherein the candidate broadcast is used to indicate the time of becoming the candidate device; The master device is determined based on the network bandwidth of the current candidate device and other candidate devices in the neighboring space, and other storage devices except the master device are determined as the slave devices; The master device is determined based on the failure rate of the current candidate device and other candidate devices in the neighboring space, and other storage devices except the master device are determined as the slave devices.

11. The method of claim 9, wherein, Before obtaining the load description information and the resource description information corresponding to each of the at least one virtual storage unit matched with the target terminal, the method further includes: A data transmission rate corresponding to each storage cluster of a fifth virtual storage unit in the at least one virtual storage unit is obtained, wherein the data transmission rate corresponding to the fifth virtual storage unit and the storage cluster is used to indicate the time required for the fifth virtual storage unit and the storage cluster to perform unit capacity data transmission; The storage cluster corresponding to the maximum data transmission rate is mapped with the fifth virtual storage unit.

12. The method according to any one of claims 1 to 11, characterized in that, The target storage unit is determined from all the storage unit groups based on the priority coefficient, including: According to the priority coefficient of each of the at least one virtual storage unit, all the storage unit groups are traversed to perform the following operations: In the case that there is a free storage unit in the current storage unit group, the free storage unit is determined as the target storage unit, wherein the free storage unit is a virtual storage unit not in a working state; In the case that there is no free storage unit in the current storage unit group, the next storage unit group is determined as the current storage unit group.

13. An electronic device, comprising: Comprise: a memory for storing a computer program; a processor for implementing the steps of the data storage method according to any one of claims 1 to 12 when executing the computer program.

Images