Data management method and related device

Abstract

The application discloses a data management method and related equipment, and relates to the technical field of data management, and comprises the following steps: in response to a data management instruction received by a first virtual machine, determining target data that needs to be managed by a second virtual machine; and calling the second virtual machine to manage the target data based on a preset network connection channel, wherein the first virtual machine and the second virtual machine are constructed based on the same chip, the preset network connection channel is constructed based on the fact that the first virtual machine and the second virtual machine are in the same network environment, the preset network connection channel is used for data transmission between the first virtual machine and the second virtual machine, the first virtual machine comprises a router module, the second virtual machine comprises a NAS module, and the data management comprises data storage and data reading. The application constructs two virtual machines based on the same chip, so that the two virtual machines replace NAS and a router respectively, a NAS system is formed only by relying on one chip, and the cost of constructing the NAS system is reduced.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data management, in particular to a data management method and related equipment. BACKGROUND

[0002] Network Attached Storage (NAS system) as an efficient data storage and sharing solution, has become the core equipment of enterprise and personal users to manage massive data. Its core function is to realize cross-platform file sharing through network protocol (such as NFS, CTF, SMB, etc.), and to centrally manage data through special hardware (such as disk array, high-speed network interface), so as to improve storage efficiency and reduce cost.

[0003] However, the implementation of the traditional NAS system needs to rely on at least one NAS and one router (two chips), which is too high in cost.

[0004] The above content is only used to assist in understanding the technical solutions of the present application, and does not represent the acknowledgement of the above content as related technology. SUMMARY

[0005] The main purpose of the present application is to provide a data management method and related equipment, aiming at solving the technical problem of high cost required by the implementation of NAS system.

[0006] In order to achieve the above purpose, the present application provides a data management method, which comprises:

[0007] In response to the data management instruction received by the first virtual machine, the target data to be managed by the second virtual machine is determined;

[0008] Based on the preset network connection channel, the second virtual machine is called to manage the target data, wherein the first virtual machine and the second virtual machine are constructed based on the same chip, the preset network connection channel is constructed based on the fact that the first virtual machine and the second virtual machine are in the same network environment, the preset network connection channel is used for data transmission between the first virtual machine and the second virtual machine, the first virtual machine includes a router module, the second virtual machine includes a NAS module, and the data management includes data storage and data reading.

[0009] In an embodiment, before the step of calling the second virtual machine to manage the target data based on the preset network connection channel, the method further comprises:

[0010] Based on the data management instruction, the current network traffic and the management demand are determined, and the management demand includes storage demand and reading demand;

[0011] predict a first hardware resource occupation rate corresponding to the first virtual machine and a second hardware resource occupation rate corresponding to the second virtual machine based on the current network traffic, the management requirement, and a preset prediction model, where the preset prediction model is used to predict a hardware resource occupation rate corresponding to a virtual machine when the virtual machine executes a storage task;

[0012] determine whether the first hardware resource occupation rate and the second hardware resource occupation rate are greater than a preset hardware occupation threshold, and if so, modify system parameters corresponding to the first virtual machine and the second virtual machine to adjust hardware resource configurations corresponding to the first virtual machine and the second virtual machine until the first hardware resource occupation rate and the second hardware resource occupation rate are less than or equal to the preset hardware occupation threshold, to obtain an adjusted first virtual machine and an adjusted second virtual machine;

[0013] The step of calling the second virtual machine to manage the target data based on the preset network connection channel further includes:

[0014] calling the adjusted second virtual machine to manage the target data based on the preset network connection channel.

[0015] In an embodiment, the step of calling the second virtual machine to manage the target data based on the preset network connection channel further includes any one of the following:

[0016] If the data management instruction is a data storage instruction, calling the first virtual machine to send the target data to the second virtual machine based on the preset network connection channel, and calling the second virtual machine to store the received target data in a target storage layer, where the target data is stored in the first virtual machine before being sent, and the target storage layer includes an SSD cache layer and an HDD storage layer.

[0017] If the data management instruction is a data read instruction, reading the target data in the target storage layer based on the preset network connection channel, and calling the second virtual machine to send the target data to the first virtual machine.

[0018] In an embodiment, the step of calling the second virtual machine to store the received target data in the target storage layer further includes:

[0019] determining a data type of the target data, where the data type of the target data includes a hot data type and a cold data type.

[0020] If the data type of the target data is the hot data type, calling the second virtual machine to store the target data in the SSD cache layer based on the data storage instruction.

[0021] If the data type of the target data is a cold data type, based on the data storage instruction, a first virtual machine is invoked to store the target data into an HDD storage layer.

[0022] In an embodiment, after the step of invoking a second virtual machine to store the target data into the SSD cache layer based on the data storage instruction, the method further comprises:

[0023] reading a cache state corresponding to the SSD cache layer, and determining whether the plurality of groups of hot data in the SSD cache layer meet the migration condition based on the cache state and a preset migration time interval;

[0024] If the plurality of groups of hot data in the SSD cache layer meet the migration condition, migrating the plurality of groups of hot data in the SSD cache layer to the HDD storage layer.

[0025] In an embodiment, after the step of migrating the plurality of groups of hot data in the SSD cache layer to the HDD storage layer, the method further comprises:

[0026] obtaining a read frequency of each group of hot data in the HDD storage layer;

[0027] based on the read frequency, storing each group of hot data in the HDD storage layer to the SSD cache layer in descending order of read frequency until a data amount cached by the SSD cache layer reaches a preset data amount threshold.

[0028] In an embodiment, before the step of obtaining the current network traffic, the management requirement, and the preset prediction model, the method further comprises:

[0029] obtaining sample data, the prediction result corresponding to the sample data being a third hardware resource occupation rate;

[0030] based on the sample data, using a current prediction model to perform prediction to obtain a fourth hardware resource occupation rate;

[0031] determining whether the third hardware resource occupation rate and the fourth hardware resource occupation rate are consistent;

[0032] If the third hardware resource occupation rate and the fourth hardware resource occupation rate are not consistent, adjusting a parameter of the current prediction model, and returning to the step of using the current prediction model to perform prediction based on the sample data to obtain the fourth hardware resource occupation rate based on the adjusted current prediction model until the third hardware resource occupation rate and the fourth hardware resource occupation rate are consistent, and obtaining the preset prediction model.

[0033] In addition, to achieve the above object, the application further provides a data management device, which comprises:

[0034] The receiving module is configured to determine target data to be managed by the second virtual machine in response to a data management instruction received by the first virtual machine;

[0035] The data management module is configured to manage the target data by the second virtual machine based on a preset network connection channel, wherein the first virtual machine and the second virtual machine are constructed based on the same chip, the preset network connection channel is constructed based on the fact that the first virtual machine and the second virtual machine are in the same network environment, the preset network connection channel is used for data transmission between the first virtual machine and the second virtual machine, the first virtual machine comprises a router module, the second virtual machine comprises a NAS module, and the data management comprises data storage and data reading.

[0036] In an embodiment, the data management device further comprises an adjusting resource module, which comprises:

[0037] The response unit is configured to determine current network traffic and management requirements based on the data management instruction, wherein the management requirements include storage requirements and reading requirements.

[0038] The first prediction unit is configured to predict a first hardware resource occupation rate corresponding to the first virtual machine and a second hardware resource occupation rate corresponding to the second virtual machine based on the current network traffic, the management requirements, and a preset prediction model, wherein the preset prediction model is used to predict a hardware resource occupation rate corresponding to a virtual machine performing a storage task.

[0039] The first judgment unit is configured to determine whether the first hardware resource occupation rate and the second hardware resource occupation rate are greater than a preset hardware occupation rate threshold, and if so, modify system parameters corresponding to the first virtual machine and the second virtual machine to adjust hardware resource configurations corresponding to the first virtual machine and the second virtual machine until the first hardware resource occupation rate and the second hardware resource occupation rate are less than or equal to the preset hardware occupation rate threshold, to obtain an adjusted first virtual machine and an adjusted second virtual machine.

[0040] In an embodiment, the data management module further comprises:

[0041] The management unit is configured to manage the target data by the adjusted second virtual machine based on the preset network connection channel.

[0042] In an embodiment, the data management module further comprises:

[0043] a first data storage unit, configured to, if the data management instruction is a data storage instruction, invoke the first virtual machine to send the target data to the second virtual machine based on a preset network connection channel, and invoke the second virtual machine to store the received target data in a target storage layer, the target data being stored in the first virtual machine before being sent, wherein the target storage layer comprises an SSD cache layer and an HDD storage layer;

[0044] a first data reading unit, configured to, if the data management instruction is a data reading instruction, read the target data in the target storage layer based on a preset network connection channel, and invoke the second virtual machine to send the target data to the first virtual machine.

[0045] In an embodiment, the data management module further comprises:

[0046] a determination unit, configured to determine a data type of the target data, wherein the data type of the target data comprises a hot data type and a cold data type;

[0047] a second data storage unit, configured to, if the data type of the target data is the hot data type, invoke the second virtual machine to store the target data in the SSD cache layer based on the data storage instruction;

[0048] a third data storage unit, configured to, if the data type of the target data is the cold data type, invoke the first virtual machine to store the target data in the HDD storage layer based on the data storage instruction.

[0049] In an embodiment, the data management module further comprises:

[0050] a second data reading unit, configured to read a cache state corresponding to the SSD cache layer, and determine whether a plurality of groups of hot data in the SSD cache layer meet a migration condition based on the cache state and a preset migration time interval;

[0051] a fourth data storage unit, configured to, if the migration condition is met, migrate the plurality of groups of hot data in the SSD cache layer to the HDD storage layer.

[0052] In an embodiment, the data management module further comprises:

[0053] a first acquisition unit, configured to acquire a reading frequency of each group of hot data in the HDD storage layer;

[0054] a fifth data storage unit, configured to store each group of hot data in the HDD storage layer in the SSD cache layer in descending order of reading frequency based on the reading frequency, until a data amount cached by the SSD cache layer reaches a preset data amount threshold.

[0055] In an embodiment, the data management further comprises a model training module, the model training module comprising:

[0056] a second obtaining unit configured to obtain sample data, the sample data corresponding to a third hardware resource occupation rate;

[0057] a second prediction unit configured to use a current prediction model to perform prediction based on the sample data, to obtain a fourth hardware resource occupation rate;

[0058] a second judging unit configured to judge whether the third hardware resource occupation rate and the fourth hardware resource occupation rate are consistent;

[0059] a training unit configured to, if the third hardware resource occupation rate and the fourth hardware resource occupation rate are not consistent, adjust parameters of the current prediction model, return to the step of using the current prediction model to perform prediction based on the sample data to obtain the fourth hardware resource occupation rate based on the adjusted current prediction model, until the third hardware resource occupation rate and the fourth hardware resource occupation rate are consistent, and obtain a preset prediction model.

[0060] In addition, to achieve the above object, the present application further provides a data management device, the device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the computer program being configured to implement the steps of the data management method as described above.

[0061] In addition, to achieve the above object, the present application further provides a storage medium, the storage medium being a computer readable storage medium, the storage medium storing a computer program, the computer program being executed by a processor to implement the steps of the data management method as described above.

[0062] In addition, to achieve the above object, the present application further provides a computer program product, the computer program product comprising a computer program, the computer program being executed by a processor to implement the steps of the data management method as described above.

[0063] The one or more technical solutions provided by the present application have at least the following technical effects:

[0064] The application provides a data management method and related equipment, relates to the technical field of data management, and is different from related technologies. In the related technologies, the implementation of a traditional NAS system needs to depend on at least one NAS and one router, and the cost is too high. In the application, first, in response to a data management instruction received by a first virtual machine, target data that needs to be managed by a second virtual machine is determined. Further, the second virtual machine is called to manage the target data based on a preset network connection channel. The first virtual machine and the second virtual machine are constructed based on the same chip. The preset network connection channel is constructed based on the fact that the first virtual machine and the second virtual machine are in the same network environment. The preset network connection channel is used for data transmission between the first virtual machine and the second virtual machine. The first virtual machine includes a router module, and the second virtual machine includes a NAS module. The data management includes data storage and data reading.

[0065] It can be understood that, in the application, the data management equipment only constructs the first virtual machine (equivalent to the router) and the second virtual machine (equivalent to the NAS) based on one chip. The first virtual machine is used for completing communication interaction (receiving the data management instruction) with the outside, and the second virtual machine is used for managing the data to be managed. In the related technologies, one NAS and one router correspond to two independent chips. Compared with this, the application reduces the number of chips in the NAS system, and thus reduces the cost of implementing the NAS system. BRIEF DESCRIPTION OF DRAWINGS

[0066] The accompanying drawings, which are incorporated into and form part of the specification, illustrate embodiments consistent with the application and, together with the specification, serve to explain the principles of the application.

[0067] In order to more clearly illustrate the technical solutions in the embodiments of the application or the prior art, the accompanying drawings needed in the embodiments or the prior art description will be briefly introduced. Obviously, for those skilled in the art, other drawings can also be obtained based on these drawings without creative labor.

[0068] Figure 1 A flowchart is provided for the data management method embodiment one of the application;

[0069] Figure 2 A data management system schematic diagram is provided for the data management method embodiment one of the application;

[0070] Figure 3 A flowchart is provided for the data management method embodiment two of the application;

[0071] Figure 4 A flowchart is provided for the data management method embodiment three of the application;

[0072] Figure 5 A module structure diagram of a data management device in an embodiment of the present application is shown in FIG. 1.

[0073] Figure 6 A device structure diagram of a hardware running environment involved in a data management method in an embodiment of the present application is shown in FIG. 2.

[0074] The object, features and advantages of the present application will be further illustrated in conjunction with the embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0075] It should be understood that the specific embodiments described herein are merely intended to explain the technical solutions of the present application, and are not intended to limit the present application.

[0076] In order to better understand the technical solutions of the present application, the specific embodiments will be described in detail below in conjunction with the accompanying drawings and specific embodiments.

[0077] The main solution of the embodiment of the present application is:

[0078] In the embodiment, the following is described with the data management device as the execution subject for convenience of description.

[0079] Because of the prior art: the implementation of the traditional NAS system needs to rely on at least one NAS and one router (two chips), which is too high in cost.

[0080] The present application provides a solution, which makes: in response to a data management instruction received by a first virtual machine, a second virtual machine is called to manage the data to be managed, wherein the first virtual machine and the second virtual machine are constructed based on the same chip, the first virtual machine is used for network communication, the second virtual machine is used for data management, and the data management includes data storage and data reading.

[0081] It can be understood that in the present application, the data management device only constructs the first virtual machine (equivalent to the router) and the second virtual machine (equivalent to the NAS) based on one chip, and the communication interaction with the outside (receiving the data management instruction) is completed through the first virtual machine, and the data to be managed is managed through the second virtual machine. In the related art, one NAS and one router correspond to two independent chips, compared with which, the present application reduces the number of chips in the NAS system, and thus reduces the cost of implementing the NAS system.

[0082] It should be noted that the execution subject of the embodiment can be a computing service device with data management, network communication and program running functions, such as a tablet computer, a personal computer, a mobile phone, etc., or an electronic device, a data management device, etc. capable of realizing the above functions. The following takes the data management device as an example to describe the embodiment and the following embodiments.

[0083] Based on this, the data management method provided in the embodiments of the present application comprises the steps of Figure 1 , Figure 1 The flowchart of the first embodiment of the data management method of the present application is shown in FIG. 1.

[0084] In the embodiment, the data management method comprises steps S10-S20:

[0085] In step S10, in response to a data management instruction received by the first virtual machine, target data that needs to be managed by the second virtual machine is determined.

[0086] It should be noted that the execution subject of the embodiment can be a computing service device with data management, network communication and program running functions, such as a tablet computer, a personal computer, a mobile phone, etc., or an electronic device, a data management device, etc. that can realize the above functions. In the following, the data management device is taken as an example to describe the embodiment and the following embodiments.

[0087] It should be noted that a virtual machine is an independent computer system simulated by software on a physical computer (host computer), which has its own operating system and application program and can run like a real computer. Multiple virtual machines can run on the same host computer, which are isolated from each other but can communicate and interact with each other through a specific mechanism.

[0088] It should be noted that the kvm virtual machine model is described in detail in the prior art Figure 2 In the present application, the first virtual machine and the second virtual machine are both constructed using the kvm virtual machine model by the data management device, the first virtual machine has a router module, which is equivalent to a router in the NAS system, the second virtual machine has a data management module, which is used to manage the storage and reading of data, and is equivalent to a NAS in the NAS system, the first virtual machine and the second virtual machine are isolated from each other. In addition, when constructing the second virtual machine, the storage and expansion interfaces that need to be shared are allocated to the virtual machine 1.

[0089] It should be noted that the network attached storage (NAS system) is a high-efficiency data storage and sharing solution. Taking data storage as an example, the router is responsible for receiving data storage instructions and receiving data to be stored, and then the router determines the target data in the data to be stored that needs to be stored in the NAS according to the data storage instructions. Further, based on the network channel established under the same network environment, the router sends the target data to the NAS for storage.

[0090] In the present application, the specific application scenario can be:

[0091] Firstly, the data management device receives a data management instruction based on the first virtual machine, and then determines target data to be managed based on the first virtual machine based on the data management instruction.

[0092] It should be noted that the target data is data determined according to the data management instruction and needs to be managed by the second virtual machine. It can be a file, a database record, a real-time generated data stream, and various forms of data collection, depending on the type of instruction and the application scenario.

[0093] It can be understood that different modules manage different data when storing or reading data, and not all data is managed by the second virtual machine. Therefore, after receiving the data management instruction, the target data (data to be managed by the second virtual machine) needs to be determined first, and then the data management device calls the second virtual machine to manage the target data.

[0094] Step S20, based on a preset network connection channel, calling the second virtual machine to manage the target data, wherein the first virtual machine and the second virtual machine are based on the same chip, the preset network connection channel is based on the first virtual machine and the second virtual machine being in the same network environment, the preset network connection channel is used for data transmission between the first virtual machine and the second virtual machine, the first virtual machine includes a router module, the second virtual machine includes a NAS module, and the data management includes data storage and data reading.

[0095] It should be noted that the preset network connection channel is a pre-set network path for data transmission. This channel is specially constructed and optimized for communication between virtual machines. This channel can ensure stable and efficient data transmission between the first virtual machine and the second virtual machine, and it is the basis for the two virtual machines to work together.

[0096] It should be noted that the two virtual machines (the first virtual machine and the second virtual machine) are created and run based on the same physical chip. This architecture can fully utilize the resources of the chip, improve the utilization of resources, and because the two virtual machines have natural affinity at the hardware level, they can work better together, reducing compatibility problems and performance loss that may be caused by hardware differences.

[0097] Specifically, the step of calling the second virtual machine to manage the target data based on the preset network connection channel further includes any one of steps S21-S22:

[0098] Step S21, if the data management instruction is a data storage instruction, based on the preset network connection channel, the first virtual machine is called to send the target data to the second virtual machine, and the second virtual machine is called to store the received target data in the target storage layer. The target data is stored in the first virtual machine before being sent, wherein the target storage layer includes an SSD cache layer and an HDD storage layer.

[0099] It can be understood that when the data management device receives a data storage instruction based on the first virtual machine, it means that specific target data needs to be saved. Further, the data management device uses a pre-set network path specially for communication between the first virtual machine and the second virtual machine to ensure that the target data can be stably and efficiently transmitted from the first virtual machine to the second virtual machine.

[0100] After the second virtual machine receives the data sent from the first virtual machine, it uses its internal NAS module to store the data in the target storage layer according to the established storage strategy and rules.

[0101] It should be noted that the SSD cache layer uses the high-performance characteristics of solid-state disks as a high-speed cache space in the system. It is mainly used for temporarily storing frequently accessed data to speed up data reading and writing and improve the overall response capability of the system. SSD has the advantages of fast read and write speed, no mechanical parts, strong shock resistance, etc., which can significantly improve the performance of the system, especially in handling a large number of random read and write operations. The storage cost of HDD is relatively low, which can provide large storage capacity at a lower price, suitable for storing a large amount of historical data, backup data, etc.

[0102] It should be noted that the HDD storage layer is a bottom-layer storage device in the system for long-term storage of a large amount of data. It provides large-capacity storage space for storing data that is not frequently accessed or has relatively low requirements for read and write speed.

[0103] It can be understood that in this embodiment, by assigning data storage tasks to a dedicated second virtual machine and using its NAS module, more efficient and more specialized data storage management can be achieved. At the same time, the hierarchical storage strategy of the SSD cache layer and the HDD storage layer takes into account the performance and cost of data storage. At the same time, based on the preset network connection channel and the router module in the first virtual machine, the stability and security of data transmission are ensured, avoiding data loss or incorrect storage due to network problems.

[0104] Specifically, the step of calling the second virtual machine to store the received target data in the target storage layer further includes steps A10-A30:

[0105] Step A10, determining a data type of the target data, wherein the data type of the target data comprises a hot data type and a cold data type;

[0106] It should be noted that the cold data refers to the data that is not frequently accessed, and usually has a lower access priority, such as monitoring video data; and the hot data refers to the data that is frequently accessed and used, and has a higher access priority.

[0107] Step A20, if the data type of the target data is the hot data type, based on the data storage instruction, calling the second virtual machine to store the target data into the SSD cache layer.

[0108] Step A30, if the data type of the target data is the cold data type, based on the data storage instruction, calling the first virtual machine to store the target data into the HDD storage layer.

[0109] It can be understood that in the embodiment, for different data types of data, a hierarchical storage strategy is adopted, and the data with a higher access priority is stored in the SSD cache layer with a cache space, so as to improve the reading efficiency in the next reading of the data, that is, improve the data management efficiency.

[0110] Specifically, after the step of based on the data storage instruction, calling the second virtual machine to store the target data into the SSD cache layer, the method further comprises steps A40-A50:

[0111] Step A40, reading a cache state corresponding to the SSD cache layer, and based on the cache state and a preset migration time interval, judging whether a plurality of groups of hot data in the SSD cache layer meet migration conditions;

[0112] It should be noted that the cache state corresponding to the SSD cache layer refers to the cache space occupancy rate of the SSD cache layer.

[0113] Step A50, if the conditions are met, migrating the plurality of groups of hot data in the SSD cache layer to the HDD storage layer.

[0114] It can be understood that in the embodiment, the SSD cache layer is only used to cache the data with a higher access priority, in order to improve the efficiency of data reading, but for the NAS module, the actual storage space is the HDD storage layer, and a large amount of data is stored based on the HDD storage layer.

[0115] Specifically, after the step of migrating the plurality of groups of hot data in the SSD cache layer to the HDD storage layer, the method further comprises steps A60-A70:

[0116] Step A60, obtaining the read frequency of each group of hot data in the HDD storage layer;

[0117] Step A70, based on the read frequency, storing each group of hot data in the HDD storage layer in the order of read frequency from high to low to the SSD cache layer until the data amount cached by the SSD cache layer reaches the preset data amount threshold.

[0118] It can be understood that in the present embodiment, as the storage data increases, the access priority of the hot data in the SSD cache layer may be lower than that of the hot data in the HDD storage layer. At this time, it is necessary to store all the hot data in the SSD cache layer and the HDD storage layer in the order of access priority (read frequency) from high to low to the SSD cache layer until the data amount cached by the SSD cache layer reaches the preset data amount threshold, so that the data in the SSD cache layer is always the group of hot data with the highest access priority, thereby improving the data management efficiency.

[0119] Step S22, if the data management instruction is a data read instruction, reading the target data in the target storage layer based on a preset network connection channel, and calling a second virtual machine to send the target data to a first virtual machine.

[0120] The present application proposes a data management method and related equipment, which relates to the technical field of data management. Compared with the related art, the implementation of the traditional NAS system needs to rely on at least one NAS and one router, which is too high in cost. In the present application, first, in response to the data management instruction received by the first virtual machine, the target data to be managed by the second virtual machine is determined. Further, based on a preset network connection channel, the second virtual machine is called to manage the target data. The first virtual machine and the second virtual machine are constructed based on the same chip. The preset network connection channel is constructed based on the fact that the first virtual machine and the second virtual machine are in the same network environment. The preset network connection channel is used for data transmission between the first virtual machine and the second virtual machine. The first virtual machine includes a router module, and the second virtual machine includes a NAS module. The data management includes data storage and data read.

[0121] It can be understood that in the present application, the data management equipment only constructs the first virtual machine (equivalent to the router) and the second virtual machine (equivalent to the NAS) based on one chip. The first virtual machine completes the communication interaction with the outside (receives the data management instruction), and the second virtual machine manages the data to be managed. In the related art, one NAS and one router correspond to two independent chips. Compared with this, the present application reduces the number of chips in the NAS system, thereby reducing the cost of implementing the NAS system.

[0122] Based on the first embodiment of the present application, in the second embodiment of the present application, the same or similar contents as the above embodiment one can refer to the above introduction, and the subsequent will not be described. On this basis, please refer to Figure 3 Before step S20, the data management method further includes steps S11-S13:

[0123] Step S11, based on the data management instruction, determine the current network traffic and management requirements, the management requirements include storage requirements and read requirements;

[0124] It should be noted that network traffic reflects the degree of network congestion and the load of data transmission, which is crucial for subsequent data management strategy adjustment.

[0125] For example, in a server of an e-commerce website, through the network traffic monitoring tool, the amount of data flowing out from the server per second can be seen in real time, such as during the promotion period, the network traffic may increase significantly, because a large number of users access the website at the same time, and operations such as viewing product information and placing orders will generate a large amount of data transmission. Understanding this traffic peak situation helps the enterprise to make advance preparation for server expansion to prevent the website from crashing due to excessive traffic.

[0126] It should be noted that the management requirement refers to the amount of data, the type of data, and the length of time for storage (or reading) required by the system.

[0127] Step S12, based on the current network traffic, management requirements and preset prediction model, predict the first hardware resource occupation rate corresponding to the first virtual machine, and predict the second hardware resource occupation rate corresponding to the second virtual machine, wherein the preset prediction model is used to predict the hardware resource occupation rate corresponding to the virtual machine when performing storage task;

[0128] It should be noted that the preset prediction model is a trained current prediction model, and the prediction model is used to predict the hardware resource occupation rate corresponding to the virtual machine when performing storage task.

[0129] It should be noted that the hardware resource occupation rate refers to the degree or proportion of various resources (such as CPU, memory, disk, network, etc.) inside the hardware devices (such as computer system, server, network equipment, etc.) being used during running. Hardware resource occupation rate reflects the utilization of hardware resources, which is an important indicator for evaluating system performance, optimizing resource allocation and troubleshooting.

[0130] In addition, it should be noted that it is generally believed that a hardware resource occupancy rate below 70% indicates that the system is running smoothly and handling tasks with ease; a hardware resource occupancy rate between 70% and 90% indicates that the system is under high load but can still run normally; and a hardware resource occupancy rate exceeding 90% indicates that the system is under excessive load and may experience lag, slow response, and other problems, requiring optimization or upgrading of the CPU.

[0131] Step S13, determining whether the first hardware resource occupancy rate and the second hardware resource occupancy rate are greater than a preset hardware occupancy threshold, and if so, modifying system parameters corresponding to the first virtual machine and the second virtual machine to adjust hardware resource configurations corresponding to the first virtual machine and the second virtual machine until the first hardware resource occupancy rate and the second hardware resource occupancy rate are less than or equal to the preset hardware occupancy threshold, obtaining an adjusted first virtual machine and an adjusted second virtual machine.

[0132] It can be understood that in this embodiment, by modifying the system parameters corresponding to the first virtual machine and the second virtual machine based on the prediction result of the preset prediction model, the hardware resource configurations corresponding to the first virtual machine and the second virtual machine are adjusted, so that the first virtual machine and the second virtual machine always maintain stable operation when processing corresponding tasks, avoiding the impact on data management efficiency due to excessive load.

[0133] Further, the step of calling the second virtual machine to manage the target data based on the preset network connection channel further comprises:

[0134] Based on the preset network connection channel, the adjusted second virtual machine is called to manage the target data.

[0135] It can be understood that in this application, using the adjusted second virtual machine to complete the corresponding task can ensure that the second virtual machine always operates efficiently and stably, avoiding the impact on data management efficiency due to excessive load.

[0136] Based on the first and second embodiments of the present application, in the third embodiment of the present application, the same or similar contents as the above-mentioned embodiments can be referred to in the above description, and will not be described again. On this basis, please refer to Figure 4 , before step S12, the data management method further comprises steps B10-B40:

[0137] Step B10, obtaining sample data, the prediction result corresponding to the sample data being a third hardware resource occupancy rate;

[0138] It should be noted that the sample data is used to train the current prediction model to improve the prediction accuracy of the current prediction model.

[0139] Step B20, based on the sample data, using the current prediction model to make a prediction, obtaining a fourth hardware resource occupation rate;

[0140] Step B30, judging whether the third hardware resource occupation rate and the fourth hardware resource occupation rate are consistent;

[0141] Step B40, if not consistent, adjusting the parameters of the current prediction model, returning to the step of using the current prediction model to make a prediction based on the sample data based on the adjusted current prediction model until the third hardware resource occupation rate and the fourth hardware resource occupation rate are consistent, obtaining a preset prediction model.

[0142] It can be understood that the third hardware resource occupation rate is the actual occupation result corresponding to the sample data, and the fourth hardware resource occupation rate is the predicted occupation result corresponding to the sample data based on the current prediction model. The predicted occupation result and the actual occupation result may not be consistent, and the model is trained.

[0143] It should be noted that the predicted occupation result and the actual occupation result are consistent, which does not mean that the predicted occupation result and the actual occupation result are exactly the same. Only when the predicted loss value corresponding to the predicted occupation result and the actual occupation result is within the range set by the staff, it is considered that the predicted occupation result and the actual occupation result are consistent. This is convenient for reducing the training amount of model training and improving the training efficiency of the model.

[0144] It can be understood that in the embodiment, the model is trained based on the sample data to improve the prediction accuracy of the model, and then a more accurate prediction result is obtained, which is convenient for more accurate adjustment of the hardware resource configuration of the virtual machine in the subsequent steps.

[0145] It should be noted that the above examples are only used to understand the present application and do not constitute a limitation on the data management method of the present application. More forms of simple transformation based on this technical concept are within the protection scope of the present application.

[0146] The present application also provides a data management device, please refer to Figure 5 , the data management device comprises:

[0147] The receiving module 10 is used for determining the target data which needs to be managed by the second virtual machine in response to the data management instruction received by the first virtual machine;

[0148] The data management module 20 is configured to invoke a second virtual machine to manage the target data based on a preset network connection channel, wherein the first virtual machine and the second virtual machine are constructed based on the same chip, the preset network connection channel is constructed based on that the first virtual machine and the second virtual machine are in the same network environment, the preset network connection channel is used for data transmission between the first virtual machine and the second virtual machine, the first virtual machine comprises a router module, the second virtual machine comprises a NAS module, and the data management comprises data storage and data reading.

[0149] In an embodiment, the data management device further comprises an adjustment resource module, and the adjustment resource module comprises:

[0150] a response unit configured to determine current network traffic and management demand based on a data management instruction, wherein the management demand comprises storage demand and reading demand;

[0151] a first prediction unit configured to predict a first hardware resource occupation rate corresponding to the first virtual machine and predict a second hardware resource occupation rate corresponding to the second virtual machine based on the current network traffic, the management demand, and a preset prediction model, wherein the preset prediction model is used to predict a hardware resource occupation rate corresponding to a virtual machine performing a storage task;

[0152] a first judgment unit configured to determine whether the first hardware resource occupation rate and the second hardware resource occupation rate are greater than a preset hardware occupation rate threshold, and if so, modify system parameters corresponding to the first virtual machine and the second virtual machine to adjust hardware resource configurations corresponding to the first virtual machine and the second virtual machine until the first hardware resource occupation rate and the second hardware resource occupation rate are less than or equal to the preset hardware occupation rate threshold, to obtain an adjusted first virtual machine and an adjusted second virtual machine;

[0153] In an embodiment, the data management module further comprises:

[0154] a management unit configured to invoke the adjusted second virtual machine to manage the target data based on the preset network connection channel.

[0155] In an embodiment, the data management module further comprises:

[0156] a first data storage unit configured to, if the data management instruction is a data storage instruction, invoke the first virtual machine to send the target data to the second virtual machine based on the preset network connection channel, and invoke the second virtual machine to store the received target data in a target storage layer, wherein the target data is stored in the first virtual machine before being sent, and the target storage layer comprises an SSD cache layer and an HDD storage layer.

[0157] a first data reading unit, configured to read the target data in the target storage layer based on a preset network connection channel if the data management instruction is a data reading instruction, and to call a second virtual machine to send the target data to a first virtual machine.

[0158] In an embodiment, the data management module further comprises:

[0159] a determination unit, configured to determine a data type of the target data, wherein the data type of the target data comprises a hot data type and a cold data type;

[0160] a second data storage unit, configured to store the target data in an SSD cache layer based on the data storage instruction if the data type of the target data is the hot data type by calling the second virtual machine.

[0161] a third data storage unit, configured to store the target data in an HDD storage layer based on the data storage instruction if the data type of the target data is the cold data type by calling the first virtual machine.

[0162] In an embodiment, the data management module further comprises:

[0163] a second data reading unit, configured to read a cache state corresponding to the SSD cache layer, and to determine whether a plurality of groups of hot data in the SSD cache layer meet a migration condition based on the cache state and a preset migration time interval.

[0164] a fourth data storage unit, configured to migrate the plurality of groups of hot data in the SSD cache layer to the HDD storage layer if the migration condition is met.

[0165] In an embodiment, the data management module further comprises:

[0166] a first obtaining unit, configured to obtain a reading frequency of each group of hot data in the HDD storage layer.

[0167] a fifth data storage unit, configured to store each group of hot data in the HDD storage layer in the SSD cache layer in descending order of reading frequency based on the reading frequency, until a data amount cached by the SSD cache layer reaches a preset data amount threshold.

[0168] In an embodiment, the data management further comprises a model training module, and the model training module comprises:

[0169] a second obtaining unit, configured to obtain sample data, wherein a predicted result corresponding to the sample data is a third hardware resource occupancy rate.

[0170] a second prediction unit, configured to predict, based on the sample data, using a current prediction model, to obtain a fourth hardware resource occupation rate;

[0171] a second judgment unit, configured to judge whether the third hardware resource occupation rate is consistent with the fourth hardware resource occupation rate;

[0172] a training unit, configured to, if the third hardware resource occupation rate is not consistent with the fourth hardware resource occupation rate, adjust a parameter of the current prediction model, return to the step of predicting, based on the sample data, using the current prediction model, to obtain the fourth hardware resource occupation rate, based on the adjusted current prediction model, until the third hardware resource occupation rate is consistent with the fourth hardware resource occupation rate, to obtain a preset prediction model.

[0173] The data management device provided in the application adopts the data management method in the above embodiments, and can solve the technical problem of data management. Compared with the prior art, the data management device provided in the application has the same beneficial effects as the data management method provided in the above embodiments, and other technical features in the data management device are the same as the features disclosed in the above embodiments, which will not be repeated here.

[0174] The application provides a data management device, which comprises at least one processor and a memory in communication connection with the at least one processor, wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the data management method in Embodiment I.

[0175] Reference will now be made to the drawings, and specific examples thereof will be illustrated. Figure 6 The data management device in the embodiments of the application can include, but is not limited to, mobile terminals such as mobile phones, notebook computers, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Descriptions), PMPs (Portable Media Players), vehicle-mounted terminals (for example, vehicle-mounted navigation terminals), and the like, and fixed terminals such as digital TVs, desktop computers, and the like. Figure 6 The data management device shown is only an example, and should not impose any limitation on the functions and use range of the embodiments of the application.

[0176] As Figure 6As shown, the data management device can include a processing apparatus 1001 (e.g., a central processing unit, a graphics processing unit, etc.) that can perform various appropriate actions and processes according to a program stored in a read only memory (ROM) 1002 or a program loaded from a storage apparatus 1003 into a random access memory (RAM) 1004. Various programs and data required for operation of the data management device are also stored in the RAM 1004. The processing apparatus 1001, the ROM 1002, and the RAM 1004 are connected to each other through a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Generally, the following systems can be connected to the I / O interface 1006: an input apparatus 1007 including, for example, a touch screen, a touch pad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; an output apparatus 1008 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; the storage apparatus 1003 including, for example, a magnetic tape, a hard disk, etc.; and a communication apparatus 1009. The communication apparatus 1009 can allow the data management device to communicate with other devices wirelessly or by wire to exchange data. Although the data management device having various systems is shown in the figure, it should be understood that all the shown systems are not required to be implemented or possessed. More or less systems can be alternatively implemented or possessed.

[0177] In particular, according to embodiments of the present disclosure, the processes described above with reference to the flowcharts can be implemented as a computer software program. For example, embodiments of the present disclosure include a computer program product comprising a computer program carried on a computer readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network through the communication apparatus, or installed from the storage apparatus 1003, or installed from the ROM 1002. When the computer program is executed by the processing apparatus 1001, the above-mentioned functions defined in the methods of embodiments of the present disclosure are performed.

[0178] The data management device provided by the present disclosure adopts the data management method in the above-mentioned embodiments, and can solve the technical problem of data management. Compared with the prior art, the data management device provided by the present disclosure has the same beneficial effects as the data management method provided by the above-mentioned embodiments, and other technical features in the data management device are the same as the features disclosed in the above-mentioned embodiments, which will not be repeated here.

[0179] It should be understood that portions of the application disclosed can be implemented in hardware, software, firmware, or combinations thereof. In the description of the embodiments above, specific features, structures, materials or characteristics can be combined in any suitable manner in one or more embodiments or examples.

[0180] The above description is merely illustrative of the application and is not intended to limit the scope of the application. Any variations and modifications that can be made by any person skilled in the art within the spirit and scope of the application are intended to be encompassed by the application. The scope of the application is defined by the appended claims.

[0181] The application provides a computer readable storage medium having stored thereon computer readable program instructions (i.e., a computer program) for performing the data management method in the above-described embodiments.

[0182] The computer readable storage medium provided by the application may, for example, be a U disk, but is not limited to an electric, magnetic, optical, electromagnetic, infrared, or semiconductor system or device, or any combination thereof. More specific examples of the computer readable storage medium can include, but are not limited to, an electric connection having one or more conductive wires, a portable computer disk, a hard disk, a random access memory (RAM), a read only memory (ROM), an erasable programmable read only memory (EPROM or flash memory), an optical fiber, a portable compact disk read only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the embodiments, the computer readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system or device. The program code contained on the computer readable storage medium can be transmitted by any suitable medium, including but not limited to an electric wire, an optical cable, an RF (Radio Frequency), etc., or any suitable combination thereof.

[0183] The computer readable storage medium described above can be contained in the data management device; or can exist separately and not be assembled into the data management device.

[0184] The computer readable storage medium described above carries one or more programs, which, when executed by the data management device, cause the data management device to:

[0185] In response to the data management instruction received by the first virtual machine, determine target data that needs to be managed by the second virtual machine;

[0186] Based on a preset network connection channel, call the second virtual machine to manage the target data, wherein the first virtual machine and the second virtual machine are constructed based on the same chip, the preset network connection channel is constructed based on the fact that the first virtual machine and the second virtual machine are in the same network environment, the preset network connection channel is used for data transmission between the first virtual machine and the second virtual machine, the first virtual machine includes a router module, the second virtual machine includes a NAS module, and the data management includes data storage and data reading.

[0187] Computer program code for carrying out operations of the present application can be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C++ or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code can execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computer (for example, through the Internet using an Internet Service Provider).

[0188] The flow diagrams and the block diagrams in the drawings are illustrations of architectures, functionalities, and operations of possible implementations of systems, methods, and computer program products according to various embodiments of present application. In this regard, each block in the flow diagrams or block diagrams can represent a module, a procedure, or a portion of code, which comprises one or more executable instructions for implementing the specified logical functions. It should also be noted that in some alternative implementations, the functions noted in the blocks can occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently or in the reverse order, depending on the functionality involved. It will also be noted that each block of the block diagrams and / or flow diagrams, and combinations thereof, can be implemented by a dedicated hardware-based system that carries out specified functions or operations, or combinations of hardware and computer instructions.

[0189] The modules described in the embodiments of the present application can be implemented in the form of software or in the form of hardware. In some cases, the name of the module does not constitute a limitation on the module itself.

[0190] The readable storage medium provided by the present application is a computer readable storage medium, which stores computer readable program instructions (i.e., a computer program) for executing the above data management method, and can solve the technical problem of data management. Compared with the prior art, the computer readable storage medium provided by the present application has the same beneficial effects as the data management method provided by the above embodiments, which will not be repeated here.

[0191] The present application also provides a computer program product, comprising a computer program, which, when executed by a processor, implements the steps of the data management method as described above.

[0192] The computer program product provided by the present application can solve the technical problem of data management. Compared with the prior art, the computer program product provided by the present application has the same beneficial effects as the data management method provided by the above embodiments, which will not be repeated here.

[0193] The above only describes some embodiments of the present application, and does not limit the patent scope of the present application. Any equivalent structural transformation, direct / indirect application in other related technical fields based on the technical concept of the present application, and the contents of the present application and the accompanying drawings are included in the patent protection scope of the present application.

Claims

1. A data management method characterized by, The data management method comprises: in response to a data management instruction received by a first virtual machine, determining target data that needs to be managed by a second virtual machine; based on a preset network connection channel, calling the second virtual machine to manage the target data, wherein the first virtual machine and the second virtual machine are based on the same chip, the preset network connection channel is based on the first virtual machine and the second virtual machine being in the same network environment, the preset network connection channel is used for data transmission between the first virtual machine and the second virtual machine, the first virtual machine comprises a router module, the second virtual machine comprises a NAS module, and the data management comprises data storage and data reading; the step of calling the second virtual machine to manage the target data based on the preset network connection channel further comprises any one of the following: if the data management instruction is a data storage instruction, based on the preset network connection channel, the first virtual machine is called to send the target data to the second virtual machine, the second virtual machine is called to store the received target data in the target storage layer, and the target data is stored in the first virtual machine before being sent, wherein the target storage layer comprises an SSD cache layer and an HDD storage layer; if the data management instruction is a data reading instruction, based on the preset network connection channel, the target data in the target storage layer is read, and the second virtual machine is called to send the target data to the first virtual machine.

2. The data management method of claim 1, wherein, Before the step of calling the second virtual machine to manage the target data based on the preset network connection channel, the following steps are further included: based on the data management instruction, determining the current network traffic and management demand, the management demand including storage demand and reading demand; based on the current network traffic, management demand and preset prediction model, predicting the first hardware resource occupation rate corresponding to the first virtual machine, and predicting the second hardware resource occupation rate corresponding to the second virtual machine, wherein the preset prediction model is used to predict the hardware resource occupation rate corresponding to the virtual machine executing the storage task; determining whether the first hardware resource occupation rate and the second hardware resource occupation rate are greater than a preset hardware occupation rate threshold, if greater, modifying the system parameters corresponding to the first virtual machine and the second virtual machine to adjust the hardware resource configuration corresponding to the first virtual machine and the second virtual machine, until the first hardware resource occupation rate and the second hardware resource occupation rate are less than or equal to the preset hardware occupation rate threshold, obtaining the adjusted first virtual machine and the adjusted second virtual machine; the step of calling the second virtual machine to manage the target data based on the preset network connection channel further comprises: based on the preset network connection channel, calling the adjusted second virtual machine to manage the target data.

3. The data management method of claim 1, wherein, The step of calling the second virtual machine to store the received target data in the target storage layer further comprises: determining the data type of the target data, wherein the data type of the target data comprises hot data type and cold data type; If the data type of the target data is a hot data type, based on the data storage instruction, a second virtual machine is called to store the target data into an SSD cache layer; If the data type of the target data is a cold data type, based on the data storage instruction, a first virtual machine is called to store the target data into an HDD storage layer.

4. The data management method of claim 3, wherein, After the step of based on the data storage instruction, calling the second virtual machine to store the target data into the SSD cache layer, the method further comprises: reading a cache state corresponding to the SSD cache layer, and based on the cache state and a preset migration time interval, judging whether a plurality of groups of hot data in the SSD cache layer meet migration conditions; If yes, migrating the plurality of groups of hot data in the SSD cache layer to the HDD storage layer.

5. The data management method of claim 3, wherein, After the step of migrating the plurality of groups of hot data in the SSD cache layer to the HDD storage layer, the method further comprises: obtaining a read frequency of each group of hot data in the HDD storage layer; based on the read frequency, storing each group of hot data in the HDD storage layer into the SSD cache layer in descending order of read frequency, until a data amount cached by the SSD cache layer reaches a preset data amount threshold.

6. The data management method of claim 2, wherein, Before the step of based on the current network traffic, management requirement and preset prediction model, the method further comprises: obtaining sample data, a prediction result corresponding to the sample data being a third hardware resource occupation rate; based on the sample data, using a current prediction model to perform prediction to obtain a fourth hardware resource occupation rate; judging whether the third hardware resource occupation rate and the fourth hardware resource occupation rate are consistent; if not, adjusting a parameter of the current prediction model, and based on the adjusted current prediction model, returning to the step of based on the sample data, using the current prediction model to perform prediction to obtain the fourth hardware resource occupation rate, until the third hardware resource occupation rate and the fourth hardware resource occupation rate are consistent, to obtain a preset prediction model.

7. A data management apparatus characterized by comprising: The data management comprises: a receiving module, configured to determine target data to be managed by a second virtual machine in response to a data management instruction received by a first virtual machine; a data management module, configured to call the second virtual machine to manage the target data based on a preset network connection channel, wherein the first virtual machine and the second virtual machine are constructed based on a same chip, the preset network connection channel is constructed based on that the first virtual machine and the second virtual machine are in a same network environment, the preset network connection channel is used for data transmission between the first virtual machine and the second virtual machine, the first virtual machine comprises a router module, the second virtual machine comprises a NAS module, and the data management comprises data storage and data reading; the data management module is further configured to implement any one of the following: If the data management instruction is a data storage instruction, based on a preset network connection channel, the first virtual machine is invoked to send the target data to a second virtual machine, the second virtual machine is invoked to store the received target data in a target storage layer, and the target data is stored in the first virtual machine before being sent, wherein the target storage layer comprises an SSD cache layer and an HDD storage layer. If the data management instruction is a data read instruction, based on a preset network connection channel, the target data in the target storage layer is read, and the second virtual machine is invoked to send the target data to the first virtual machine.

8. A data management device, characterized by comprising: The device comprises a memory, a processor, and a computer program stored on the memory and executable on the processor, and the computer program is configured to implement the steps of the data management method according to any one of claims 1 to 6.

9. A storage medium, characterized by The storage medium is a computer readable storage medium, and the storage medium stores a computer program, and the computer program is executed by a processor to implement the steps of the data management method according to any one of claims 1 to 6.

Images