Data management method and related equipment

Abstract

The invention discloses a data management method and related equipment, and relates to the technical field of data management, and the method comprises the steps: responding to a data management instruction received by a first virtual machine, and determining target data needing to be managed by a second virtual machine; based on a preset network connection channel, a second virtual machine is called to manage the target data, the first virtual machine and the second virtual machine are constructed based on the same chip, and the preset network connection channel is constructed based on the fact that the first virtual machine and the second virtual machine are in the same network environment. The preset network connection channel is used for data transmission between a first virtual machine and a second virtual machine, the first virtual machine comprises a router module, the second virtual machine comprises an NAS module, and the data management comprises data storage and data reading. According to the application, the two virtual machines are constructed based on the same chip, so that the two virtual machines respectively replace the NAS and the router, and the NAS system is formed by only depending on one chip, so that the cost of constructing the NAS system is reduced.

Description

Technical Field

[0001] This application relates to the technical field of data management, and in particular, to data management methods and related devices. Background Art

[0002] As an efficient data storage and sharing solution, Network Attached Storage (NAS system) has become the core device for enterprises and individual users to manage massive data. Its core function is to achieve cross-platform file sharing through network protocols (such as NFS, CTF, SMB, etc.), and to centrally manage data through dedicated hardware (such as disk arrays, high-speed network interfaces), thereby improving storage efficiency and reducing costs.

[0003] However, the implementation of traditional NAS systems requires at least one NAS and one router (two chips), resulting in high costs.

[0004] The above content is only used to assist in understanding the technical solution of this application, and does not represent an admission that the above content is related art. Summary of the Invention

[0005] The main purpose of this application is to provide a data management method and related devices, aiming to solve the technical problem of high costs required for the implementation of NAS systems.

[0006] To achieve the above purpose, this application proposes a data management method, which includes:

[0007] In response to a data management instruction received by a first virtual machine, determine target data to be managed by a second virtual machine;

[0008] Based on a preset network connection channel, call the second virtual machine to manage the target data, where the first virtual machine and the second virtual machine are built based on the same chip, the preset network connection channel is built based on the first virtual machine and the second virtual machine being in the same network environment, the preset network connection channel is used for data transmission between the first virtual machine and the second virtual machine, the first virtual machine includes a router module, the second virtual machine includes a NAS module, and the data management includes data storage and data reading.

[0009] In one embodiment, before the step of calling the second virtual machine to manage the target data based on the preset network connection channel, it further includes:

[0010] Based on the data management instruction, determine the current network traffic and management requirements, where the management requirements include storage requirements and reading requirements;

[0011] Predict the first hardware resource occupancy rate corresponding to the first virtual machine and the second hardware resource occupancy rate corresponding to the second virtual machine based on the current network traffic, management requirements, and a preset prediction model, where the preset prediction model is used to predict the hardware resource occupancy rate corresponding to a virtual machine when executing a storage task;

[0012] Determine whether the first hardware resource occupancy rate and the second hardware resource occupancy rate are greater than a preset hardware occupancy rate threshold. If so, modify the system parameters corresponding to the first virtual machine and the second virtual machine to adjust the hardware resource configuration corresponding to the first virtual machine and the second virtual machine until the first hardware resource occupancy rate and the second hardware resource occupancy rate are less than or equal to the preset hardware occupancy rate threshold, and obtain the adjusted first virtual machine and the adjusted second virtual machine;

[0013] The step of invoking the second virtual machine to manage the target data based on a preset network connection channel further includes:

[0014] Based on a preset network connection channel, invoke the adjusted second virtual machine to manage the target data.

[0015] In an embodiment, the step of invoking the second virtual machine to manage the target data based on a preset network connection channel further includes any one of the following:

[0016] If the data management instruction is a data storage instruction, based on a preset network connection channel, invoke the first virtual machine to send the target data to the second virtual machine, and invoke the second virtual machine to store the received target data in a target storage layer. The target data is stored in the first virtual machine before being sent, where the target storage layer includes an SSD cache layer and an HDD storage layer;

[0017] If the data management instruction is a data read instruction, based on a preset network connection channel, read the target data in the target storage layer, and invoke the second virtual machine to send the target data to the first virtual machine.

[0018] In an embodiment, the step of invoking the second virtual machine to store the received target data in the target storage layer further includes:

[0019] Determine the data type of the target data, where the data type of the target data includes a hot data type and a cold data type;

[0020] If the data type of the target data is a hot data type, based on the data storage instruction, invoke the second virtual machine to store the target data in the SSD cache layer;

[0021] If the data type of the target data is a cold data type, based on the data storage instruction, call the first virtual machine to store the target data in the HDD storage layer.

[0022] In one embodiment, after the step of calling the second virtual machine to store the target data in the SSD cache layer based on the data storage instruction, the method further includes:

[0023] Read the cache status corresponding to the SSD cache layer, and based on the cache status and the preset transfer time interval, determine whether multiple groups of hot data in the SSD cache layer meet the transfer conditions;

[0024] If satisfied, transfer multiple groups of hot data in the SSD cache layer to the HDD storage layer.

[0025] In one embodiment, after the step of transferring multiple groups of hot data in the SSD cache layer to the HDD storage layer, the method further includes:

[0026] Obtain the read times of each group of hot data in the HDD storage layer;

[0027] Based on the read times, store each group of hot data in the HDD storage layer in descending order of read times to the SSD cache layer until the data volume cached in the SSD cache layer reaches the preset data volume threshold.

[0028] In one embodiment, before the step of based on the current network traffic, management requirements, and preset prediction model, the method further includes:

[0029] Obtain sample data, and the prediction result corresponding to the sample data is the third hardware resource occupancy rate;

[0030] Based on the sample data, use the current prediction model to make a prediction to obtain the fourth hardware resource occupancy rate;

[0031] Judge whether the third hardware resource occupancy rate is consistent with the fourth hardware resource occupancy rate;

[0032] If not, adjust the parameters of the current prediction model, and based on the adjusted current prediction model, return to the step of based on the sample data, using the current prediction model to make a prediction to obtain the fourth hardware resource occupancy rate until the third hardware resource occupancy rate is consistent with the fourth hardware resource occupancy rate to obtain the preset prediction model.

[0033] In addition, to achieve the above object, the present application also proposes a data management device, and the data management device includes:

[0034] A receiving module, which is configured to determine target data to be managed by a second virtual machine in response to a data management instruction received by a first virtual machine;

[0035] A data management module, which is configured to call the second virtual machine to manage the target data based on a preset network connection channel. Wherein, the first virtual machine and the second virtual machine are built based on the same chip, the preset network connection channel is built based on the first virtual machine and the second virtual machine being in the same network environment, the preset network connection channel is used for data transmission between the first virtual machine and the second virtual machine, the first virtual machine includes a router module, the second virtual machine includes a NAS module, and the data management includes data storage and data reading.

[0036] In one embodiment, the data management device further includes an adjustment resource module, and the adjustment resource module includes:

[0037] A response unit, which is configured to determine the current network traffic and management requirements based on the data management instruction, and the management requirements include storage requirements and reading requirements;

[0038] A first prediction unit, which is configured to predict a first hardware resource occupancy rate corresponding to the first virtual machine and a second hardware resource occupancy rate corresponding to the second virtual machine based on the current network traffic, management requirements, and a preset prediction model. Wherein, the preset prediction model is used to predict the hardware resource occupancy rate corresponding to the virtual machine when executing a storage task;

[0039] A first judgment unit, which is configured to judge whether the first hardware resource occupancy rate and the second hardware resource occupancy rate are greater than a preset hardware occupancy rate threshold. If so, modify the system parameters corresponding to the first virtual machine and the second virtual machine to adjust the hardware resource configurations corresponding to the first virtual machine and the second virtual machine until the first hardware resource occupancy rate and the second hardware resource occupancy rate are less than or equal to the preset hardware occupancy rate threshold, so as to obtain an adjusted first virtual machine and an adjusted second virtual machine;

[0040] In one embodiment, the data management module further includes:

[0041] A management unit, which is configured to call the adjusted second virtual machine to manage the target data based on a preset network connection channel.

[0042] In one embodiment, the data management module further includes:

[0043] A first data storage unit, configured to, if the data management instruction is a data storage instruction, based on a preset network connection channel, call the first virtual machine to send the target data to the second virtual machine, and call the second virtual machine to store the received target data in a target storage layer, where the target data is stored in the first virtual machine before being sent, and the target storage layer includes an SSD cache layer and an HDD storage layer;

[0044] A first data reading unit, configured to, if the data management instruction is a data reading instruction, based on a preset network connection channel, read the target data in the target storage layer, and call the second virtual machine to send the target data to the first virtual machine.

[0045] In one embodiment, the data management module further includes:

[0046] A determination unit, configured to determine the data type of the target data, where the data type of the target data includes a hot data type and a cold data type;

[0047] A second data storage unit, configured to, if the data type of the target data is a hot data type, based on the data storage instruction, call the second virtual machine to store the target data in the SSD cache layer;

[0048] A third data storage unit, configured to, if the data type of the target data is a cold data type, based on the data storage instruction, call the first virtual machine to store the target data in the HDD storage layer.

[0049] In one embodiment, the data management module further includes:

[0050] A second data reading unit, configured to read the cache status corresponding to the SSD cache layer, and based on the cache status and a preset transfer time interval, determine whether multiple groups of hot data in the SSD cache layer meet the transfer condition;

[0051] A fourth data storage unit, configured to, if the condition is met, transfer multiple groups of hot data in the SSD cache layer to the HDD storage layer.

[0052] In one embodiment, the data management module further includes:

[0053] A first acquisition unit, configured to acquire the number of times each group of hot data in the HDD storage layer is read;

[0054] A fifth data storage unit, configured to, based on the number of times read, store each group of hot data in the HDD storage layer in descending order of the number of times read in the SSD cache layer until the data volume cached in the SSD cache layer reaches a preset data volume threshold.

[0055] In one embodiment, the data management further includes a model training module, and the model training module includes:

[0056] A second acquisition unit, configured to acquire sample data, and a predicted result corresponding to the sample data is a third hardware resource occupancy rate;

[0057] A second prediction unit, configured to perform prediction based on the sample data by using a current prediction model to obtain a fourth hardware resource occupancy rate;

[0058] A second judgment unit, configured to judge whether the third hardware resource occupancy rate is consistent with the fourth hardware resource occupancy rate;

[0059] A training unit, configured to, if inconsistent, adjust parameters of the current prediction model, and based on the adjusted current prediction model, return to the step of performing prediction based on the sample data by using the current prediction model to obtain a fourth hardware resource occupancy rate, until the third hardware resource occupancy rate is consistent with the fourth hardware resource occupancy rate, so as to obtain a preset prediction model.

[0060] In addition, to achieve the above object, the present application further provides a data management device, and the device includes: a memory, a processor, and a computer program stored on the memory and executable on the processor, and the computer program is configured to implement the steps of the data management method as described above.

[0061] In addition, to achieve the above object, the present application further provides a storage medium, and the storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium, and when the computer program is executed by a processor, the steps of the data management method as described above are implemented.

[0062] In addition, to achieve the above object, the present application further provides a computer program product, and the computer program product includes a computer program, and when the computer program is executed by a processor, the steps of the data management method as described above are implemented.

[0063] One or more technical solutions provided by the present application have at least the following technical effects:

[0064] The present application proposes a data management method and related devices, which relate to the technical field of data management. Compared with the related art where the implementation of a traditional NAS system requires at least one NAS and one router, resulting in high costs, in the present application, first, in response to a data management instruction received by a first virtual machine, target data to be managed by a second virtual machine is determined. Further, based on a preset network connection channel, the second virtual machine is called to manage the target data, where the first virtual machine and the second virtual machine are built based on the same chip, the preset network connection channel is built based on the first virtual machine and the second virtual machine being in the same network environment, the preset network connection channel is used for data transmission between the first virtual machine and the second virtual machine, the first virtual machine includes a router module, the second virtual machine includes a NAS module, and the data management includes data storage and data reading.

[0065] It can be understood that in the present application, a data management device only builds a first virtual machine (equivalent to a router) and a second virtual machine (equivalent to a NAS) based on one chip. The first virtual machine completes communication and interaction with the outside (receives data management instructions), and the second virtual machine manages the data to be managed. In the related art, one NAS and one router correspond to two independent chips. In comparison, the present application reduces the number of chips in the NAS system, and thus reduces the cost of implementing the NAS system. BRIEF DESCRIPTION OF THE DRAWINGS

[0066] The accompanying drawings herein are incorporated into the specification and form a part of the specification, showing embodiments consistent with the present application and used together with the specification to explain the principles of the present application.

[0067] To more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, for those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0068] Figure 1 It is a schematic flow chart provided for the first embodiment of the data management method of the present application;

[0069] Figure 2 It is a schematic diagram of a data management system provided for the first embodiment of the data management method of the present application;

[0070] Figure 3 It is a schematic flow chart provided for the second embodiment of the data management method of the present application;

[0071] Figure 4 It is a schematic flow chart provided for the third embodiment of the data management method of the present application;

[0072] Figure 5 It is a schematic diagram of the module structure of the data management device according to an embodiment of the present application;

[0073] Figure 6 It is a schematic diagram of the device structure of the hardware operating environment involved in the data management method according to an embodiment of the present application.

[0074] The realization of the purpose, functional features and advantages of the present application will be further described with reference to the embodiments and the accompanying drawings. Specific embodiments

[0075] It should be understood that the specific embodiments described herein are only used to explain the technical solutions of the present application and are not used to limit the present application.

[0076] In order to better understand the technical solutions of the present application, the following will be described in detail in conjunction with the accompanying drawings of the specification and specific embodiments.

[0077] The main solution of the embodiment of the present application is:

[0078] In this embodiment, for the convenience of description, the following will be described with a data management device as the execution subject.

[0079] Due to the prior art: The implementation of the traditional NAS system requires at least one NAS and one router (two chips), and the cost is too high.

[0080] The present application provides a solution, so that: in response to a data management instruction received by the first virtual machine, the second virtual machine is called to manage the data to be managed, wherein the first virtual machine and the second virtual machine are built based on the same chip, the first virtual machine is used for network communication, the second virtual machine is used for data management, and the data management includes data storage and data reading.

[0081] It can be understood that in the present application, the data management device only builds the first virtual machine (equivalent to a router) and the second virtual machine (equivalent to a NAS) based on one chip, completes communication and interaction with the outside through the first virtual machine (receives data management instructions), and manages the data to be managed through the second virtual machine. In the related art, one NAS and one router correspond to two independent chips. In comparison, the present application reduces the number of chips in the NAS system, and thus reduces the cost of implementing the NAS system.

[0082] It should be noted that the execution subject of this embodiment can be a computing service device with data management, network communication and program running functions, such as a tablet computer, a personal computer, a mobile phone, etc., or an electronic device, a data management device, etc. that can implement the above functions. The following will take the data management device as an example to illustrate this embodiment and the following embodiments.

[0083] Based on this, an embodiment of the present application provides a data management method. Referring to Figure 1 , Figure 1 which is a schematic flowchart of the first embodiment of the data management method of the present application.

[0084] In this embodiment, the data management method includes steps S10 to S20:

[0085] Step S10, in response to a data management instruction received by the first virtual machine, determine the target data that needs to be managed by the second virtual machine;

[0086] It should be noted that the execution subject of this embodiment can be a computing service device with data management, network communication, and program running functions, such as a tablet computer, a personal computer, a mobile phone, etc., or an electronic device, a data management device, etc. that can implement the above functions. Hereinafter, a data management device is taken as an example to illustrate this embodiment and the following embodiments.

[0087] It should be noted that a virtual machine: an independent computer system simulated by software on a physical computer (host computer), which has its own operating system and application programs and can run like a real computer. Multiple virtual machines can run on the same host computer, and they are isolated from each other, but can communicate and interact with data through a specific mechanism.

[0088] It should be noted that referring to Figure 2 , in the present application, both the first virtual machine and the second virtual machine are constructed by the data management device using the kvm virtual machine model. The first virtual machine has a router module, which is equivalent to the router in the NAS system. The second virtual machine has a data management module for managing data storage and reading, which is equivalent to the NAS in the NAS system. The first virtual machine and the second virtual machine are isolated from each other. Additionally, when constructing the second virtual machine, the storage and expansion interfaces that need to be shared are allocated to virtual machine 1.

[0089] It should be noted that network attached storage (NAS system), as an efficient data storage and sharing solution, taking data storage as an example, the router is responsible for receiving data storage instructions and receiving the data to be stored. Then, according to the data storage instructions, the router determines the target data in the data to be stored that needs to be stored in the NAS. Further, based on the network channel established under the same network environment, the router sends the target data to the NAS for storage.

[0090] In the present application, the specific application scenario can be:

[0091] First, the data management device receives a data management instruction based on the first virtual machine. Then, based on the data management instruction, the data management device determines the target data to be managed based on the first virtual machine.

[0092] It should be noted that the target data is the data that needs to be managed by the second virtual machine as determined by the data management instruction. It can be various forms of data sets such as files, database records, real-time generated data streams, etc., depending on the type of instruction and the application scenario.

[0093] It can be understood that when storing or reading data, different modules manage different data, and not all data is managed using the second virtual machine. Therefore, after receiving the data management instruction, it is necessary to first determine the target data (the data that needs to be managed by the second virtual machine), and then the data management device calls the second virtual machine to manage the target data.

[0094] Step S20, based on a preset network connection channel, call the second virtual machine to manage the target data, where the first virtual machine and the second virtual machine are built based on the same chip, the preset network connection channel is built based on the first virtual machine and the second virtual machine being in the same network environment, the preset network connection channel is used for data transmission between the first virtual machine and the second virtual machine, the first virtual machine includes a router module, the second virtual machine includes a NAS module, and the data management includes data storage and data reading.

[0095] It should be noted that the preset network connection channel is a pre-set network path for data transmission, and this channel is specifically built and optimized for communication between virtual machines. This channel can ensure stable and efficient data transmission between the first virtual machine and the second virtual machine, and it is the basis for the two virtual machines to work together.

[0096] It should be noted that both virtual machines (the first virtual machine and the second virtual machine) are created and run based on the same physical chip. This architecture can make full use of the resources of the chip, improve resource utilization, and because the two virtual machines have natural affinity at the hardware level, they can work together better and reduce compatibility problems and performance losses that may be caused by hardware differences.

[0097] Specifically, the step of calling the second virtual machine to manage the target data based on the preset network connection channel further includes any one of steps S21 to S22:

[0098] Step S21: If the data management instruction is a data storage instruction, based on a preset network connection channel, call the first virtual machine to send the target data to the second virtual machine, and call the second virtual machine to store the received target data in the target storage layer. The target data is stored in the first virtual machine before being sent. The target storage layer includes an SSD cache layer and an HDD storage layer.

[0099] It can be understood that when the data management device receives a data storage instruction based on the first virtual machine, it means that specific target data needs to be saved. Furthermore, the data management device uses a pre-set network path dedicated to communication between the first virtual machine and the second virtual machine to ensure that the target data can be stably and efficiently transmitted from the first virtual machine to the second virtual machine.

[0100] After the second virtual machine receives the data sent from the first virtual machine, it will use its internal NAS module to store the data in the target storage layer according to the established storage policies and rules.

[0101] It should be noted that the SSD cache layer utilizes the high-performance characteristics of solid-state drives as a high-speed cache space in the system. It is mainly used to temporarily store frequently accessed data to accelerate data reading and writing speeds and improve the overall response ability of the system. SSDs have the advantages of fast read and write speeds, no mechanical components, and strong seismic resistance, which can significantly improve the system's performance, especially in handling a large number of random read and write operations. The storage cost of HDDs is relatively low, and it can provide a large storage capacity at a low price, suitable for storing a large amount of historical data, backup data, etc.

[0102] It should be noted that the HDD storage layer is the underlying storage device in the system for long-term storage of a large amount of data. It provides a large storage space for storing data that is not frequently accessed or data with relatively low requirements for read and write speeds.

[0103] It can be understood that in this embodiment, by assigning the data storage task to a dedicated second virtual machine and using its NAS module, more efficient and professional data storage management can be achieved. At the same time, the hierarchical storage strategy of the SSD cache layer and the HDD storage layer takes into account both the performance and cost of data storage. At the same time, based on the preset network connection channel and the router module in the first virtual machine, the stability and security of data during transmission are ensured, avoiding data loss or incorrect storage caused by network problems.

[0104] Specifically, the step of calling the second virtual machine to store the received target data in the target storage layer further includes steps A10 to A30:

[0105] Step A10, determine the data type of the target data, where the data type of the target data includes a hot data type and a cold data type;

[0106] It should be noted that cold data refers to data that is not frequently accessed and usually has a lower access priority, such as surveillance video data, etc.; while hot data refers to data that is frequently accessed and used and has a higher access priority.

[0107] Step A20, if the data type of the target data is a hot data type, based on the data storage instruction, call the second virtual machine to store the target data in the SSD cache layer;

[0108] Step A30, if the data type of the target data is a cold data type, based on the data storage instruction, call the first virtual machine to store the target data in the HDD storage layer.

[0109] It can be understood that in this embodiment, for data of different data types, a hierarchical storage strategy is adopted, and data with a higher access priority is stored in the SSD cache layer with cache space to facilitate improving the read efficiency when reading this data next time, that is, improving the data management efficiency.

[0110] Specifically, after the step of calling the second virtual machine to store the target data in the SSD cache layer based on the data storage instruction, steps A40 - A50 are further included:

[0111] Step A40, read the cache status corresponding to the SSD cache layer, and based on the cache status and a preset transfer time interval, determine whether multiple groups of hot data in the SSD cache layer meet the transfer conditions;

[0112] It should be noted that the cache status corresponding to the SSD cache layer refers to the cache space occupancy rate of the SSD cache layer.

[0113] Step A50, if satisfied, transfer multiple groups of hot data in the SSD cache layer to the HDD storage layer.

[0114] It can be understood that in this embodiment, the SSD cache layer is only used to cache data with a higher access priority to improve the data read efficiency. However, for the NAS module, the actual storage space is the HDD storage layer, and a large amount of data is stored based on the HDD storage layer.

[0115] Specifically, after the step of transferring multiple groups of hot data in the SSD cache layer to the HDD storage layer, steps A60 - A70 are further included:

[0116] Step A60: Obtain the read times of each group of hot data in the HDD storage layer;

[0117] Step A70: Based on the read times, store each group of hot data in the HDD storage layer into the SSD cache layer in descending order of read times until the data volume cached in the SSD cache layer reaches a preset data volume threshold.

[0118] It can be understood that in this embodiment, as the stored data increases, the access priority of the hot data in the SSD cache layer may be lower than that of the hot data in the HDD storage layer. At this time, it is necessary to store the hot data in descending order of access priority (read times) into the SSD cache layer according to all the hot data in the SSD cache layer and the HDD storage layer until the data volume cached in the SSD cache layer reaches a preset data volume threshold, so that the data in the SSD cache layer is always the multi-group hot data with the highest access priority. Furthermore, the data management efficiency is improved.

[0119] Step S22: If the data management instruction is a data read instruction, read the target data in the target storage layer based on a preset network connection channel, and call the second virtual machine to send the target data to the first virtual machine.

[0120] This application proposes a data management method and related devices, which relates to the technical field of data management. Compared with the related art, in which the implementation of a traditional NAS system requires at least one NAS and one router and the cost is too high, in this application, first, in response to a data management instruction received by the first virtual machine, determine the target data to be managed by the second virtual machine. Further, based on a preset network connection channel, call the second virtual machine to manage the target data, where the first virtual machine and the second virtual machine are built based on the same chip, the preset network connection channel is built based on the fact that the first virtual machine and the second virtual machine are in the same network environment, the preset network connection channel is used for data transmission between the first virtual machine and the second virtual machine, the first virtual machine includes a router module, the second virtual machine includes a NAS module, and the data management includes data storage and data reading.

[0121] It can be understood that in this application, the data management device only builds a first virtual machine (equivalent to a router) and a second virtual machine (equivalent to a NAS) based on one chip, completes communication and interaction with the outside through the first virtual machine (receiving data management instructions), and manages the data to be managed through the second virtual machine. In the related art, one NAS and one router correspond to two independent chips. Compared with this, this application reduces the number of chips in the NAS system, and thus reduces the cost of implementing the NAS system.

[0122] Based on the first embodiment of this application, in the second embodiment of this application, for the same or similar content as in the above-mentioned first embodiment, reference can be made to the above introduction and will not be elaborated hereinafter. On this basis, please refer to Figure 3 , before step S20, the data management method further includes steps S11 to S13:

[0123] Step S11, based on the data management instruction, determine the current network traffic and management requirements, where the management requirements include storage requirements and reading requirements;

[0124] It should be noted that network traffic reflects the busyness of the network and the load of data transmission, which is crucial for subsequent adjustment of data management strategies.

[0125] For example, in the server of an e-commerce website, through a network traffic monitoring tool, it is possible to see in real time the amount of data flowing out of the server every second. For example, during a promotional event, the network traffic may increase significantly because a large number of users access the website simultaneously, and operations such as viewing product information and placing orders will generate a large amount of data transmission. Understanding this traffic peak situation helps enterprises prepare for server expansion in advance to prevent the website from crashing due to excessive traffic.

[0126] It should be noted that management requirements refer to the amount of data that the system needs to store (or read), the data type, and the storage (or reading) time length, etc.

[0127] Step S12, based on the current network traffic, management requirements, and a preset prediction model, predict the occupancy rate of the first hardware resources corresponding to the first virtual machine, and predict the occupancy rate of the second hardware resources corresponding to the second virtual machine, where the preset prediction model is used to predict the occupancy rate of the hardware resources corresponding to the virtual machine when performing a storage task;

[0128] It should be noted that the preset prediction model is the currently trained prediction model, and the prediction model is used to predict the occupancy rate of the hardware resources corresponding to the virtual machine when performing a storage task.

[0129] It should be noted that the hardware resource occupancy rate refers to the degree or proportion of various internal resources (such as CPU, memory, disk, network, etc.) of hardware devices such as computer systems, servers, and network devices being used during operation. The hardware resource occupancy rate reflects the utilization of hardware resources and is an important indicator for evaluating system performance, optimizing resource allocation, and troubleshooting.

[0130] Additionally, it should be noted that generally, a hardware resource occupancy rate below 70% indicates that the system runs relatively easily and can handle tasks with ease; a hardware resource occupancy rate between 70% - 90% indicates that the system is under a relatively high load but can still run normally; exceeding 90% indicates that the system load is too high, and problems such as lag and slow response may occur, and it is necessary to consider optimizing processes or upgrading the CPU.

[0131] Step S13, determine whether the first hardware resource occupancy rate and the second hardware resource occupancy rate are greater than a preset hardware occupancy rate threshold. If so, modify the system parameters corresponding to the first virtual machine and the second virtual machine to adjust the hardware resource configurations corresponding to the first virtual machine and the second virtual machine until the first hardware resource occupancy rate and the second hardware resource occupancy rate are less than or equal to the preset hardware occupancy rate threshold, obtaining the adjusted first virtual machine and the adjusted second virtual machine.

[0132] It can be understood that in this embodiment, based on the prediction result of the preset prediction model, the system parameters corresponding to the first virtual machine and the second virtual machine are modified to adjust the hardware resource configurations corresponding to the first virtual machine and the second virtual machine, so that the first virtual machine and the second virtual machine can always run stably when processing corresponding tasks, avoiding affecting the data management efficiency due to too high a load.

[0133] Furthermore, the step of invoking the second virtual machine to manage the target data based on the preset network connection channel further includes:

[0134] Based on the preset network connection channel, invoke the adjusted second virtual machine to manage the target data.

[0135] It can be understood that in this application, using the adjusted second virtual machine to complete the corresponding tasks can ensure that the second virtual machine always runs efficiently and stably, avoiding affecting the data management efficiency due to too high a load.

[0136] Based on the first embodiment and the second embodiment of this application, in the third embodiment of this application, the same or similar content as in the above-mentioned first embodiment can be referred to the above introduction and will not be repeated hereinafter. On this basis, please refer to Figure 4 , before step S12, the data management method further includes steps B10 - B40:

[0137] Step B10, obtain sample data, and the prediction result corresponding to the sample data is the third hardware resource occupancy rate;

[0138] It should be noted that the sample data is used to train the current prediction model to improve the prediction accuracy of the current prediction model.

[0139] Step B20: Based on the sample data, use the current prediction model to make a prediction to obtain the fourth hardware resource occupancy rate.

[0140] Step B30: Determine whether the third hardware resource occupancy rate is consistent with the fourth hardware resource occupancy rate.

[0141] Step B40: If they are inconsistent, adjust the parameters of the current prediction model. Based on the adjusted current prediction model, return to the step of using the current prediction model to make a prediction based on the sample data to obtain the fourth hardware resource occupancy rate until the third hardware resource occupancy rate is consistent with the fourth hardware resource occupancy rate, and obtain the preset prediction model.

[0142] It can be understood that the third hardware resource occupancy rate is the actual occupancy result corresponding to the sample data, and the fourth hardware resource occupancy rate is the predicted occupancy result corresponding to the sample data based on the current prediction model. The predicted occupancy result may not be consistent with the actual occupancy result, and thus the model is trained.

[0143] It should be noted that the consistency between the predicted occupancy result and the actual occupancy result does not mean that the predicted occupancy result is exactly the same as the actual occupancy result. As long as the predicted loss value corresponding to the predicted occupancy result and the actual occupancy result is within the range set by the staff, it is considered that the predicted occupancy result is consistent with the actual occupancy result, which is convenient for reducing the training amount of model training and improving the training efficiency of the model.

[0144] It can be understood that in this embodiment, the model is trained based on the sample data to improve the prediction accuracy of the model. Furthermore, a more accurate prediction result is obtained, which is convenient for more accurately adjusting the hardware resource configuration of the virtual machine in the subsequent steps.

[0145] It should be noted that the above examples are only for understanding the present application and do not constitute a limitation on the data management method of the present application. Based on this technical concept, more forms of simple transformations are within the protection scope of the present application.

[0146] The present application also provides a data management device. Please refer to Figure 5 The data management device includes:

[0147] A receiving module 10, which is used to determine the target data to be managed by the second virtual machine in response to the data management instruction received by the first virtual machine.

[0148] Data management module 20, which is used to manage the target data by invoking the second virtual machine based on a preset network connection channel. Herein, the first virtual machine and the second virtual machine are built based on the same chip, the preset network connection channel is built based on the fact that the first virtual machine and the second virtual machine are in the same network environment, the preset network connection channel is used for data transmission between the first virtual machine and the second virtual machine, the first virtual machine includes a router module, the second virtual machine includes a NAS module, and the data management includes data storage and data reading.

[0149] In one embodiment, the data management device further includes an adjustment resource module, and the adjustment resource module includes:

[0150] A response unit, which is used to determine the current network traffic and management requirements based on a data management instruction, and the management requirements include storage requirements and reading requirements;

[0151] A first prediction unit, which is used to predict the occupancy rate of the first hardware resources corresponding to the first virtual machine and the occupancy rate of the second hardware resources corresponding to the second virtual machine based on the current network traffic, management requirements and a preset prediction model. Herein, the preset prediction model is used to predict the occupancy rate of the hardware resources corresponding to the virtual machine when performing a storage task;

[0152] A first judgment unit, which is used to judge whether the occupancy rate of the first hardware resources and the occupancy rate of the second hardware resources are greater than a preset hardware occupancy rate threshold. If so, modify the system parameters corresponding to the first virtual machine and the second virtual machine to adjust the hardware resource configuration corresponding to the first virtual machine and the second virtual machine until the occupancy rate of the first hardware resources and the occupancy rate of the second hardware resources are less than or equal to the preset hardware occupancy rate threshold, so as to obtain an adjusted first virtual machine and an adjusted second virtual machine;

[0153] In one embodiment, the data management module further includes:

[0154] A management unit, which is used to manage the target data by invoking the adjusted second virtual machine based on a preset network connection channel.

[0155] In one embodiment, the data management module further includes:

[0156] A first data storage unit, which is used to, if the data management instruction is a data storage instruction, based on a preset network connection channel, invoke the first virtual machine to send the target data to the second virtual machine, and invoke the second virtual machine to store the received target data in a target storage layer. The target data is stored in the first virtual machine before being sent. Herein, the target storage layer includes an SSD cache layer and an HDD storage layer;

[0157] A first data reading unit, configured to, if the data management instruction is a data reading instruction, read the target data in the target storage layer based on a preset network connection channel, and call a second virtual machine to send the target data to a first virtual machine.

[0158] In one embodiment, the data management module further includes:

[0159] A determination unit, configured to determine the data type of the target data, where the data type of the target data includes a hot data type and a cold data type;

[0160] A second data storage unit, configured to, if the data type of the target data is a hot data type, call a second virtual machine to store the target data into an SSD cache layer based on the data storage instruction;

[0161] A third data storage unit, configured to, if the data type of the target data is a cold data type, call a first virtual machine to store the target data into an HDD storage layer based on the data storage instruction.

[0162] In one embodiment, the data management module further includes:

[0163] A second data reading unit, configured to read the cache status corresponding to the SSD cache layer, and based on the cache status and a preset transfer time interval, determine whether multiple groups of hot data in the SSD cache layer meet the transfer condition;

[0164] A fourth data storage unit, configured to, if the condition is met, transfer multiple groups of hot data in the SSD cache layer to the HDD storage layer.

[0165] In one embodiment, the data management further includes a model training module, and the model training module includes:

[0166] A second acquisition unit, configured to acquire sample data, and a prediction result corresponding to the sample data is a third hardware resource occupancy rate;

[0167] A fifth data storage unit, configured to, based on the reading times, store each group of hot data in the HDD storage layer into the SSD cache layer in descending order of the reading times until the data volume cached in the SSD cache layer reaches a preset data volume threshold.

[0168] In one embodiment, the data management further includes a model training module, and the model training module includes:

[0169] A second acquisition unit, configured to acquire sample data, and a prediction result corresponding to the sample data is a third hardware resource occupancy rate;

[0170] A second prediction unit, configured to perform a prediction using a current prediction model based on the sample data to obtain a fourth hardware resource occupancy rate;

[0171] A second determination unit, configured to determine whether the third hardware resource occupancy rate is consistent with the fourth hardware resource occupancy rate;

[0172] A training unit, configured to, if they are not consistent, adjust parameters of the current prediction model, and based on the adjusted current prediction model, return to the step of performing a prediction using the current prediction model based on the sample data to obtain a fourth hardware resource occupancy rate, until the third hardware resource occupancy rate is consistent with the fourth hardware resource occupancy rate, so as to obtain a preset prediction model.

[0173] The data management device provided in this application adopts the data management method in the above embodiment, and can solve the technical problems of data management. Compared with the prior art, the beneficial effects of the data management device provided in this application are the same as those of the data management method provided in the above embodiment, and other technical features in the data management device are the same as the features disclosed in the method of the above embodiment, and will not be elaborated herein.

[0174] This application provides a data management device, where the data management device includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein, the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the data management method in the first embodiment above.

[0175] Next, refer to Figure 6 , which shows a schematic structural diagram of a data management device suitable for implementing the embodiments of this application. The data management device in the embodiments of this application may include, but is not limited to, mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Descriptions: tablet computers), PMPs (Portable Media Players), in-vehicle terminals (such as in-vehicle navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 6 The shown data management device is only an example, and should not bring any limitation to the functions and usage scopes of the embodiments of this application.

[0176] As Figure 6As shown, the data management device may include a processing device 1001 (such as a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 1002 or a program loaded from a storage device 1003 into a random access memory (RAM) 1004. In the RAM 1004, various programs and data required for the operation of the data management device are also stored. The processing device 1001, the ROM 1002, and the RAM 1004 are connected to each other through a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Generally, the following systems can be connected to the I / O interface 1006: an input device 1007 including, for example, a touch screen, a touch pad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; an output device 1008 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 1003 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 1009. The communication device 1009 can allow the data management device to communicate with other devices wirelessly or wiredly to exchange data. Although the figure shows a data management device with various systems, it should be understood that it is not required to implement or have all the shown systems. Instead, more or fewer systems can be implemented or had.

[0177] Specifically, according to the embodiments disclosed in the present application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, the embodiments disclosed in the present application include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program contains program codes for executing the methods shown in the flowcharts. In such an embodiment, the computer program can be downloaded and installed from a network through the communication device, or installed from the storage device 1003, or installed from the ROM 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the methods of the embodiments disclosed in the present application are executed.

[0178] The data management device provided by the present application adopts the data management method in the above-mentioned embodiment and can solve the technical problems of data management. Compared with the prior art, the beneficial effects of the data management device provided by the present application are the same as those of the data management method provided by the above-mentioned embodiment, and the other technical features in the data management device are the same as those disclosed in the method of the previous embodiment and will not be elaborated here.

[0179] It should be understood that each part disclosed in this application can be implemented by hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics can be combined in a suitable manner in any one or more embodiments or examples.

[0180] As described above, only the specific embodiments of this application are provided, but the protection scope of this application is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed in this application, and all should be covered within the protection scope of this application. Therefore, the protection scope of this application should be subject to the protection scope of the claims.

[0181] This application provides a computer-readable storage medium with computer-readable program instructions (i.e., computer programs) stored thereon, and the computer-readable program instructions are used to execute the data management method in the above embodiments.

[0182] The computer-readable storage medium provided by this application can be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems or devices, or any combination of the above. More specific examples of computer-readable storage media can include, but are not limited to: electrical connections with one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM) or flash memory, optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the above. In this embodiment, the computer-readable storage medium can be any tangible medium that contains or stores a program, and this program can be used by or combined with an instruction execution system or device. The program code contained on the computer-readable storage medium can be transmitted by any appropriate medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination of the above.

[0183] The above computer-readable storage medium can be included in a data management device; it can also exist separately without being assembled into the data management device.

[0184] The above computer-readable storage medium carries one or more programs, and when the above one or more programs are executed by a data management device, the data management device is caused to:

[0185] In response to a data management instruction received by a first virtual machine, determine target data that needs to be managed by a second virtual machine;

[0186] Based on a preset network connection channel, invoke the second virtual machine to manage the target data, where the first virtual machine and the second virtual machine are built based on the same chip, the preset network connection channel is built based on the first virtual machine and the second virtual machine being in the same network environment, the preset network connection channel is used for data transmission between the first virtual machine and the second virtual machine, the first virtual machine includes a router module, the second virtual machine includes a NAS module, and the data management includes data storage and data reading.

[0187] Computer program code for performing the operations of this application can be written in one or more programming languages or combinations thereof. The above-mentioned programming languages include object-oriented programming languages such as Java, Smalltalk, C++, and also include conventional procedural programming languages such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, executed as an independent software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computer (for example, by using an Internet service provider to connect through the Internet).

[0188] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architectures, functions, and operations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in the flowchart or block diagram can represent a module, a program segment, or a part of code that contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than marked in the accompanying drawings. For example, two consecutive blocks shown may actually be executed substantially in parallel, and they may sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagram and / or flowchart, and the combination of blocks in the block diagram and / or flowchart, can be implemented by a dedicated hardware-based system for performing the specified functions or operations, or can be implemented by a combination of dedicated hardware and computer instructions.

[0189] The modules involved in the embodiments of this application can be implemented in software or in hardware. In some cases, the name of the module does not constitute a limitation on the unit itself.

[0190] The readable storage medium provided by this application is a computer-readable storage medium. The computer-readable storage medium stores computer-readable program instructions (i.e., computer programs) for executing the above data management method, and can solve the technical problems of data management. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided by this application are the same as those of the data management method provided by the above embodiments, and will not be elaborated here.

[0191] This application also provides a computer program product, including a computer program, and the steps of the data management method as described above are implemented when the computer program is executed by a processor.

[0192] The computer program product provided by this application can solve the technical problems of data management. Compared with the prior art, the beneficial effects of the computer program product provided by this application are the same as those of the data management method provided by the above embodiments, and will not be elaborated here.

[0193] The above are only some embodiments of this application, and do not limit the patent scope of this application. Any equivalent structural transformation made under the technical concept of this application by using the content of the specification and drawings of this application, or any direct / indirect application in other related technical fields, is included in the patent protection scope of this application.

Claims

1. A data management method, characterized in that, The described data management method includes: In response to a data management instruction received by a first virtual machine, determining target data that needs to be managed by a second virtual machine; Based on a preset network connection channel, invoking the second virtual machine to manage the target data, wherein the first virtual machine and the second virtual machine are built based on the same chip, the preset network connection channel is built based on the first virtual machine and the second virtual machine being in the same network environment, the preset network connection channel is used for data transmission between the first virtual machine and the second virtual machine, the first virtual machine includes a router module, the second virtual machine includes a NAS module, and the data management includes data storage and data reading.

2. The data management method according to claim 1, wherein Before the step of invoking the second virtual machine to manage the target data based on the preset network connection channel, it further includes: Based on the data management instruction, determining the current network traffic and management requirements, where the management requirements include storage requirements and reading requirements; Based on the current network traffic, management requirements, and a preset prediction model, predicting the occupancy rate of the first hardware resources corresponding to the first virtual machine and predicting the occupancy rate of the second hardware resources corresponding to the second virtual machine, wherein the preset prediction model is used to predict the occupancy rate of hardware resources corresponding to the virtual machine when executing a storage task; Judging whether the occupancy rate of the first hardware resources and the occupancy rate of the second hardware resources are greater than a preset hardware occupancy rate threshold. If so, modifying the system parameters corresponding to the first virtual machine and the second virtual machine to adjust the hardware resource configuration corresponding to the first virtual machine and the second virtual machine until the occupancy rate of the first hardware resources and the occupancy rate of the second hardware resources are less than or equal to the preset hardware occupancy rate threshold, obtaining an adjusted first virtual machine and an adjusted second virtual machine; The step of invoking the second virtual machine to manage the target data based on the preset network connection channel further includes: Based on the preset network connection channel, invoking the adjusted second virtual machine to manage the target data.

3. The data management method according to claim 1, characterized in that The step of invoking the second virtual machine to manage the target data based on the preset network connection channel further includes any one of the following: If the data management instruction is a data storage instruction, based on the preset network connection channel, invoking the first virtual machine to send the target data to the second virtual machine, and invoking the second virtual machine to store the received target data in a target storage layer, where the target data is stored in the first virtual machine before being sent, and the target storage layer includes an SSD cache layer and an HDD storage layer; If the data management instruction is a data reading instruction, based on the preset network connection channel, reading the target data in the target storage layer and invoking the second virtual machine to send the target data to the first virtual machine.

4. The data management method according to claim 3, characterized in that, The step of invoking the second virtual machine to store the received target data in the target storage layer further includes: Determining the data type of the target data, where the data type of the target data includes a hot data type and a cold data type; If the data type of the target data is a hot data type, based on the data storage instruction, call the second virtual machine to store the target data in the SSD cache layer; If the data type of the target data is a cold data type, based on the data storage instruction, call the first virtual machine to store the target data in the HDD storage layer.

5. The data management method according to claim 4, wherein After the step of calling the second virtual machine to store the target data in the SSD cache layer based on the data storage instruction, the following steps are further included: Read the cache status corresponding to the SSD cache layer, and based on the cache status and a preset transfer time interval, determine whether multiple groups of hot data in the SSD cache layer meet the transfer conditions; If satisfied, transfer multiple groups of hot data in the SSD cache layer to the HDD storage layer.

6. The data management method according to claim 4, wherein After the step of transferring multiple groups of hot data in the SSD cache layer to the HDD storage layer, the following steps are further included: Obtain the read times of each group of hot data in the HDD storage layer; Based on the read times, store each group of hot data in the HDD storage layer in descending order of the read times in the SSD cache layer in sequence until the data volume cached in the SSD cache layer reaches a preset data volume threshold.

7. The data management method according to claim 2, characterized in that, Before the step based on the current network traffic, management requirements, and preset prediction model, the following steps are further included: Obtain sample data, and the prediction result corresponding to the sample data is the third hardware resource occupancy rate; Based on the sample data, use the current prediction model for prediction to obtain the fourth hardware resource occupancy rate; Judge whether the third hardware resource occupancy rate is consistent with the fourth hardware resource occupancy rate; If not consistent, adjust the parameters of the current prediction model, and based on the adjusted current prediction model, return to the step of using the current prediction model for prediction based on the sample data to obtain the fourth hardware resource occupancy rate until the third hardware resource occupancy rate is consistent with the fourth hardware resource occupancy rate to obtain a preset prediction model.

8. A data management device, characterized in that, The data management includes: A receiving module, which is used to determine the target data that needs to be managed by the second virtual machine in response to the data management instruction received by the first virtual machine; A data management module, which is used to call the second virtual machine to manage the target data based on a preset network connection channel. Among them, the first virtual machine and the second virtual machine are built based on the same chip, the preset network connection channel is built based on the first virtual machine and the second virtual machine being in the same network environment, the preset network connection channel is used for data transmission between the first virtual machine and the second virtual machine, the first virtual machine includes a router module, the second virtual machine includes a NAS module, and the data management includes data storage and data reading.

9. A data management device, characterized in that, The device includes: a memory, a processor, and a computer program stored on the memory and executable on the processor. The computer program is configured to implement the steps of the data management method according to any one of claims 1 to 7.

10. A storage medium, characterized in that, The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, the steps of the data management method according to any one of claims 1 to 7 are implemented.

Images