Data storage method and device, storage medium and electronic equipment

Abstract

The invention discloses a data storage method and device, a storage medium and electronic equipment, and relates to the technical field of computers.The method comprises the steps that firstly, a data storage request of a user is received; then, under the condition that the network connection with the network file system server is abnormal, obtaining a target node which is in normal network connection with the network file system server in the cluster; acquiring a mounting point of the target node through a secure file transfer protocol; and finally processing the data storage request based on the mounting point. Under the condition that the network connection between the current node and the network file system server is abnormal, a target node which is normally connected with the network file system server can be selected from other nodes of the cluster, connection with the target node is established based on a secure file transfer protocol, and a data storage request is processed through a mounting point of the target node. And the data storage service corresponding to the request is completed, the data storage service can be normally provided, and the reliability of the storage service is improved.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technologies, and in particular, to a data storage method, apparatus, storage medium, and electronic device. Background Art

[0002] With the increase in the complexity of cloud platform services, the services of the cloud platform have become more complex. Data storage is a key service that connects the entire cloud platform services, and its stability is also crucial.

[0003] Currently, in a cloud platform, there are multiple nodes. When a user uses a certain node for data storage, the node needs to connect to a Network File System (NFS) server, and then call the NFS protocol to store data in the NFS server.

[0004] However, if there is a problem with the network connection between the node requested by the user and the NFS server, it will cause the data storage service corresponding to the request to be unable to be completed. Summary of the Invention

[0005] The present disclosure provides a data storage method, apparatus, storage medium, and electronic device. Its main purpose is to solve the problem in the related art that if there is a problem with the network connection between the node requested by the user and the NFS server, it will cause the data storage service corresponding to the request to be unable to be completed.

[0006] In a first aspect, the present application provides a data storage method, including:

[0007] Receiving a data storage request from a user;

[0008] In the case of an abnormal network connection with the network file system server, obtaining a target node in the cluster that has a normal network connection with the network file system server;

[0009] Obtaining the mount point of the target node through a secure file transfer protocol;

[0010] Processing the data storage request based on the mount point.

[0011] In a second aspect, the present application provides a data storage apparatus, including:

[0012] A receiving module configured to receive a data storage request from a user;

[0013] An obtaining module configured to obtain a target node in the cluster that has a normal network connection with the network file system server in the case of an abnormal network connection with the network file system server;

[0014] An obtaining module configured to obtain the mount point of the target node through a secure file transfer protocol;

[0015] A processing module, configured to process a data storage request based on a mount point.

[0016] In a third aspect, the present application provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the method of the first aspect is implemented.

[0017] In a fourth aspect, the present application provides an electronic device, including a storage medium, a processor, and a computer program stored on the storage medium and executable on the processor. When the processor executes the computer program, the method of the first aspect is implemented.

[0018] In a fifth aspect, the present application provides a computer program product, on which a computer program is stored. When the computer program is executed by a processor, the method of the first aspect is implemented.

[0019] The data storage method, apparatus, storage medium, and electronic device provided by the present disclosure, wherein the method includes: first, receiving a data storage request from a user; then, in the case of an abnormal network connection with a network file system server, obtaining a target node in the cluster that has a normal network connection with the network file system server; obtaining the mount point of the target node through a secure file transfer protocol; and finally, processing the data storage request based on the mount point. The present application can receive the data storage request sent by the user through the current node. In the case of an abnormal network connection between the current node and the network file system server, that is, when data storage cannot be performed through the network file system, a target node with a normal network connection with the network file system server can be selected from other nodes in the cluster, a connection with the target node can be established based on the secure file transfer protocol, and the data storage request can be processed through the mount point of the target node to complete the data storage service corresponding to the request. The network file system protocol and the secure file transfer protocol are configured for each node to provide a dual-protocol redundant storage service. Thus, when the network connection between a certain node in the cluster and the network file system server is abnormal, a data storage service can be provided through the secure file transfer protocol, improving the reliability of the storage service.

[0020] It should be understood that the content described in this part is not intended to identify the key or important features of the embodiments of the present application, nor is it used to limit the scope of the present application. Other features of the present application will become easily understood through the following description. Description of the Drawings

[0021] To more clearly illustrate the embodiments of the present application, the following will briefly introduce the drawings required for the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0022] Figure 1 Shows a schematic diagram of an example provided by an embodiment of the present application;

[0023] Figure 2 Shows a schematic flowchart of a data storage method provided by an embodiment of the present application;

[0024] Figure 3 Shows a schematic diagram of another example provided by an embodiment of the present application;

[0025] Figure 4 Shows a schematic diagram of yet another example provided by an embodiment of the present application;

[0026] Figure 5 Shows a schematic structural diagram of a data storage device provided by an embodiment of the present application. Detailed implementation manners

[0027] Next, the technical solutions in the embodiments of the present application will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without making creative efforts belong to the protection scope of the present application.

[0028] It should be noted that in the description of the present application, the terms "include", "comprise" or any other variant thereof are intended to cover a non-exclusive inclusion, so that a process, method, article or device including a series of elements not only includes those elements but also includes other elements not expressly listed, or further includes elements inherent to such process, method, article or device. The terms "first", "second", etc. in the present application are used to distinguish similar objects and are not used to describe a specific order or sequence.

[0029] With the maturity of cloud computing technology, more and more services no longer run on physical servers but use cloud hosts, and the cloud platform for managing cloud hosts plays an important role in this process. Correspondingly, as the complexity of services on the cloud increases, the complexity of the cloud platform's services gradually increases, and storing files is a key service that connects the entire cloud platform's services, and its stability is also crucial. When using storage in a traditional cloud platform, taking NFS storage as an example, each node separately calls the NFS storage. However, when using this method, if there is a problem with the network between this node and the NFS, it will cause the storage service to be unable to operate normally, that is, the requests received by this node cannot be processed.

[0030] In traditional cloud platforms, the global content library is usually implemented through NFS, a protocol that allows nodes to access files on a remote server (such as an NFS server) as if they were local files. The NFS server can be a machine or virtual machine running the NFS service, responsible for exporting (sharing) one or more directories as a network file system. These directories are called "export points" or "shared points" and can be mounted and used by other machines (such as NFS clients). The NFS client can include a machine or virtual machine that connects to the NFS server to access its shared directory. The client can mount the shared directory on the NFS server to an empty local directory (mount point) through the network for remote operations.

[0031] Exemplarily, as Figure 1 shown, a system architecture diagram of the related art is presented. A user can randomly send a storage request to any node in the cluster, such as Node 1, Node 2, or Node 3. Each node represents a computing unit, such as a server or virtual machine, and communicates with the NFS server providing NFS storage services through the network respectively. There is a mount point on each node for mounting the NFS storage directory into the local file system, enabling the node to access remote storage resources as if they were local files. The NFS storage area can be a shared storage area, providing cross-node data access capabilities. Each node is connected to the NFS storage area through its mount point to obtain the required data. When processing a user request, a node needs to read data from or write data to the NFS storage area. However, if there is a problem with the network between a certain node and the NFS storage area, the node will not be able to access the storage resources normally, resulting in service interruption. For example: If the network connection between Node 1 and the NFS storage area is interrupted, Node 1 will not be able to continue processing requests that depend on the NFS storage area. In this case, even if the hardware and software of Node 1 are working properly, it will become unavailable because it cannot access the NFS storage area, lacking effective high availability and self-recovery mechanisms.

[0032] However, this method has the following problems: If there is a problem with the network between a certain node and the NFS server, the node will not be able to access the storage resources it needs, resulting in the inability to carry out the storage service normally; when it is detected that the NFS storage service is unavailable, a simple handling method is to mark the entire node as unavailable and schedule the request to other nodes. This way will cause the node to be idle due to NFS problems even if the hardware and software of the node are working properly, resulting in resource waste; traditional health check mechanisms often only monitor whether the NFS service is available and lack comprehensive health assessment criteria, making it difficult to ensure the high availability and self-recovery of the system. Therefore, when creating a global content library using external storage (NFS) in the related art, there are problems of inability to achieve high availability and single inspection metrics.

[0033] In order to improve the technical problem in the current related technologies that if there is a problem with the network connection between the node requested by the user and the NFS server, the data storage service corresponding to the request cannot be completed.

[0034] This embodiment provides a data storage method, as Figure 2 shown, the method includes the following steps:

[0035] Step 101, receive the user's data storage request.

[0036] Among them, the node may include the node that receives the data storage request sent by the user, and the data storage request may include, but is not limited to, information such as file name, path, operation type (read / write), etc.

[0037] In some embodiments, any node in the cluster can be used to receive the data storage request sent by the user. Each node in the cluster can be configured with the NFS protocol and the secure file transfer protocol. Exemplarily, a cloud platform node can create a mount point, mount NFS, be able to provide the NFS service normally, and deploy SFTP on each node of the cloud platform, and expose the mount point through SFTP, so as to achieve highly available storage with dual-protocol redundancy.

[0038] Specifically, the NFS protocol may refer to a distributed file system protocol that provides file-level data access through a network, allowing users to access files located on remote computers as if they were local files, so as to achieve the purpose of sharing files across multiple computing nodes. Correspondingly, NFS storage may refer to a data storage and sharing solution based on the NFS protocol, which can refer to the NFS server itself (i.e., the machine that provides the shared file system), or the shared storage area provided by the NFS server, that is, the directories that are exported for other machines to access. Through the NFS protocol, one or more storage areas can be shared on the network.

[0039] Exemplarily, in a cloud platform environment, the NFS server can provide a shared storage area, allowing all nodes accessing the platform to be able to access and share the same data resources, so as to build a global content library, and cross-cloud platform file transfer can be achieved. That is, NFS storage involves both the NFS server that provides the service and the specific storage area shared by the server. This setting enables multiple different nodes to efficiently and conveniently share and access the same set of files, and is very suitable for application scenarios that require centralized management and distribution of a large number of files.

[0040] Correspondingly, the Secure File Transfer Protocol (SFTP) is a secure way to provide file access, transfer, and management over a network through the Secure Shell (SSH) protocol. It provides data encryption to ensure the security of the file transfer process and also provides data integrity verification to ensure that the data has not been modified midway. It can simplify the cross-network data exchange process while ensuring data security. The SFTP protocol can be used to establish a transmission connection between different nodes. In this configuration method, in addition to configuring NFS, SFTP is also configured as a backup protocol. When NFS is unavailable, the affected nodes can obtain the required data from other healthy nodes through SFTP to ensure business continuity. Based on the SFTP protocol, the node processing the data storage request is switched to achieve a highly available global content library.

[0041] Step 102: In the case of an abnormal network connection with the network file system server, obtain the target node in the cluster that has a normal network connection with the network file system server.

[0042] In some embodiments, each node in the cluster regularly self-checks or is checked by an external monitoring system for its health status to determine whether the network connection with the network file system server is abnormal, that is, to determine whether the NFS storage service can be normally provided through the mount point of each node. Specifically, if the NFS service of the current node receiving the data storage request is unavailable, which is an abnormal state, the node status data of each node in the cluster can be collected based on methods such as scheduled tasks, log analysis, and performance monitoring tools. Based on the collected node status data, analysis and judgment are performed to evaluate the node status of each node, and the target node with a normal node status is selected from other nodes. The target node can be a node with a normal network connection with the network file system server and capable of serving as the SFTP server for other nodes. Correspondingly, the selection of the target node can also be determined according to various policies, such as node scoring, load balancing, predefined priority lists, etc.

[0043] Step 103: Obtain the mount point of the target node through the Secure File Transfer Protocol.

[0044] In some embodiments, each node can be configured with a mount point, and the mount point can include the specific directory where the file system is mounted. Once the file system is mounted on a certain directory, all the contents of the file system can be accessed through this directory, which is convenient for building a global content library, thereby realizing the effective management and use of data.

[0045] Exemplarily, the current node that receives a data storage request can connect to a selected target node based on the SFTP protocol, and provide an NFS storage service through the mount point configured by the target node. That is, the current node can act as an SFTP client, and the target node can act as an SFTP server, and call the target node to complete the data storage operation based on the NFS protocol. In this way, when there are abnormal nodes in the cluster, the node that processes the data storage request can be switched based on the SFTP protocol, and a highly available global content library can be achieved through the redundant configuration of the NFS and SFTP dual protocols, ensuring the normal operation of the service to the greatest extent, and improving the reliability, availability, and operation and maintenance efficiency of the system.

[0046] Step 104: Process the data storage request based on the mount point.

[0047] In some embodiments, the current node that receives a data storage request indirectly obtains the mount point information of the target node through the SFTP protocol, and based on the directory corresponding to the mount point of the target node, directs to this path for data storage to complete the data storage service corresponding to this request.

[0048] Compared with the current existing technologies, in this embodiment, the current node can receive the data storage request sent by the user. When the network connection between the current node and the network file system server is abnormal, that is, when data storage cannot be performed through the network file system, a target node with a normal network connection to the network file system server can be selected from other nodes in the cluster, establish a connection with the target node based on the Secure File Transfer Protocol, process the data storage request through the mount point of the target node, and complete the data storage service corresponding to this request. The network file system protocol and the Secure File Transfer Protocol are configured for each node to provide a dual-protocol redundant storage service. Thus, when the network connection between a certain node in the cluster and the network file system server is abnormal, a data storage service can be provided through the Secure File Transfer Protocol, improving the reliability of the storage service.

[0049] Optionally, to further illustrate the specific implementation process of the method in this embodiment, before obtaining the target node in the cluster with a normal network connection to the network file system server, the method further includes: calling a timing detection task to detect the node status data corresponding to the nodes in the cluster.

[0050] Exemplarily, a timing detection task (timing task) can be created to continuously detect the node status data of each node. Among them, the node status data can include indicators such as heartbeat signals, the number of errors accessing NFS (error rate), the response speed of NFS, and the resource utilization rate of the node (CPU, memory, disk), etc., to evaluate the node status of each node from multiple dimensions and improve the accuracy of the evaluation results.

[0051] Optionally, step 102 may specifically include: determining the node status corresponding to a node according to the node status data of the nodes in the cluster; and determining the target node according to the node status corresponding to the node.

[0052] Exemplarily, the node status can be evaluated by using different node status data. The node status may include but is not limited to normal status (healthy status), sub-normal status, (sub-healthy status), abnormal status (excluded status), etc., so as to distinguish each node, facilitate the selection of the target node, and determine the available storage services of each node.

[0053] Optionally, determining the node status corresponding to a node according to the node status data of the nodes in the cluster may specifically include: obtaining the status score corresponding to the node status data by using the preset evaluation functions respectively corresponding to the node status data; determining the node score corresponding to the node according to the status score and the weight corresponding to the node status data; and determining the node status corresponding to the node based on the node score corresponding to the node.

[0054] In some embodiments, a monitoring tool can be used to obtain the node status data of each node, and the preset evaluation functions respectively corresponding to the node status data are used to calculate the status scores corresponding to the node status data of each node. Then, according to the influence degree of each node status data on the overall evaluation result, the weights corresponding to each node status data (evaluation indicators) are preset, and the node scores of the overall evaluation of each node are calculated in combination with the weights. Finally, according to the scores corresponding to the node scores, the node statuses of each node are distinguished, realizing the multi-dimensional evaluation of the node health condition, improving the professionalism of node evaluation, and facilitating the selection of the optimal node to process the data storage request sent by the user.

[0055] In some embodiments, the preset evaluation functions may include the evaluation functions respectively corresponding to each node status data. For example: heartbeat detection normalization function, error rate normalization function, resource utilization rate evaluation function, response time normalization function, etc. The specific descriptions of each function are as follows:

[0056] The heartbeat detection normalization function is as follows:

[0057] S1(t) = e -d / (T*Tt)

[0058] Correspondingly, considering the heartbeat interval T and the number of failures T1 within 30 seconds, and ensuring that the function value is zero after the heartbeat detection fails. In the formula, S1(t) can represent the index score corresponding to the heartbeat signal. If the heartbeat interval is larger, the number of failures is required to be smaller. The default is 1, that is, one failure is allowed for the detection once per second within 30s, otherwise a negative number is returned and the heartbeat detection fails.

[0059] The error rate normalization function is as follows:

[0060]

[0061] Correspondingly, if an error occurs when a node requests NFS, then when the next request comes, this node should be avoided as much as possible. In the formula, S2(t) can represent the metric score corresponding to the error rate, t can represent the number of failures, and T can represent the maximum tolerated number of errors. When the number of times increases, the return value of this function decreases rapidly.

[0062] The resource utilization rate evaluation function is as follows:

[0063]

[0064] Correspondingly, since the utilization rate has periodicity, it is suitable to use a cosine function for evaluation. In the formula, S3(t) represents the metric score corresponding to the resource utilization rate, u is the resource utilization rate, Y is the utilization rate threshold, with a default value of 0.8. When u is larger, the returned value is smaller. When the utilization rate reaches 60%, the score is 0.86; when it reaches 80%, the score is 0.5; when it reaches 90%, it is only 0.07, effectively warning of the risk of resource exhaustion. When the utilization rate of a certain node is too high, then this node is not suitable for using local NFS storage, which is likely to cause slow data acquisition and increased memory usage. The SFTP method can be used to give priority to ensuring the business.

[0065] The response time normalization function is as follows:

[0066]

[0067] Correspondingly, when the NFS interface return rate of a certain node becomes slower and slower, it is necessary to consider that there is a problem between this node and NFS, and it is necessary to switch to SFTP to obtain data from other nodes to give priority to ensuring the business. In the formula, S4(t) can represent the metric score corresponding to the response time, x can represent the response time, and T can represent the threshold, such as 300ms. When the response time increases, the return value of the function decreases.

[0068] In some embodiments, after determining the scores of multiple evaluation metrics, the multi-dimensional evaluation health check algorithm can be used to calculate the comprehensive health score of each node, so as to determine which nodes in the cluster are available when abnormal nodes need to be switched. Exemplarily, taking the four dimensions of heartbeat, error rate, response time, and utilization rate to evaluate the health status of the current node as an example, the four-dimensional evaluation health check algorithm can be expressed as:

[0069]

[0070] In the formula, H(x) is the node score obtained by evaluating based on the four evaluation metrics, Si(x) can represent the metric score corresponding to the i-th metric, K iIt can represent the weight value corresponding to the index score returned by the i-th dimension. Using this formula, the scores of each dimension can be weighted and added to obtain the multi-dimensional comprehensive score of the node. Specifically, a timing monitoring mechanism can be started. By monitoring the data of each node and through the health check mechanism of four-dimensional comprehensive measurement, the final H can be obtained, and then it can be determined whether the node uses NFS or SFTP.

[0071] In this way, not only can it be checked whether the NFS service of the node is available, but also a more comprehensive node health assessment can be provided based on multiple dimensions such as response time and heartbeat signal.

[0072] Optionally, based on the node score, the node status of the node is determined. Specifically, it can include: determining the node status of the node according to the preset evaluation threshold corresponding to different statuses and the node score.

[0073] Among them, the preset evaluation threshold can be the evaluation threshold for different node statuses. Exemplarily, if it is necessary to divide the node status into three levels, the first evaluation threshold and the second evaluation threshold can be preset. By comparing each node score with different preset evaluation thresholds respectively, the evaluation interval corresponding to the node score is determined, and then the node status is determined.

[0074] Optionally, according to the preset evaluation threshold corresponding to different statuses and the node score, the node status of the node is determined. Specifically, it can include: if the node score of the node is greater than or equal to the first evaluation threshold, the node status of the node is determined to be the normal state; if the node score of the node is less than the first evaluation threshold and greater than or equal to the second evaluation threshold, the node status of the node is determined to be the sub-normal state; if the node score of the node is less than the second evaluation threshold, the node status of the node is determined to be the abnormal state.

[0075] In some embodiments, to determine the status of each node, a preset evaluation model can be used. Based on the preset evaluation threshold, the node scores are divided into multiple levels to obtain multiple node statuses. When the current node does not meet the condition of using NFS to obtain files, it is necessary to degrade to use SFTP to ensure the operation of the service first. Exemplarily, three statuses of the node can be defined, such as: healthy status, sub-healthy status, and excluded status. The preset evaluation model (such as the three-state health model) used to define the three node statuses can be expressed as:

[0076]

[0077] Among them, H1 can be the first evaluation threshold corresponding to the normal state (healthy state), and H2 can be the second evaluation threshold corresponding to the degraded state (sub-healthy state). Based on these two evaluation thresholds, the node score H(x) can be divided into three intervals. Exemplarily, if the H(x) of a node is greater than or equal to H1, the node can be determined to be in the healthy state; if the H(x) of the node is greater than or equal to H2 and less than H1, the node can be determined to be in the sub-healthy state; if the H(x) of the node is less than H2, the node can be determined to be in the excluded state. Based on the state evaluation results of the three-state health model, it helps to switch between NFS storage and SFTP storage and adaptively adjust according to its own state.

[0078] Exemplarily, each node is divided into three states of normal state. When the nodes in the excluded state reach the health standard in three consecutive evaluations, they are set to monitoring. Similarly for the sub-healthy state. If the normal state is evaluated as sub-healthy or excluded once, it is immediately set to the excluded state to ensure the normal operation of the service as much as possible.

[0079] Optionally, according to the node state corresponding to the node, the target node is determined. Specifically, it may include: randomly selecting a target node from the nodes in the normal state.

[0080] In some embodiments, after the current node determines the node states of each node according to the node state data, if there are multiple nodes in the normal state, a node can be randomly selected from the nodes in the normal state as the target node, which serves as the SFTP server of the current node, provides a file storage path, and uses the mount point of the target node to process the data storage request received by the current node for data storage.

[0081] Optionally, the method of this embodiment may further include: determining the available storage service of the node according to the node state.

[0082] Exemplarily, when the node is in the healthy state, it can be determined that the node can both use the SFTP service and serve as an SFTP server for other nodes to call; when the node is in the sub-healthy state, it can be determined that the node can use the local NFS storage but not as an SFTP server in the excluded state; when the node is in the excluded state, it can be determined that the node cannot use the local NFS service and needs to call the NFS in the normal state through SFTP. Thus, according to the results of the node state check, the storage service strategy is dynamically adjusted, solving the problems encountered by the cloud platform when using the NFS storage scheme, significantly improving the reliability and flexibility of the global content library, and ensuring the continuity of the service. For example, only when the NFS is truly unavailable and cannot be quickly restored, it switches to SFTP or other backup schemes, rather than simply marking the entire node as unavailable, reducing resource waste.

[0083] In this way, the available storage services of each node can be distinguished according to the node status, and different storage services can be provided by nodes in different states, so as to provide highly available storage services and ensure the stability of the system.

[0084] Optionally, based on the node score, the node status of the node is determined, and specifically, it further includes: updating the node status corresponding to the node according to the node score of the node within a preset detection period.

[0085] Optionally, a preset recovery rule can be used to recover the node status of the node. The preset recovery rule includes that if the node score of the node is greater than or equal to the first evaluation threshold for multiple consecutive times, the node status of the node is restored to the normal state; if the node score of the node is greater than or equal to the second evaluation threshold for multiple consecutive times and less than the first evaluation threshold, the node status of the node is restored to the sub-normal state.

[0086] In some embodiments, an adaptive recovery mechanism can be provided. Nodes in the excluded state can also be restored to the sub-healthy or normal state. To ensure its availability, a preset recovery model can be used to restore the node status. The preset recovery model can be expressed as:

[0087]

[0088] Among them, recovery can represent the state of node recovery. The preset recovery rule can be: if the node score that meets the preset score condition of the target state is detected multiple consecutive times (such as three times), the node status can be restored to the target state. Exemplarily, if it is detected that the node score of a certain node is greater than or equal to H1 for 3 consecutive times, the node status of the node can be restored to the healthy state; if it is detected that the node score of a certain node is greater than or equal to H2 for 3 consecutive times and less than H1, the node status of the node can be restored to the sub-healthy state. When a fault is detected, a predefined recovery process is triggered immediately, such as automatically mounting a standby storage or redirecting traffic to other healthy nodes, thereby reducing the need for manual intervention, improving the response speed, providing a more stable and efficient service experience for users, reducing resource waste at the same time, and improving the overall operation efficiency.

[0089] In this way, the node score can be updated in real time according to the node evaluation metrics detected in real time, and then based on the detection result of the node score, the node status of the node is updated, a status recovery mechanism is provided, and the automatic update of the node status is realized.

[0090] As a possible implementation method, such as Figure 3As shown, a highly available global content library solution based on dual - protocol redundancy is provided. A single storage has the characteristics of fast transmission rate and high efficiency. However, in order to achieve high availability and ensure the normal operation of the business as much as possible, the SFTP method is added to provide services for nodes with abnormal NFS. Based on NFS, SFTP is configured at each mount point to form a heterogeneous redundant layer. If the current node is unavailable, the storage file is automatically operated by requesting the SFTP of other available nodes to ensure the normal operation of the business. The specific implementation steps are as follows:

[0091] 1). First, create a mount point on the cloud platform node and mount NFS to be able to provide NFS services normally.

[0092] 2). Deploy SFTP on each node of the cloud platform and expose the mount point through SFTP.

[0093] 3). Create a scheduled task to continuously detect the heartbeat of each node, the number of errors accessing NFS, the response speed of NFS, and the resource utilization rate of the node.

[0094] 4). Calculate the comprehensive health score through the heartbeat detection normalization function, error rate normalization function, response speed normalization function, and node resource normalization function.

[0095] 5). Determine the NFS health status of each node through the three - state health model. If it is in a normal state, use NFS storage normally and can provide SFTP services externally. If it is in a sub - normal state, NFS storage can be used but SFTP services cannot be provided externally. If it is in an excluded state, NFS storage cannot be used and only the SFTP services of other nodes can be accessed.

[0096] 6). Through the adaptive mechanism, if a node in the excluded state reaches the normal state three times in a row, it is set to healthy, and the same applies to the sub - healthy state. However, if a node in the normal state is sub - healthy or excluded once, it is set to sub - healthy or excluded. Maximize the normal operation of the business.

[0097] Specifically, such as Figure 4As shown in the figure, a monitoring module framework diagram is presented. SFTP storage is added to form a dual - protocol redundant storage solution. A four - dimensional health assessment algorithm is designed to evaluate the health status of the NFS of this node through four dimensions: heartbeat survival, error rate, response time, and utilization rate, significantly improving the accuracy of fault determination and reducing the situations of frequent switching or switching failures. A three - state health model, namely the normal state, sub - normal state, and excluded state, is designed to determine the switch from NFS to SFTP through these three states, enabling the shortest - time switch in case of failures to ensure the normal operation of services. The adaptive recovery mechanism can reduce the requests of high - load NFS and make it return to normal. Based on the global content library of the dual - protocol redundant scheme of NFS and SFTP, the four - dimensional health determination algorithm and its formula, the three - state health model and their respective switching strategies and adaptive strategies, a complete cloud storage high - availability solution is formed, significantly improving the reliability, availability, and operation and maintenance efficiency of the system, with prominent substantial features and remarkable progress.

[0098] In a specific application scenario, this embodiment adopts the dual - protocol redundant method to provide storage services through NFS and SFTP simultaneously. NFS provides normal storage services, while SFTP ensures the high availability of storage. A multi - dimensional health assessment method is adopted to monitor multiple dimensions such as heartbeat, error rate, NFS response time, and node utilization rate, and the final node score is obtained through a weighted method to improve the accuracy of node status assessment. According to the results of node status checks, the storage service strategy is dynamically adjusted. When problems occur when the cloud platform uses the NFS storage solution, storage services are provided through SFTP, significantly improving the reliability and flexibility of the global content library, ensuring the continuity of business processing, providing a more stable and efficient service experience for users, reducing resource waste, and improving the overall operation efficiency.

[0099] Compared with the current existing technologies, this embodiment adopts the dual - protocol redundant method to provide storage services through NFS and SFTP simultaneously. NFS provides normal storage services, while SFTP ensures the high availability of storage. A multi - dimensional health assessment method is adopted to monitor multiple dimensions such as heartbeat, error rate, NFS response time, and node utilization rate, and the final node score is obtained through a weighted method to improve the accuracy of node status assessment. According to the results of node status checks, the storage service strategy is dynamically adjusted. When problems occur when the cloud platform uses the NFS storage solution, storage services are provided through SFTP, significantly improving the reliability and flexibility of the global content library, ensuring the continuity of business processing, providing a more stable and efficient service experience for users, reducing resource waste, and improving the overall operation efficiency.

[0100] An embodiment of the present application also provides a data storage device as Figure 1 the specific implementation of the method shown inFigure 5 As shown in the figure, the device includes: a receiving module 31, an obtaining module 32, and a processing module 33.

[0101] The receiving module 31 is configured to receive a data storage request from a user.

[0102] The obtaining module 32 is configured to obtain a target node in the cluster that has a normal network connection with the network file system server when the network connection with the network file system server is abnormal.

[0103] The obtaining module 32 is configured to obtain the mount point of the target node through a secure file transfer protocol.

[0104] The processing module 33 is configured to process the data storage request based on the mount point.

[0105] In some examples of this embodiment, the obtaining module 32 is specifically configured to determine the node status corresponding to a node based on the node status data of the nodes in the cluster; and determine the target node according to the node status corresponding to the node.

[0106] In some examples of this embodiment, the obtaining module 32 is specifically configured to use the preset evaluation functions corresponding to the node status data respectively to obtain the status score corresponding to the node status data; determine the node score corresponding to the node according to the status score and the weight corresponding to the node status data; and determine the node status corresponding to the node based on the node score corresponding to the node.

[0107] In some examples of this embodiment, the obtaining module 32 is specifically configured to determine the node status corresponding to the node according to the preset evaluation thresholds corresponding to different states and the node score.

[0108] In some examples of this embodiment, the obtaining module 32 is specifically configured to, if the node score of a node is greater than or equal to the first evaluation threshold, determine the node status of the node as the normal state; and determine the target node according to the node status corresponding to the node, including: randomly selecting a target node from the nodes in the normal state.

[0109] In some examples of this embodiment, the obtaining module 32 is further specifically configured to update the node status corresponding to the node according to the node score of the node within a preset detection period.

[0110] In some examples of this embodiment, the obtaining module 32 is specifically configured to call a timing detection task to detect the node status data corresponding to the nodes in the cluster.

[0111] It should be noted that for other corresponding descriptions of the various functional units involved in the data storage device provided in this embodiment, reference can be made to the corresponding descriptions in Figure 1 and details are not described herein again.

[0112] Based on the method as described above Figure 1 shown, correspondingly, this embodiment also provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, it implements the method as described above Figure 1 shown.

[0113] Based on the method as described above Figure 1 shown, correspondingly, this embodiment also provides a computer program product, on which a computer program is stored, and when the computer program is executed by a processor, it implements the method as described above Figure 1 shown.

[0114] Based on such an understanding, the technical solution of this application can be embodied in the form of a software product, and this software product can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.), and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods of various implementation scenarios of this application.

[0115] Based on the method as described above Figure 1 shown, and Figure 5 the virtual device embodiment shown, in order to achieve the above purpose, this embodiment of this application also provides an electronic device, such as a personal computer, a server, and this device includes a storage medium and a processor; the storage medium is used to store a computer program; the processor is used to execute the computer program to implement the method as described above Figure 1 shown.

[0116] In some embodiments, the above-mentioned physical device may further include a user interface, a network interface, a camera, a radio frequency (RF) circuit, sensors, an audio circuit, a WI-FI module, and so on. The user interface may include a display screen (Display), an input unit such as a keyboard (Keyboard), etc., and optionally the user interface may further include a USB interface, a card reader interface, etc. The network interface may include a standard wired interface, a wireless interface (such as a WI-FI interface), etc. in some embodiments.

[0117] Those skilled in the art can understand that the above-mentioned physical device structure provided in this embodiment does not constitute a limitation on the physical device, and it may include more or fewer components, or combine some components, or have different component arrangements.

[0118] The storage medium may further include an operating system and a network communication module. The operating system is a program for managing the hardware and software resources of the above-mentioned physical devices, and supports the operation of information processing programs and other software and / or programs. The network communication module is used to implement communication between components inside the storage medium, as well as communication between other hardware and software in the information processing physical device.

[0119] Through the description of the above embodiments, those skilled in the art can clearly understand that the present application can be implemented by means of software plus a necessary general hardware platform, or can also be implemented by hardware. By applying the solution of this embodiment, compared with the current existing technologies, this embodiment adopts the dual-protocol redundancy method, provides storage services simultaneously through NFS and SFTP, NFS provides normal storage services, while SFTP ensures the high availability of storage, and adopts a multi-dimensional health assessment method. By monitoring multiple dimensions such as heartbeat, error rate, NFS response time, node utilization rate, etc., and obtaining the final node score in a weighted manner to improve the accuracy of node status assessment. According to the results of node status checks, dynamically adjust the storage service strategy. When the cloud platform encounters problems when using the NFS storage solution, provide storage services through SFTP, significantly improve the reliability and flexibility of the global content library, ensure the continuity of business processing, and at the same time provide users with a more stable and efficient service experience, reduce resource waste, and improve the overall operation efficiency.

[0120] It should be noted that in this article, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the term "comprising", "including" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements not only includes those elements, but also includes other elements not expressly listed, or also includes elements inherent to such process, method, article or device. Without further limitation, an element defined by the statement "including a..." does not exclude the existence of additional identical elements in the process, method, article or device including the element.

[0121] The above are only specific embodiments of the present application, enabling those skilled in the art to understand or implement the present application. Various modifications to these embodiments will be obvious to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to these embodiments herein, but will be in accordance with the widest scope consistent with the principles and novel features claimed herein.

Claims

1. A data storage method, characterized in that, Including: Receiving a data storage request from a user; When the network connection with the network file system server is abnormal, obtaining a target node in the cluster that has a normal network connection with the network file system server; Obtaining the mount point of the target node through a secure file transfer protocol; Processing the data storage request based on the mount point.

2. The method according to claim 1, characterized in that, The obtaining of the target node in the cluster that has a normal network connection with the network file system server includes: Determining the node status corresponding to the node according to the node status data of the nodes in the cluster; Determining the target node according to the node status corresponding to the node.

3. The method according to claim 2, wherein The determining of the node status corresponding to the node according to the node status data of the nodes in the cluster includes: Using the preset evaluation functions corresponding to the node status data respectively to obtain the status scores corresponding to the node status data; Determining the node score corresponding to the node according to the status score and the weight corresponding to the node status data; Determining the node status corresponding to the node based on the node score corresponding to the node.

4. The method according to claim 3, characterized in that, The determining of the node status corresponding to the node based on the node score corresponding to the node includes: Determining the node status corresponding to the node according to the preset evaluation thresholds corresponding to different statuses and the node score.

5. The method according to claim 4, wherein The determining of the node status corresponding to the node according to the preset evaluation thresholds corresponding to different statuses and the node score includes: If the node score of the node is greater than or equal to the first evaluation threshold, determining the node status of the node as the normal status; The determining of the target node according to the node status corresponding to the node includes: Randomly selecting the target node from the nodes in the normal status.

6. The method according to claim 3, wherein The method further includes: Updating the node status corresponding to the node according to the node score of the node within a preset detection period.

7. The method according to claim 2, wherein Before the obtaining of the target node in the cluster that has a normal network connection with the network file system server, the method further includes: Invoking a timing detection task to detect the node status data corresponding to the nodes in the cluster.

8. A data storage device, characterized in that, Including: A receiving module configured to receive a data storage request from a user; An obtaining module configured to obtain a target node in the cluster that has a normal network connection with the network file system server when the network connection with the network file system server is abnormal; An obtaining module configured to obtain the mount point of the target node through a secure file transfer protocol; A processing module configured to process the data storage request based on the mount point.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method according to any one of claims 1 to 7.

10. An electronic device, comprising a storage medium, a processor, and a computer program stored on the storage medium and executable on the processor, characterized in that, When the processor executes the computer program, it implements the method according to any one of claims 1 to 7.

Images