Data backup method, system, device and equipment based on dynamic strategy and medium

Abstract

This disclosure provides a data backup method, system, apparatus, device, and medium based on dynamic strategies, relating to the field of data storage technology. The method includes: receiving a file upload request from a client, wherein the file upload request is used to request the upload of one or more files to be backed up; determining whether the file to be backed up is a file being uploaded for the first time; if the file to be backed up is a file being uploaded for the first time, determining the file priority of the file to be backed up based on the entity relationship triplet of the file to be backed up; if the file to be backed up is not a file being uploaded for the first time, determining the file priority of the file to be backed up based on the number of times the file to be backed up is accessed; and backing up the file to be backed up according to the file priority. This disclosure determines the file priority based on the file information of the file to be backed up, and then backs up the file to be backed up according to the file priority, which can reduce the amount of data backed up in the cloud server and reduce the service cost of cloud storage.

Description

Technical Field

[0001] This disclosure relates to the field of data storage technology, and in particular to a data backup method, system, apparatus, device, and medium based on dynamic strategies. Background Technology

[0002] With the rapid development of internet technology, more and more cloud computing services have emerged, such as Infrastructure as a Service (IaaS) and Platform as a Service (PaaS). ，PaaS Software as a Service (SaaS) is another cloud storage service. Cloud storage services offer greater flexibility in data management and access, and can efficiently expand databases. Therefore, various cloud computing services store data on cloud servers. However, storing data on cloud servers is typically more expensive than storing data locally. How to reduce the cost of backup storage of redundant data on cloud servers and lower the cost of cloud storage services is a pressing technical problem that needs to be solved.

[0003] It should be noted that the information disclosed in the background section above is only used to enhance the understanding of the background of this disclosure, and therefore may include information that does not constitute prior art known to those skilled in the art. Summary of the Invention

[0004] This disclosure provides a data backup method, system, device, equipment, and medium based on dynamic strategies, which at least to some extent overcomes the technical problem of backing up and storing large amounts of redundant data on cloud servers in related technologies.

[0005] Other features and advantages of this disclosure will become apparent from the following detailed description, or may be learned in part from practice of this disclosure.

[0006] According to one aspect of this disclosure, a data backup method based on a dynamic strategy is provided. The method includes: receiving a file upload request from a client, wherein the file upload request is for requesting the upload of one or more files to be backed up; determining whether the file to be backed up is a file being uploaded for the first time; if the file to be backed up is a file being uploaded for the first time, determining the file priority of the file to be backed up based on the entity relation triplet of the file to be backed up; if the file to be backed up is not a file being uploaded for the first time, determining the file priority of the file to be backed up based on the number of times the file to be backed up is accessed; and backing up the file to be backed up according to the file priority of the file to be backed up.

[0007] In some embodiments, backing up the file to be backed up according to its file priority includes: determining one or more servers to store the file to be backed up according to its file priority; and storing the file to be backed up on one or more servers.

[0008] In some embodiments, determining the file priority of the file to be backed up based on the entity relation triples of the file to be backed up includes: obtaining the file content of the file to be backed up; extracting entity relation triples from the file content of the file to be backed up; obtaining the number of times the entity relation triples appear in a preset corpus; and determining the file priority of the file to be backed up based on the number of times the entity relation triples appear in the preset corpus.

[0009] In some embodiments, determining whether the file to be backed up is a file uploaded for the first time includes: generating an MD5 value for the file to be backed up using an MD5 message digest algorithm; and determining whether the file to be backed up is a file uploaded for the first time based on the MD5 value of the file to be backed up.

[0010] In some embodiments, after backing up the files to be backed up according to their file priorities, the method further includes: receiving a file deletion request from a client, wherein the file deletion request is used to request the deletion of one or more files; determining whether the files to be deleted exist; and deleting the files to be deleted if they exist.

[0011] In some embodiments, after backing up the file to be backed up according to its file priority, the method further includes: receiving a file access request from a client, wherein the file access request is used to request access to one or more files; determining whether the file to be accessed exists; if the file to be accessed exists, returning the corresponding file to the client and recording the access count of the corresponding file.

[0012] According to another aspect of this disclosure, a data backup system based on a dynamic strategy is also provided, comprising: a client, a master server, and at least one slave server; the master server includes: a priority calculation module, a data management module, and a data backup module; wherein, the client is used to send a file upload request to the master server, wherein the file upload request is used to request the upload of one or more files to be backed up; the data management module is used to receive the file upload request from the client and determine whether the file to be backed up requested by the file upload request is a file uploaded for the first time; the priority calculation module is used to: if the file to be backed up is a file uploaded for the first time, determine the file priority of the file to be backed up based on the entity relationship triplet of the file to be backed up; if the file to be backed up is not a file uploaded for the first time, determine the file priority of the file to be backed up based on the number of times the file to be backed up is accessed; the data backup module is used to determine whether to back up the file to be backed up on the master server or the slave server based on the file priority of the file to be backed up.

[0013] According to another aspect of this disclosure, a data backup device based on a dynamic strategy is also provided. The device includes: a file receiving module for receiving files to be backed up uploaded by a client; a file identification module for determining whether the file to be backed up is a file uploaded for the first time; a first priority calculation module for determining the file priority of the file to be backed up based on the entity relationship triplet of the file to be backed up if the file to be backed up is a file uploaded for the first time; a second priority calculation module for determining the file priority of the file to be backed up based on the number of times the file to be backed up is accessed if the file to be backed up is not a file uploaded for the first time; and a file backup module for backing up the file to be backed up according to the file priority of the file to be backed up.

[0014] According to another aspect of this disclosure, an electronic device is also provided, comprising: a processor; and a memory for storing executable instructions of the processor; wherein the processor is configured to execute the data backup method based on dynamic strategy described above by executing the executable instructions.

[0015] According to another aspect of this disclosure, a computer-readable storage medium is also provided, on which a computer program is stored, which, when executed by a processor, implements the data backup method based on dynamic strategy described in any one of the preceding claims.

[0016] According to another aspect of this disclosure, a computer program product is also provided, including a computer program that, when executed by a processor, implements the data backup method based on a dynamic strategy as described above.

[0017] The data backup method, system, apparatus, device, and medium based on dynamic strategies provided in the embodiments of this disclosure, upon receiving a file upload request from a client requesting the upload of one or more files to be backed up, determines whether the files to be backed up are being uploaded for the first time. If the files are being uploaded for the first time, the file priority of the files to be backed up is determined based on the entity relationship triplet of the files to be backed up. If the files are not being uploaded for the first time, the file priority of the files to be backed up is determined based on the number of times the files are accessed. Finally, the files to be backed up are backed up according to their file priority. By determining the file priority based on the file information of the files to be backed up, and then backing up the files according to their file priority, the amount of data to be backed up in the cloud server can be reduced, thereby lowering the service cost of cloud storage.

[0018] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0019] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure. It is obvious that the drawings described below are merely some embodiments of this disclosure, and those skilled in the art can obtain other drawings based on these drawings without any inventive effort.

[0020] Figure 1 This diagram illustrates an application system architecture according to an embodiment of the present disclosure.

[0021] Figure 2 This diagram illustrates a data backup method based on a dynamic strategy according to an embodiment of the present disclosure.

[0022] Figure 3 This diagram illustrates a data backup system based on a dynamic strategy according to an embodiment of the present disclosure.

[0023] Figure 4 This diagram illustrates a specific implementation flowchart of a data backup method based on a dynamic strategy in an embodiment of this disclosure.

[0024] Figure 5 This diagram illustrates the implementation architecture of a joint named entity recognition and relation extraction method according to an embodiment of the present disclosure.

[0025] Figure 6 This diagram illustrates a data structure according to an embodiment of the present disclosure.

[0026] Figure 7 This diagram illustrates a file upload process based on a dynamic strategy in an embodiment of this disclosure.

[0027] Figure 8 This diagram illustrates a file deletion process based on a dynamic strategy, as shown in an embodiment of the present disclosure.

[0028] Figure 9 This diagram illustrates a file access flowchart based on a dynamic policy in an embodiment of this disclosure.

[0029] Figure 10 This diagram illustrates a data backup device based on a dynamic strategy according to an embodiment of the present disclosure.

[0030] Figure 11 This diagram illustrates a structural block diagram of an electronic device according to an embodiment of the present disclosure;

[0031] Figure 12 A schematic diagram of a computer-readable storage medium is shown in an embodiment of the present disclosure. Detailed Implementation

[0032] Exemplary embodiments will now be described more fully with reference to the accompanying drawings. However, these exemplary embodiments can be implemented in many forms and should not be construed as limited to the examples set forth herein; rather, they are provided so that this disclosure will be more comprehensive and complete, and will fully convey the concept of the exemplary embodiments to those skilled in the art. The described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.

[0033] Furthermore, the accompanying drawings are merely illustrative of this disclosure and are not necessarily drawn to scale. The same reference numerals in the drawings denote the same or similar parts, and therefore repeated descriptions of them will be omitted. Some block diagrams shown in the drawings are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities may be implemented in software, in one or more hardware modules or integrated circuits, or in different network and / or processor devices and / or microcontroller devices.

[0034] To facilitate understanding, before introducing the embodiments of this disclosure, the following explanations are provided for several terms involved in the embodiments of this disclosure:

[0035] The specific implementation methods of the embodiments of this disclosure will now be described in detail with reference to the accompanying drawings.

[0036] Figure 1 A schematic diagram of an exemplary application system architecture is shown, illustrating an application of the dynamic strategy-based data backup method described in this disclosure. For example... Figure 1 As shown, the system architecture may include: cloud server 10 and client 20.

[0037] It should be noted that the medium providing the communication link between the cloud server 10 and the client 20 can be a wired network or a wireless network.

[0038] Optionally, the aforementioned wireless or wired networks use standard communication technologies and / or protocols. The network is typically the Internet, but can also be any network, including but not limited to Local Area Networks (LANs), Metropolitan Area Networks (MANs), Wide Area Networks (WANs), mobile, wired or wireless networks, private networks, or any combination of virtual private networks. In some embodiments, technologies and / or formats including Hyper Text Markup Language (HTML), Extensible Markup Language (XML), etc., are used to represent data exchanged over the network. Furthermore, conventional encryption technologies such as Secure Socket Layer (SSL), Transport Layer Security (TLS), Virtual Private Networks (VPNs), and Internet Protocol Security (IPsec) can be used to encrypt all or some links. In other embodiments, custom and / or dedicated data communication technologies can be used to replace or supplement the aforementioned data communication technologies.

[0039] Client 20 can be various electronic devices, including but not limited to smartphones, tablets, laptops, desktop computers, smart speakers, smartwatches, wearable devices, augmented reality devices, virtual reality devices, etc.

[0040] Optionally, the clients of the applications installed on different clients 20 may be the same, or clients of the same type of application based on different operating systems. Depending on the terminal platform, the specific form of the application client may also differ; for example, the application client may be a mobile client, a PC client, etc.

[0041] The cloud server 10 can be a server that provides various cloud services. Optionally, the server can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers.

[0042] In some embodiments, the cloud server 10 provided in this disclosure includes multiple servers and adopts a master-slave server architecture.

[0043] Those skilled in the art will know that Figure 1 The number of clients and cloud servers shown is merely illustrative; any number of clients and cloud servers can be used depending on actual needs. This disclosure does not limit this.

[0044] Under the above system architecture, this disclosure provides a data backup method based on dynamic strategies, which can be executed by any electronic device with computing power.

[0045] In some embodiments, the data backup method based on dynamic policies provided in this disclosure can be executed by the cloud server of the above-described system architecture; in other embodiments, the data backup method based on dynamic policies provided in this disclosure can be implemented by the cloud server in the above-described system architecture and the client through interaction; in still other embodiments, the data backup method based on dynamic policies provided in this disclosure can be executed by a third-party device that communicates with the cloud server in the above-described system architecture.

[0046] Figure 2 This invention discloses a flowchart of a data backup method based on a dynamic strategy, as shown in an embodiment of the present disclosure. Figure 2 As shown in the embodiments of this disclosure, the data backup method based on dynamic strategies may include the following steps:

[0047] S202, Receive a file upload request from the client, wherein the file upload request is used to request the upload of one or more files to be backed up.

[0048] It should be noted that the aforementioned client can be any terminal device that communicates with the cloud server, including but not limited to mobile phones, laptops, and computers; the aforementioned file upload request can be a request sent by the user through the client to upload one or more files to be backed up to the cloud server.

[0049] S204, determine whether the file to be backed up is the first file uploaded.

[0050] In practice, it can be determined whether the file to be backed up is the first file uploaded based on the identification information of the file to be backed up; in one embodiment, the identification information of the file to be backed up can be, but is not limited to, the MD5 value.

[0051] In some embodiments, determining whether the file to be backed up is a file uploaded for the first time includes: generating an MD5 value for the file to be backed up using an MD5 message digest algorithm; and determining whether the file to be backed up is a file uploaded for the first time based on the MD5 value of the file to be backed up.

[0052] S206. If the file to be backed up is the first file uploaded, then the file priority of the file to be backed up is determined based on the entity relationship triplet of the file to be backed up.

[0053] It should be noted that the entity relation triples in this embodiment can be entity relation triple data (including entity, relation, entity) extracted from the file content of the file to be backed up using the joint named entity recognition and relation extraction method.

[0054] In some embodiments, the above-mentioned S206 can be implemented by the following steps: obtaining the file content of the file to be backed up; extracting entity relation triples from the file content of the file to be backed up; obtaining the number of times the entity relation triples appear in a preset corpus; and determining the file priority of the file to be backed up based on the number of times the entity relation triples appear in the preset corpus.

[0055] In practice, multiple threshold ranges can be set to determine the priority of multiple files based on the number of times the entity relation triples appear in the preset corpus.

[0056] S208. If the file to be backed up is not the first file uploaded, the file priority of the file to be backed up is determined based on the number of times the file has been accessed.

[0057] In practice, multiple threshold ranges can be set to determine the priority of multiple files based on the number of times the files to be backed up are accessed.

[0058] S210 backs up the files to be backed up according to their file priority.

[0059] In some embodiments, backing up a file to be backed up according to its file priority includes: determining one or more servers to store the file to be backed up according to its file priority; and storing the file to be backed up on one or more servers.

[0060] In some embodiments, after backing up the files to be backed up according to their file priorities, the data backup method based on dynamic policies provided in this disclosure may further include the following steps: receiving a file deletion request from a client, wherein the file deletion request is used to request the deletion of one or more files; determining whether the file to be deleted exists; and deleting the file to be deleted if it exists.

[0061] In some embodiments, after backing up the files to be backed up according to their file priorities, the data backup method based on dynamic strategies provided in this disclosure may further include the following steps: receiving a file access request from a client, wherein the file access request is used to request access to one or more files; determining whether the file to be accessed exists; if the file to be accessed exists, returning the corresponding file to the client and recording the number of times the corresponding file is accessed.

[0062] Based on the same inventive concept, this disclosure also provides a data backup system based on a dynamic strategy, as described in the following embodiments. Since the principle by which this system embodiment solves the problem is similar to that of the above method embodiments, the implementation of this system embodiment can refer to the implementation of the above method embodiments, and repeated details will not be elaborated further.

[0063] Figure 3 This illustration shows a schematic diagram of a data backup system based on a dynamic strategy according to an embodiment of the present disclosure, such as... Figure 3 As shown, the system includes: client 20, master server 10A, and at least one slave server 10B. Figure 3 The image shows three slave servers.

[0064] In some embodiments, the master server 10 may include a priority calculation module, a data management module, and a data backup module.

[0065] Furthermore, in some embodiments, the data management module may include a maintenance submodule and a management submodule.

[0066] The process involves the client sending a file upload request to the master server to request the upload of one or more files to be backed up. The data management module receives the file upload request from the client and determines whether the file to be backed up is being uploaded for the first time. If the file is being uploaded for the first time, the priority calculation module determines the file priority based on the entity relationship triplet of the file. If the file is not being uploaded for the first time, the priority calculation module determines the file priority based on the number of times the file has been accessed. Finally, the data backup module determines whether to back up the file on the master server or the slave server based on the file priority.

[0067] In some embodiments, the master server 10A is further configured to: generate the MD5 value of the file to be backed up using the MD5 message digest algorithm; and determine whether the file to be backed up is the first file uploaded based on the MD5 value of the file to be backed up.

[0068] In some embodiments, the master server 10A is further configured to: receive a file deletion request from a client, wherein the file deletion request is for requesting the deletion of one or more files; determine whether the file to be deleted exists; and if the file to be deleted exists, delete the file to be deleted.

[0069] In some embodiments, the master server 10A is further configured to: receive a file access request from a client, wherein the file access request is used to request access to one or more files; determine whether the file to be accessed exists; if the file to be accessed exists, return the corresponding file to the client and record the access count of the corresponding file.

[0070] The functions of each part of the above system are as follows:

[0071] Client: The client mainly provides users with the function of inputting requests. It is generally divided into three types: user upload file, user delete file, and user access file. After the user inputs a request, the client establishes a connection with the main server and sends the corresponding request to the main server.

[0072] Master server: The master server mainly receives user requests and executes the corresponding processes (file upload process, file deletion process, file access process) according to the user requests.

[0073] Slave Server: The slave server mainly receives requests from the master server and performs corresponding operations (file backup, file deletion) according to the instructions.

[0074] The data backup module mainly backs up or deletes corresponding files on the master and slave servers based on the parameters passed by the data management module.

[0075] When the data management module sends a file backup request, the data backup module determines the number of files that need to be backed up based on the file's priority. Based on the number of backups, it selects the server closest to the user from the slave servers for backup and generates the corresponding file storage address, which is then returned to the data management module.

[0076] When the data management module sends a file deletion request, the data backup module performs the file deletion operation on the master and slave servers according to the file's storage address.

[0077] The system provided in this embodiment uses multiple strategies to dynamically reduce data backups in cloud servers, thereby reducing enterprise storage costs.

[0078] Preferably, the strategy selected by the user includes an access count priority strategy, which means setting the priority of a file by comparing the current access count of the file with the access count threshold specified by the user during system initialization.

[0079] Specifically, the access count priority strategy sets file priority by comparing the current access count of a file with the access count threshold K (which has three values: K1, K2, and K3) specified by the user during system initialization. The specific steps are as follows:

[0080] 1) Receive the file access count parameter from the management submodule.

[0081] 2) Determine if the number of visits (Count) is less than K1. If the condition is met, set the priority to 0 and end the priority strategy based on the number of visits.

[0082] 3) Determine if the number of visits (Count) is greater than K1 and less than K2. If the condition is met, set the priority to 1 and end the priority strategy based on the number of visits.

[0083] 4) Determine if the number of visits (Count) is greater than K2 and less than K3. If the condition is met, set the priority to 2 and end the priority strategy based on the number of visits.

[0084] 5) If the count is greater than the threshold K3, and the condition is met, the priority is set to 3, and the access count priority strategy is terminated.

[0085] Preferably, the strategy selected by the user includes an entity relation number priority strategy. The entity relation number priority strategy refers to extracting entity relation triples from the file content using a predetermined method, and setting the file priority by comparing the number of times the entity relation triples appear in the user-defined entity relation triple corpus with the entity relation number threshold specified by the user during system initialization.

[0086] Specifically, the entity relation count priority strategy refers to using a joint named entity recognition and relation extraction method to extract entity relation triples from the file content, and then setting the file priority by comparing the number of times the entity relation triples appear in the user-defined entity relation triple (entity, relation, entity) corpus Corpus with the entity relation count threshold E (three values ​​E1, E2, and E3) specified by the user during system initialization. The specific steps are as follows:

[0087] 1) When it is determined that the file is being uploaded for the first time, the combined named entity recognition and relation extraction method will be used to extract all triples from the file content, forming a set of entity-relation triples T = {T1, T2, ..., T...}. n}

[0088] 2) Match the set of triples T with the user-defined corpus Corpus in turn. If it is in the corpus, increment the corresponding entity relation number Entity by 1, and finally get the size of Entity.

[0089] 3) Determine if the number of entity relationships (Entity) is less than E1. If the condition is met, set the priority to 0 and end the entity relationship priority strategy.

[0090] 4) Determine if the number of entity relationships (Entity) is greater than E1 and less than E2. If the condition is met, set the priority to 1 and end the entity relationship priority strategy.

[0091] 5) Determine if the number of entity relationships (Entity) is greater than E2 and less than E3. If the condition is met, set the priority to 2 and end the entity relationship priority strategy.

[0092] 6) If the number of entity relationships (Entity) is greater than the threshold E3, and the condition is met, the priority is set to 3, and the entity relationship priority strategy ends.

[0093] Preferably, the priority calculation module is used for:

[0094] When a file is uploaded for the first time, the entity relation count priority strategy is applied; when a file is not uploaded for the first time, the access count priority strategy is applied. The priority calculation module calculates the file priority based on the above two strategies and dynamically adjusts the file priority during user access.

[0095] Specifically, the main function of the priority calculation module is to calculate the priority of each file according to the strategy selected by the user, as follows: Figure 4 As shown, this module employs two strategies: access count priority and entity relationship count priority. When a file is uploaded for the first time, the entity relationship count priority strategy is applied; otherwise, the access count priority strategy is applied. Based on these two strategies, file priority can be calculated and dynamically adjusted during user access, achieving low-cost file storage.

[0096] Preferably, the predetermined method is a joint named entity recognition and relation extraction method, and the specific steps of the joint named entity recognition and relation extraction method include:

[0097] Read the file content and use text preprocessing methods to convert each sentence in the file content into a sentence composed of words;

[0098] The sentence is fed into the encoding layer, which outputs word vectors. The encoding layer consists of a pre-trained BERT model and a pooling layer.

[0099] The output word vector of the encoding layer is input into the named entity recognition decoding layer to obtain the output word vector of the named entity recognition decoding layer. The named entity recognition decoding layer consists of GRU, Softmax, and pooling layers.

[0100] The output word vectors of the entity recognition decoding layer are input into the relation extraction decoding layer to obtain the final entity relation triples;

[0101] Concatenate every two entities and their corresponding relations into a triple, and count and output the set of all entity-relation triples for each file.

[0102] Specifically, the joint named entity recognition and relation extraction method is the core of the entity relation number-first strategy, and its implementation architecture is as follows: Figure 5 As shown, it specifically includes:

[0103] 1) Read the file content and use text preprocessing methods (word segmentation, stop word removal, case conversion, etc.) to convert each sentence in the file content into a sentence composed of words: sentence = {w1, w2, ..., w...} i}, where i = 1, 2, 3, ..., n.

[0104] 2) The sentence is fed into the encoding layer to obtain the embedding vector E = (e1,…,e i-1 ,e i ), i = 1, 2, ..., n. The encoding layer consists of a pre-trained BERT model and pooling layers, as shown below.

[0105] ①The sentence obtained by the pre-trained BERT model is (c,t1,t2,…,t). i ), where c represents the context embedding vector of the entire sentence, t i It is the embedding vector corresponding to the i-th word in the sentence.

[0106] ② For the i-th word, its embedding vector is obtained by pooling it together with the embedding vectors of the following k words. i The formula is shown below.

[0107] e i ∶=pool(t i ,t i+1 ,…,t i+k-1 (1)

[0108] ③ Output the word vector E = (e1,…,e2) for each sentence. i-1 ,ei ), i = 1, 2, ..., n are the outputs of the coding layer.

[0109] 3) Obtain the output word vector E = (e1,…,e1) of the encoding layer. i-1 ,e i After i = 1, 2, ..., n, the data is sent to the named entity recognition decoding layer, which consists of GRU, Softmax, and pooling layers, as shown below.

[0110] ① Define the embedding matrix based on the known entity categories.

[0111] ② Convert each word vector e i Its corresponding entity relationship embedding vector z is obtained by splicing i The formula is shown below.

[0112]

[0113] ③Use GRU to obtain z i The hidden layer vector h i The formula is shown below.

[0114] h i =GRU(z i ,h i-1 (3)

[0115] ④ Use Softmax to obtain the final entity label corresponding to each word vector. The formula is shown below, W seq ,b seq It refers to weights and biases.

[0116]

[0117] ⑤ All word vectors e after the predicted labels i The word vector e is obtained after the pooling layer. s This provides input for the subsequent relation extraction decoding layer, as shown in the formula below.

[0118] e s ∶=pool(e i ,e i+1 ,…,e i+k-1 (5)

[0119] 4) Obtain the output word vector E = (e1,…,e1) of the named entity recognition decoding layer. s-1 ,e s After s = 1, 2, ..., n, the data is fed into the relation extraction and decoding layer to obtain the final entity relation triplet, as shown below.

[0120] ① For every two word vectors s1 and s2, concatenate them according to the context embedding vector c obtained from BERT above, and output the intermediate vector x. r (s1,s2), the formula is shown below.

[0121] x r (s1,s2):=[e(s1);c(s1,s2);e(s2)] (6)

[0122] ②The intermediate vector x r (s1,s2) passes through a sigmoid layer to obtain the final relation. The formula is shown below. b rel It refers to weights and biases.

[0123]

[0124] 5) Concatenate every two entities E1, E2 with their corresponding relation R to form a triple T = [E1; R; E2]. Count all entity-relation triples in each file, and finally output the entity-relation triple set T = {T1, T2, ..., n}.

[0125] Preferably, the data management module includes a maintenance submodule, which creates a data table Table for recording file storage information during system initialization. The table includes fields such as file name, file MD5 value, file storage address, file access count, and file priority.

[0126] Specifically, the main function of the maintenance submodule is to initialize and maintain the file storage data table. Specifically, during system initialization, the maintenance submodule creates a data table (Table) to record file storage information. The specific fields include filename (Name), file MD5 hash (MD5), file storage address (Address), file access count (Count), and file priority (containing four priorities: 0, 1, 2, and 3, corresponding to storing 0, 1, 2, and 3 data backups on the slave server, respectively). The data structure is as follows: Figure 6 As shown. After the data table is created, when subsequent users make different requests, the management module fills in or modifies the data of the corresponding file in the data table based on the parameters passed by the management module. If the parameter passed by the management module is the first file upload, the module adds a row to the data table and fills in the corresponding values ​​(Name, MD5, Address, Count, Priority) in each field according to the passed parameters.

[0127] Preferably, the data management module includes a management submodule, which performs operations such as file uploading, file deletion, and file access according to user requests.

[0128] Preferably, when a user sends a file upload request, the management submodule uses the MD5 algorithm to generate a 128-bit ID for the file, and determines whether the file already exists based on the data table Table in the maintenance submodule. If the file does not exist, the priority module is called to calculate the priority of the file. Based on the priority value, the management submodule calls the data backup module to store the file on the slave server and regenerate the file storage addresses on the master server and slave server, and passes the corresponding parameters to the maintenance submodule.

[0129] Specifically, such as Figure 7As shown in the file upload process, when a user sends a file upload request, the management submodule uses the MD5 algorithm to generate a 128-bit ID for the file and checks whether the file already exists based on the data table in the maintenance submodule. If it does not exist, the priority module is called to calculate the file's priority. After obtaining the priority, if the file priority is not 0, the data backup module is called to store the file on the slave server and generate the file storage address on both the master and slave servers. If the file priority is 0, the file is only stored on the master server and the storage address is generated. After obtaining the storage address, the management submodule sets the access count to 0 and passes the corresponding parameters to the maintenance submodule. The maintenance submodule records the corresponding information of the file in the data table. If the management submodule determines that the file already exists, the maintenance submodule increments the file access count (Count) by 1 and returns it to the management submodule. The management submodule then sends the access count to the priority module. If the file access count is greater than the user-specified threshold K (threshold K has three values: K1, K2, and K3), the file priority (Priority) is updated according to the user-defined threshold (Priority is set to 0 if the access count is less than K1, 1 if the access count is greater than K1 and less than K2, 2 if the access count is greater than K2 and less than K3, and 3 if the access count is greater than K3). The management submodule then returns the priority to the management submodule. Based on the priority value, the management submodule calls the data backup module to store the file on the slave server and regenerates the file storage address (Address) on the master server and slave servers (selecting 0, 1, 2, or 3 slave servers to store the file based on the priority value). The management submodule then passes the corresponding parameters to the maintenance submodule, which updates the corresponding information for the file in the data table. Otherwise, the process ends.

[0130] Preferably, when a user sends a file deletion request, the management submodule uses the MD5 algorithm to generate a 128-bit ID for the file and determines whether the file already exists based on the data table Table in the maintenance submodule. If the file exists, the storage address of the file is sent to the data backup module, which performs the corresponding file deletion operation on the master server and slave server. If the file does not exist, the user is prompted that the file does not exist, and the process ends.

[0131] Specifically, such as Figure 8As shown in the file deletion process, when a user sends a file deletion request, the management submodule uses the MD5 algorithm to generate a 128-bit ID for the file and determines whether the file already exists based on the data table Table in the maintenance submodule. If it exists, the storage address of the file is sent to the data backup module, which then performs the corresponding file deletion operation on the master server and slave server. If the file does not exist, the user is prompted that the file does not exist, and the process ends.

[0132] Preferably, when a user sends a file deletion request, the management submodule uses the MD5 algorithm to generate a 128-bit ID for the file, and determines whether the file already exists based on the data table in the maintenance submodule. If it exists, the management submodule calculates the network distance between the storage address and the user based on the file's storage address, selects the file closest to the user, and sends it back to the user. If the number of file accesses exceeds the threshold specified by the user, the management submodule calls the data backup module to store the file on the slave server and regenerates the file storage addresses on the master server and slave server, and passes the corresponding parameters to the maintenance submodule. The maintenance submodule then updates the corresponding information of the file in the data table.

[0133] Specifically, such as Figure 9 As shown in the file access process, when a user sends a file deletion request, the management submodule uses the MD5 algorithm to generate a 128-bit ID for the file and checks the data table in the maintenance submodule to see if the file already exists. If it exists, the management submodule calculates the network distance between the storage address and the user based on the file's storage address, selects the file closest to the user (sends a TCP handshake request, and selects the storage address with the shortest time interval based on the time interval), and sends it back to the user. At the same time, the maintenance submodule increments the file access count by 1 and returns it to the management submodule. If the file access count is greater than the threshold K specified by the user, the management submodule calls the data backup module to store the file on the slave server and regenerates the file storage addresses on the master server and slave server, and passes the corresponding parameters to the maintenance submodule. The maintenance submodule updates the corresponding information of the file in the data table. Otherwise, the process ends.

[0134] As described above, the data backup system based on dynamic strategies provided in this embodiment involves a client sending file requests, including but not limited to file upload requests, file deletion requests, and file access requests. The master server receives these requests and responds according to the instructions of the data management module, priority calculation module, and data backup module. The server primarily executes data backup or deletion operations based on the instructions from the data backup module. All three modules of this system run on a cloud computing master server. The data management module maintains a data table for file storage and generates a 128-bit ID for each uploaded file using the MD5 message digest algorithm. Based on the ID and the result calculated by the priority calculation module, it determines whether the file needs to be backed up. If backup is required, it notifies the data backup module to perform the corresponding operation. The priority calculation module calculates the priority of each file based on the user-defined strategy, the number of file accesses, and a joint named entity recognition and relation extraction method, and returns the result to the data management module. The data backup module sends relevant instructions to the master and slave servers according to the commands from the data management module to perform file backup or deletion, thereby achieving low-cost file storage.

[0135] This disclosure embodiment introduces a file priority calculation strategy (low-priority files are only saved as is, while high-priority files are backed up) to dynamically back up data. Compared with existing data backup methods, this can reduce the cost of data backup and optimize cloud service data storage.

[0136] Based on the same inventive concept, this disclosure also provides a data backup device based on a dynamic strategy, as described in the following embodiments. Since the principle by which this device solves the problem is similar to that of the method embodiments described above, the implementation of this device embodiment can refer to the implementation of the method embodiments described above, and repeated details will not be elaborated further.

[0137] Figure 10 This illustration shows a schematic diagram of a data backup device based on a dynamic strategy according to an embodiment of the present disclosure, such as... Figure 10 As shown, the device includes: a file receiving module 101, a file identification module 102, a first priority calculation module 103, a second priority calculation module 104, and a file backup module 105.

[0138] The system includes a file receiving module 101 for receiving files uploaded by the client to be backed up; a file identification module 102 for determining whether the file to be backed up is the first file uploaded; a first priority calculation module 103 for determining the file priority of the file to be backed up based on the entity relationship triplet if the file to be backed up is the first file uploaded; a second priority calculation module 104 for determining the file priority of the file to be backed up based on the number of times the file to be backed up is accessed if the file to be backed up is not the first file uploaded; and a file backup module 105 for backing up the file to be backed up based on its file priority.

[0139] It should be noted that the file receiving module 101, file identification module 102, first priority calculation module 103, second priority calculation module 104, and file backup module 105 mentioned above correspond to S202 to S210 in the method embodiment. The examples and application scenarios implemented by the above modules and corresponding steps are the same, but are not limited to the content disclosed in the above method embodiment. It should be noted that the above modules, as part of the apparatus, can be executed in a computer system such as a set of computer-executable instructions.

[0140] In some embodiments, the file backup module 105 is further configured to: determine one or more servers for storing the file to be backed up based on the file priority of the file to be backed up; and store the file to be backed up on one or more servers.

[0141] In some embodiments, the first priority calculation module 103 is further configured to: obtain the file content of the file to be backed up; extract entity relation triples from the file content of the file to be backed up; obtain the number of times the entity relation triples appear in a preset corpus; and determine the file priority of the file to be backed up based on the number of times the entity relation triples appear in the preset corpus.

[0142] In some embodiments, the file recognition module 102 is further configured to: generate the MD5 value of the file to be backed up using the MD5 digest algorithm; and determine whether the file to be backed up is the first file uploaded based on the MD5 value of the file to be backed up.

[0143] In some embodiments, the data backup device based on dynamic policies provided in this disclosure may further include: a data deletion module 106, configured to receive a file deletion request from a client, wherein the file deletion request is used to request the deletion of one or more files; determine whether the file to be deleted exists; and if the file to be deleted exists, delete the file to be deleted.

[0144] In some embodiments, the data backup device based on dynamic strategy provided in this disclosure may further include: a data access module 107, configured to receive a file access request from a client, wherein the file access request is used to request access to one or more files; determine whether the file to be accessed exists; if the file to be accessed exists, return the corresponding file to the client and record the access count of the corresponding file.

[0145] Those skilled in the art will understand that various aspects of this disclosure can be implemented as a system, method, or program product. Therefore, various aspects of this disclosure can be specifically implemented in the following forms: a completely hardware implementation, a completely software implementation (including firmware, microcode, etc.), or a combination of hardware and software aspects, collectively referred to herein as a "circuit," "module," or "system."

[0146] The following reference Figure 11 To describe an electronic device 1100 according to such an embodiment of the present disclosure. Figure 11 The electronic device 1100 shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments disclosed herein.

[0147] like Figure 11 As shown, the electronic device 1100 is manifested in the form of a general-purpose computing device. The components of the electronic device 1100 may include, but are not limited to: at least one processing unit 1110, at least one storage unit 1120, and a bus 1130 connecting different system components (including storage unit 1120 and processing unit 1110).

[0148] The storage unit stores program code that can be executed by the processing unit 1110, causing the processing unit 1110 to perform the steps described in the "Exemplary Methods" section of this specification according to various exemplary embodiments of this disclosure. For example, the processing unit 1110 can perform the following steps of the above method embodiments: receiving a file upload request from a client, wherein the file upload request is used to request the upload of one or more files to be backed up; determining whether the file to be backed up is a file being uploaded for the first time; if the file to be backed up is a file being uploaded for the first time, determining the file priority of the file to be backed up based on the entity relationship triplet of the file to be backed up; if the file to be backed up is not a file being uploaded for the first time, determining the file priority of the file to be backed up based on the number of times the file to be backed up has been accessed; and backing up the file to be backed up according to the file priority of the file to be backed up.

[0149] Storage unit 1120 may include a readable medium in the form of a volatile storage unit, such as random access memory (RAM) 11201 and / or cache memory 11202, and may further include a read-only memory (ROM) 11203.

[0150] Storage unit 1120 may also include a program / utility 11204 having a set (at least one) of program modules 11205, such program modules 11205 including but not limited to: operating system, one or more application programs, other program modules and program data, each or some combination of these examples may include an implementation of a network environment.

[0151] Bus 1130 can represent one or more of several types of bus structures, including a memory cell bus or memory cell controller, a peripheral bus, a graphics acceleration port, a processing unit, or a local bus using any of the various bus structures.

[0152] Electronic device 1100 can also communicate with one or more external devices 1140 (e.g., keyboard, pointing device, Bluetooth device, etc.), and with one or more devices that enable a user to interact with electronic device 1100, and / or with any device that enables electronic device 1100 to communicate with one or more other computing devices (e.g., router, modem, etc.). This communication can be performed via input / output (I / O) interface 1150. Furthermore, electronic device 1100 can also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN), and / or public networks, such as the Internet) via network adapter 1160. As shown, network adapter 1160 communicates with other modules of electronic device 1100 via bus 1130. It should be understood that, although not shown in the figures, other hardware and / or software modules can be used in conjunction with electronic device 1100, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0153] From the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein can be implemented by software or by combining software with necessary hardware. Therefore, the technical solutions according to the embodiments of this disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, USB flash drive, external hard drive, etc.) or on a network, including several instructions to cause a computing device (such as a personal computer, server, terminal device, or network device, etc.) to execute the methods according to the embodiments of this disclosure.

[0154] In particular, according to embodiments of this disclosure, the process described above with reference to the flowchart can be implemented as a computer program product, which includes a computer program that, when executed by a processor, implements the above-described data backup method based on a dynamic strategy.

[0155] In exemplary embodiments of this disclosure, a computer-readable storage medium is also provided, which may be a readable signal medium or a readable storage medium. Figure 12 This illustration shows a schematic diagram of a computer-readable storage medium according to an embodiment of the present disclosure, such as... Figure 12 As shown, the computer-readable storage medium 1200 stores a program product capable of implementing the methods described above in this disclosure. In some possible embodiments, various aspects of this disclosure may also be implemented as a program product comprising program code that, when run on a client, causes the client to perform the steps described in the "Exemplary Methods" section of this specification according to various exemplary embodiments of this disclosure.

[0156] More specific examples of computer-readable storage media in this disclosure may include, but are not limited to: electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0157] In this disclosure, a computer-readable storage medium may include a data signal propagated in baseband or as part of a carrier wave, carrying readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A readable signal medium may also be any readable medium other than a readable storage medium, capable of transmitting, propagating, or transmitting a program for use by or in connection with an instruction execution system, apparatus, or device.

[0158] Optionally, the program code contained on the computer-readable storage medium may be transmitted using any suitable medium, including but not limited to wireless, wired, optical fiber, RF, etc., or any suitable combination thereof.

[0159] In practical implementation, program code for performing the operations of this disclosure can be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java and C++, and conventional procedural programming languages ​​such as C or similar languages. The program code can execute entirely on the user's computing device, partially on the user's device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing device can be connected to the user's computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0160] It should be noted that although several modules or units for the device used to perform actions have been mentioned in the detailed description above, this division is not mandatory. In fact, according to embodiments of this disclosure, the features and functions of two or more modules or units described above can be embodied in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided and embodied by multiple modules or units.

[0161] Furthermore, although the steps of the method in this disclosure are described in a specific order in the accompanying drawings, this does not require or imply that the steps must be performed in that specific order, or that all the steps shown must be performed to achieve the desired result. Additional or alternative steps may be omitted, multiple steps may be combined into one step, and / or a step may be broken down into multiple steps.

[0162] From the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein can be implemented by software or by combining software with necessary hardware. Therefore, the technical solutions according to the embodiments of this disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, USB flash drive, external hard drive, etc.) or on a network, including several instructions to cause a computing device (such as a personal computer, server, mobile terminal, or network device, etc.) to execute the methods according to the embodiments of this disclosure.

[0163] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This disclosure is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the appended claims.

Claims

1. A data backup method based on dynamic strategies, characterized in that, include: Receive a file upload request from a client, wherein the file upload request is used to request the upload of one or more files to be backed up; Determine whether the file to be backed up is the first file uploaded; If the file to be backed up is the first file uploaded, the file priority of the file to be backed up is determined according to the number of times the entity relation triple of the file to be backed up appears in the preset corpus. If the file to be backed up is not the first file uploaded, then the file priority of the file to be backed up is determined according to the number of times the file to be backed up has been accessed. Based on the file priority of the file to be backed up, it is determined whether to back up the file on the master server or the slave server. Low-priority files are only saved as is, while high-priority files are backed up.

2. The data backup method based on dynamic strategy according to claim 1, characterized in that, Determining whether the file to be backed up is a file uploaded for the first time includes: The MD5 value of the file to be backed up is generated using the MD5 message digest algorithm. Based on the MD5 value of the file to be backed up, determine whether the file is the first file uploaded.

3. The data backup method based on dynamic strategy according to claim 1, characterized in that, After backing up the files to be backed up according to their file priorities, the method further includes: Receive a file deletion request from a client, wherein the file deletion request is used to request the deletion of one or more files; Determine if the file to be deleted exists; If there are files to be deleted, then delete those files.

4. The data backup method based on dynamic strategy according to claim 1, characterized in that, After backing up the files to be backed up according to their file priorities, the method further includes: Receive a file access request from a client, wherein the file access request is used to request access to one or more files; Determine if the file to be accessed exists; If a file exists to be accessed, the corresponding file is returned to the client, and the number of times the corresponding file has been accessed is recorded.

5. A data backup system based on dynamic strategies, characterized in that, include: Client, master server, and at least one slave server; The main server includes: a priority calculation module, a data management module, and a data backup module; The client is used to send a file upload request to the main server, wherein the file upload request is used to request the upload of one or more files to be backed up; The data management module is used to receive file upload requests from clients and determine whether the file to be backed up in the file upload request is the first file uploaded. The priority calculation module is used to: if the file to be backed up is the first file uploaded, determine the file priority of the file to be backed up based on the number of times the entity relation triplet of the file to be backed up appears in the preset corpus; if the file to be backed up is not the first file uploaded, determine the file priority of the file to be backed up based on the number of times the file to be backed up is accessed. The data backup module is used to determine whether to back up the file to be backed up on the master server or the slave server according to the file priority of the file to be backed up. Low priority files only save the file itself, while high priority files are backed up.

6. A data backup device based on a dynamic strategy, characterized in that, include: The file receiving module is used to receive backup files uploaded by the client. The file recognition module is used to determine whether the file to be backed up is the first file uploaded. The first priority calculation module is used to determine the file priority of the file to be backed up based on the number of times the entity relation triplet of the file to be backed up appears in the preset corpus if the file to be backed up is the first file uploaded. The second priority calculation module is used to determine the file priority of the file to be backed up based on the number of times the file to be backed up has been accessed if the file to be backed up is not the first file uploaded. The file backup module is used to determine whether to back up the file to be backed up on the master server or the slave server according to the file priority of the file to be backed up. Low priority files only save the file itself, while high priority files are backed up.

7. An electronic device, characterized in that, include: processor; as well as Memory for storing the executable instructions of the processor; The processor is configured to execute the data backup method based on any one of claims 1 to 4 by executing the executable instructions.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the data backup method based on dynamic strategy as described in any one of claims 1 to 4.

Images