File synchronization method and device, computer equipment and storage medium

By filtering and matching file systems in cloud-edge collaborative scenarios, determining synchronization order and encrypting processing, the problem of low synchronization efficiency of heterogeneous file systems is solved, and efficient and secure file synchronization is achieved.

CN119003474APending Publication Date: 2024-11-22NANJING HUADUN ELECTRIC POWER INFORMATION SAFETY EVALUATION CO LTD
View PDF 3 Cites 0 Cited by

Patent Information

Application Number
CN202411212770.2
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-30
Publication Date
2024-11-22

AI Technical Summary

Technical Problem

In the cloud-edge collaboration scenario, the file synchronization efficiency of heterogeneous file systems is low, and a single protocol is difficult to adapt to the diverse file system needs.

Method used

By obtaining the synchronization policies for files to be synchronized and configured between the central cloud and the edge cloud, filtering the matching target file system, determining the synchronization order based on the system status information, and synchronizing files in sequence based on the order, and improving security using encryption.

Benefits of technology

Improve file synchronization efficiency and security, and realize efficient and secure data transmission between heterogeneous file systems.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119003474A_ABST
    Figure CN119003474A_ABST
Patent Text Reader

Abstract

The invention relates to a file synchronization method and device, computer equipment, a storage medium and a computer program product. The method comprises the steps that a central cloud is connected with a plurality of edge clouds, the central cloud comprises a central file system used for managing files, each edge cloud comprises an edge file system used for managing the files, and the to-be-synchronized files stored in the central file system and a central synchronization strategy configured for the central file system are obtained. In each edge file system, a plurality of target file systems matched with the center synchronization strategy are screened, the synchronization direction of the to-be-synchronized file is accurately determined, and the file synchronization efficiency is improved. And determining the respective synchronization sequence of each target file system according to the respective system state information of each target file system, determining the synchronization sequence by adopting the method, and sequentially synchronizing the to-be-synchronized file into each target file system based on each synchronization sequence, so that the ordered synchronization of the to-be-synchronized file is realized, and the file synchronization efficiency is improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of data security technology, and in particular to a file synchronization method, apparatus, computer equipment, storage medium and computer program product. Background Art

[0002] In the cloud-edge collaboration scenario, the central cloud and edge cloud services need to synchronize files in heterogeneous file systems. These heterogeneous file system data file transfer protocols are diverse, and file synchronization tools with a single protocol are difficult to adapt to this demand.

[0003] In the traditional method, a file synchronization tool is used to convert files, and the converted files are synchronously transmitted, but there is still a problem of low synchronization efficiency. Summary of the invention

[0004] Based on this, it is necessary to provide a file synchronization method, apparatus, computer device, computer-readable storage medium and computer program product that can improve synchronization efficiency in response to the above technical problems.

[0005] In a first aspect, the present application provides a file synchronization method. The method is applied to a central cloud, the central cloud is connected to multiple edge clouds respectively, the central cloud includes a central file system for managing files, and each edge cloud includes an edge file system for managing files, and the method includes:

[0006] Acquire the files to be synchronized stored in the central file system and the central synchronization policy configured for the central file system; select multiple target file systems matching the central synchronization policy in each of the edge file systems; determine the synchronization order of each of the target file systems according to the system status information of each of the target file systems; and synchronize the files to be synchronized to each of the target file systems in sequence based on the synchronization orders.

[0007] In one of the embodiments, based on each of the synchronization orders, synchronizing the files to be synchronized to each of the target file systems in sequence, including: obtaining a plurality of candidate encryption methods in the central cloud; screening a target encryption method that matches each of the target file systems among the candidate encryption methods; encrypting the files to be synchronized according to the target encryption method to obtain encrypted files of the files to be synchronized; based on each of the synchronization orders, synchronizing the encrypted files to each of the target file systems in sequence.

[0008] In one of the embodiments, determining the synchronization order of each target file system according to the system status information of each target file system includes: determining the expected synchronization duration of each target file system according to the system status information of each target file system; and determining the synchronization order of each target file system according to the expected synchronization duration.

[0009] In one of the embodiments, determining the synchronization order of each of the target file systems according to each of the expected synchronization times includes: sorting each of the target file systems according to each of the expected synchronization times to obtain a file system sequence; dividing each of the target file systems into a plurality of system groups according to the position of each of the target file systems in the file system sequence; the number of target file systems included in each of the system groups is the same as the number of file synchronization threads of the central cloud; and for each of the target file systems, determining the synchronization order of the system group to which the target file system belongs as the synchronization order of the target file system.

[0010] In one of the embodiments, based on each synchronization order, synchronizing the files to be synchronized to each target file system in sequence includes: determining, from each target file system, a first file system that has completed file synchronization, and a second file system that has not completed file synchronization; synchronizing the files to be synchronized from the first file system to the second file system.

[0011] In a second aspect, the present application further provides a file synchronization method, the method being applied to an edge cloud, the edge cloud being connected to a central cloud, the edge cloud comprising an edge file system for managing files, the central cloud comprising multiple central file systems for managing files, the method comprising:

[0012] The method comprises the steps of: obtaining files to be synchronized stored in the edge file system and an edge synchronization strategy configured for the edge file system; screening a plurality of target file systems matching the edge synchronization strategy in each of the central file systems; determining a synchronization order for each of the target file systems according to system status information of each of the target file systems; and synchronizing the files to be synchronized to each of the target file systems in sequence based on the synchronization orders.

[0013] In a third aspect, the present application also provides a file synchronization device. The device comprises:

[0014] An acquisition module is used to acquire the files to be synchronized stored in the central file system and the central synchronization policy configured for the central file system; a screening module is used to screen multiple target file systems that match the central synchronization policy in each of the edge file systems; an order determination module is used to determine the synchronization order of each of the target file systems according to the system status information of each of the target file systems; and a synchronization module is used to synchronize the files to be synchronized to each of the target file systems in sequence based on the synchronization order.

[0015] In a fourth aspect, the present application further provides a computer device. The computer device includes a memory and a processor, the memory stores a computer program, and the processor implements the following steps when executing the computer program:

[0016] Acquire the files to be synchronized stored in the central file system and the central synchronization policy configured for the central file system; select multiple target file systems matching the central synchronization policy in each of the edge file systems; determine the synchronization order of each of the target file systems according to the system status information of each of the target file systems; and synchronize the files to be synchronized to each of the target file systems in sequence based on the synchronization orders.

[0017] In a fifth aspect, the present application further provides a computer-readable storage medium. The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the following steps are implemented:

[0018] Acquire the files to be synchronized stored in the central file system and the central synchronization policy configured for the central file system; select multiple target file systems matching the central synchronization policy in each of the edge file systems; determine the synchronization order of each of the target file systems according to the system status information of each of the target file systems; and synchronize the files to be synchronized to each of the target file systems in sequence based on the synchronization orders.

[0019] In a sixth aspect, the present application further provides a computer program product. The computer program product includes a computer program, and when the computer program is executed by a processor, the following steps are implemented:

[0020] Acquire the files to be synchronized stored in the central file system and the central synchronization policy configured for the central file system; select multiple target file systems matching the central synchronization policy in each of the edge file systems; determine the synchronization order of each of the target file systems according to the system status information of each of the target file systems; and synchronize the files to be synchronized to each of the target file systems in sequence based on the synchronization orders.

[0021] In the above-mentioned file synchronization method, device, computer equipment, storage medium and computer program product, the central cloud is connected to multiple edge clouds respectively, the central cloud contains a central file system for managing files, and each edge cloud contains an edge file system for managing files. By obtaining the files to be synchronized stored in the central file system and the central synchronization policy configured for the central file system, multiple target file systems matching the central synchronization policy are screened in each edge file system to accurately determine the synchronization direction of the files to be synchronized, which is conducive to improving the efficiency of file synchronization. According to the system status information of each target file system, the synchronization order of each target file system is determined, and the above-mentioned method is used to determine the synchronization order. Based on each synchronization order, the files to be synchronized are synchronized to each target file system in turn, thereby achieving orderly synchronization of the files to be synchronized and improving the efficiency of file synchronization. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] FIG1 is a diagram of an application environment of a file synchronization method in one embodiment;

[0023] FIG2 is a schematic diagram of a flow chart of a file synchronization method in one embodiment;

[0024] FIG3 is a flowchart of a file collaborative management method for a cloud-edge collaborative scenario in one embodiment;

[0025] FIG4 is a schematic diagram of the architecture of a central cloud and an edge cluster in one embodiment;

[0026] FIG5 is a structural block diagram of a file synchronization device in one embodiment;

[0027] FIG. 6 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0028] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0029] The file synchronization method provided in the embodiment of the present application can be applied to the application environment shown in Figure 1. Among them, the central server 102 and the edge server 104 communicate. The data storage system can store the data that the central server 102 and the edge server 104 need to process. The data storage system can be integrated on the central server 102 and the edge server 104, or it can be placed on the cloud or other network servers.

[0030] The central server 102 obtains the files to be synchronized stored in the central file system and the central synchronization policy configured for the central file system. The central server 102 selects multiple target file systems that match the central synchronization policy in each edge file system. The central server 102 determines the synchronization order of each target file system according to the system status information of each target file system. The central server 102 synchronizes the files to be synchronized to each target file system in turn based on each synchronization order.

[0031] The central server 102 and the edge server 104 may be implemented as independent servers or a server cluster consisting of multiple servers.

[0032] In one embodiment, as shown in FIG. 2 , a file synchronization method is provided, which is described by taking the method applied to the central server in FIG. 1 as an example, including:

[0033] S202, obtaining files to be synchronized stored in the central file system and a central synchronization policy configured for the central file system.

[0034] Among them, it is applied to the central cloud, the central cloud is connected to multiple edge clouds respectively, the central cloud includes a central file system for managing files, and each edge cloud includes an edge file system for managing files.

[0035] The central cloud refers to a large collection of computing resources and services located at the center of the network. It usually includes a large number of servers, storage devices, and network devices, which are used to provide various cloud computing services. The central cloud is the core part of cloud computing, responsible for managing and scheduling the resources and services of the entire cloud computing system. The central cloud is based on high-performance computers and provides computing services to all walks of life.

[0036] Edge cloud refers to a set of computing resources and services deployed at the edge of the network. These resources and servers are organized into a cluster for better management and utilization. Edge cloud can be a physical server, virtual machine or container, etc. Edge cloud is distributed at the edge of the network to be close to where data is generated and used. Based on high-performance computers, edge cloud provides computing services to all walks of life, or to institutions or units.

[0037] A large amount of data exchange is involved between the central cloud and the edge cloud, for example, the transmission of file data. More specifically, the central server transmits the file data in the central cloud to the edge cloud.

[0038] The central file system for managing files refers to a file system that has the function of managing files or has management services, and managing files means analyzing, calculating, forwarding, synchronizing, etc. the files. The central file system can be a MinIO file system, a Ceph file system, an FTP file system, etc.

[0039] The edge file system for managing files refers to a file system that has a file management function or a management service, and file management refers to analyzing, calculating, forwarding, synchronizing, etc. the files. The edge file system may be an SFTP file system, a MinIO file system, etc.

[0040] The number of central file systems in the central cloud can be multiple, and the number of edge file systems for an edge cloud can also be at least one. Data exchange is involved between file systems, and the objects of data exchange can be files. The computers that the central cloud relies on have stronger computing power and storage capacity than the computers that the edge cloud relies on. In the cloud-edge collaboration scenario, the central file system of the central cloud needs to synchronize the files to be synchronized with the edge file system of the edge cloud.

[0041] The files to be synchronized may be a set of file data stored in a central file system.

[0042] The central synchronization strategy represents the mapping relationship between the central file system and the edge file system.

[0043] Specifically, the central server obtains the to-be-synchronized files stored in the central file system and the central synchronization policy configured for the central file system.

[0044] In one of the embodiments, in response to a configuration request for a file system, the central server obtains a central synchronization policy configured for the central file system from a page to which the configuration request belongs.

[0045] In one embodiment, the central server obtains the to-be-synchronized files from a database corresponding to the central file system, wherein the database is used to persist the to-be-synchronized files and the central synchronization strategy.

[0046] In one of the embodiments, the central server obtains the central synchronization policy configured for the central file system from a database corresponding to the central file system.

[0047] S204: Filter multiple target file systems that match the central synchronization policy in each edge file system.

[0048] There are multiple edge file systems, and the central synchronization strategy represents a mapping relationship between the central file system and the edge file system.

[0049] The screening means screening multiple edge file systems to obtain a target file system that matches the central synchronization policy.

[0050] Specifically, the central server can parse the mapping relationship between the central file system and the edge file system to obtain the target system type, target deployment location, etc. of the edge file system. The target system type indicates the type of the edge file system, such as SFTP file system, MinIO file system, etc. The target deployment location can indicate the physical storage location of the server corresponding to the edge cloud to which the edge file system belongs.

[0051] In one embodiment, the central server may screen the edge systems according to the target system type and the system types of the edge file systems to obtain the target file system whose system type is the target system type.

[0052] In one embodiment, the central server may screen the edge systems according to the target deployment location of the edge file system and the respective deployment locations of the edge file systems to obtain the target file system whose deployment location is the target deployment location.

[0053] In one example, the central server may jointly screen the edge systems according to the target deployment location and target system type of the edge file system to obtain a target file system whose deployment location is the target deployment location and whose system type is the target system type.

[0054] S206: Determine the synchronization order of each target file system according to the system status information of each target file system.

[0055] The system status information indicates the server status information of the edge server (edge ​​cloud) to which the target file system belongs. If there are multiple target file systems in an edge cloud, the system status information of the target file system may indicate the system resource status information allocated to the target file system.

[0056] The system status information includes IO idle rate, CPU load rate, network status, etc. More specifically, the system status information indicates the IO idle rate, CPU load rate, network status, etc. of the server to which the target file system belongs.

[0057] The system status information represents the ability and efficiency of the target file system to receive files. In the cloud-edge collaboration scenario, the central server often needs to synchronize the files to be synchronized in the central file system to multiple edge file systems, so the order of synchronization is involved.

[0058] The synchronization order may indicate the order of the start synchronization times of synchronizing the same to-be-synchronized file to different target file systems.

[0059] For example, a scenario in which a certain file to be synchronized is synchronized to target file systems 1, 2, and 3.

[0060] Specifically, the central server synchronizes the files to be synchronized in the central file system to target file system 1, target file system 2, and target file system 3. Due to the limitation on the number of threads used for synchronization in the central server, the central server needs to determine the synchronization order corresponding to target file system 1, target file system 2, and target file system 3.

[0061] Specifically, the central server may determine the synchronization order of each target file system according to at least one of the IO idle rate, CPU load rate, and network status of each target file system.

[0062] In one of the embodiments, the central server can determine the synchronization order of each target file system according to the IO idle rate of each target file system. The higher the IO idle rate, the greater the synchronization order, and the lower the IO idle rate, the smaller the synchronization order. That is, the central server will give priority to sending the files to be synchronized to the target file systems with lower IO idle rates, thereby reducing the total synchronization time consumed by each synchronization process.

[0063] In one of the embodiments, the central server can determine the synchronization order of each target file system according to the CPU load rate of each target file system. The higher the CPU load rate, the smaller the synchronization order, and the lower the CPU load rate, the larger the synchronization order. That is, the central server will give priority to sending the files to be synchronized to the target file systems with high CPU load rates, thereby reducing the total synchronization time consumed by each synchronization process.

[0064] In one embodiment, the central server may determine the synchronization order of each target file system according to the IO idle rate of each target file system and the CPU load rate of each target file system.

[0065] Specifically, the central server may first determine the expected synchronization duration of each target file system according to the system status information of each target file system, and determine the synchronization order of each target file system according to the expected synchronization duration.

[0066] S208: Based on each synchronization order, synchronize the files to be synchronized to each target file system in sequence.

[0067] Limited by the number of threads of the central server, the number of tasks to be synchronized simultaneously for the synchronized files is less than or equal to the number of threads.

[0068] Specifically, after obtaining the synchronization order of each target file system, the central server synchronizes the files to be synchronized to each target file system in sequence according to the synchronization times.

[0069] In one embodiment, the central server can synchronize the to-be-synchronized files in the central file system to the target file system. The central server can also control the target file system that has completed synchronization to copy the to-be-synchronized files to the target file that has not completed synchronization.

[0070] In the above file synchronization method, the central cloud is connected to multiple edge clouds respectively. The central cloud contains a central file system for managing files, and each edge cloud contains an edge file system for managing files. By obtaining the files to be synchronized stored in the central file system and the central synchronization policy configured for the central file system, multiple target file systems matching the central synchronization policy are screened in each edge file system to accurately determine the synchronization direction of the files to be synchronized, which is conducive to improving the efficiency of file synchronization. According to the system status information of each target file system, the synchronization order of each target file system is determined. The above method is used to determine the synchronization order. Based on each synchronization order, the files to be synchronized are synchronized to each target file system in turn, thereby achieving orderly synchronization of the files to be synchronized and improving the efficiency of file synchronization.

[0071] In one embodiment, based on each synchronization order, the files to be synchronized are synchronized to each target file system in sequence, including: obtaining multiple candidate encryption methods in the central cloud, screening the target encryption method that matches each target file system among the candidate encryption methods, encrypting the files to be synchronized according to the target encryption method to obtain the encrypted files of the files to be synchronized, and synchronizing the encrypted files to each target file system in sequence based on each synchronization order.

[0072] The candidate encryption method refers to a method for encrypting the to-be-synchronized file, and the candidate encryption method can be used to encrypt the to-be-synchronized file to improve the security of the synchronization process of the to-be-synchronized file. For example, the candidate encryption method may include symmetric encryption, asymmetric encryption, hash algorithm, confidentiality protocol, etc.

[0073] The target encryption method refers to the encryption method adapted by the target file system. More specifically, when the target file system has a decryption method corresponding to the candidate encryption method, the candidate encryption method is determined as the target encryption method.

[0074] The encryption process may refer to performing encryption process on the file to be synchronized to obtain an encrypted file to be synchronized, that is, an encrypted file.

[0075] Specifically, the central server synchronizes the encrypted file to the target file system. Since the target file system has a decryption method for the target encryption method, the target file system can decrypt the encrypted file based on the decryption method to obtain the file to be synchronized. This can improve the security of the file to be synchronized during the synchronization transmission process and avoid data cracking and data leakage.

[0076] In one implementation, the central server sends the encrypted file to the target file system through different communication protocols. The communication protocol may be a communication protocol adapted to the file system. Specifically, the central server determines the communication protocol adapted to the type of the central file system and the type of the target file system based on the type of the central file system and the type of the target file system. The communication protocols include TCP / IP protocol, UDP / IP protocol, HTTPS protocol, and HTTP protocol.

[0077] In this embodiment, multiple candidate encryption methods are obtained in the central cloud. Among the candidate encryption methods, the target encryption method that matches each target file system is screened, and the files to be synchronized are encrypted according to the target encryption method to obtain the encrypted files of the files to be synchronized. Based on each synchronization order, the encrypted files are synchronized to each target file system in turn to avoid the risk of data leakage and improve the security of the synchronization transmission process.

[0078] In one embodiment, the synchronization order of each target file system is determined according to the system status information of each target file system, including: determining the expected synchronization duration of each target file system according to the system status information of each target file system, and determining the synchronization order of each target file system according to the expected synchronization duration.

[0079] The system status information includes IO idle rate, CPU load rate, network status, etc. More specifically, the system status information not only indicates the system status at the current moment, but also represents the changes in the system status in the past period. In other words, the system status information represents the system status at the current moment and before. The system status reflects the resource utilization of the computer or server that the target file system relies on.

[0080] The expected synchronization duration indicates the duration required for the target file system to receive and preliminarily process the files to be synchronized. The preliminarily processing may be decrypting the encrypted files and storing the decrypted files to be synchronized in the corresponding storage location in the target file system.

[0081] The longer the expected synchronization time is, the longer it will take for the target file system to synchronize the files to be synchronized. To ensure that the total time consumed by each target file system to synchronize the files to be synchronized is the shortest, you can first synchronize the files to be synchronized to the target file system with a longer expected synchronization time, and then synchronize the files to be synchronized to the target file system with a shorter expected synchronization time.

[0082] Specifically, the central server may perform sorting based on the expected synchronization durations, and determine the synchronization order of each target file system according to the sorting result.

[0083] In one embodiment, the central server can sort based on the expected synchronization time, obtain the sorting results of the expected synchronization time, and lower the synchronization order of the target file system with a longer synchronization time, that is, the target file system with the to-be-synchronized files is synchronized first, and raise the synchronization order of the target file system with a shorter synchronization time.

[0084] In this embodiment, the expected synchronization duration of each target file system is determined according to the system status information of each target file system, and the synchronization order of each target file system is determined according to each expected synchronization duration, which can shorten the total time for synchronizing the files to be synchronized to each target file system and improve the overall synchronization efficiency of the systems to be synchronized.

[0085] In one embodiment, according to each expected synchronization duration, the synchronization order of each target file system is determined, including: sorting each target file system according to each expected synchronization duration to obtain a file system sequence, dividing each target file system into a plurality of system groups according to a position of each target file system in the file system sequence, and for each target file system, determining the synchronization order of the system group to which the target file system belongs as the synchronization order of the target file system.

[0086] The file system sequence represents a set of permutations with a certain direction formed by each target file system. The target file systems can be sorted from small to large according to each expected synchronization time to obtain the file system sequence, or the target file systems can be sorted from large to small according to each expected synchronization time to obtain the file system sequence.

[0087] The number of target file systems contained in each system group is the same as the number of file synchronization threads in the central cloud.

[0088] The number of synchronization threads indicates the number of target file systems in the central cloud that can simultaneously send files to be synchronized to the target file system. For example, the number of target file systems included in each system group and the number of file synchronization threads with the central cloud can be 3 or other numbers. The number of synchronization threads depends on the performance of the central cloud and is not limited here.

[0089] The system group includes target file systems whose number is the number of synchronization threads. Each target file system in the system group can be regarded as receiving files to be synchronized sent by the central file system at the same time.

[0090] Specifically, the central server sorts the target file systems according to the expected synchronization durations to obtain a file system sequence, and divides the target file systems into a plurality of system groups according to the positions of the target file systems in the file system sequence.

[0091] In one embodiment, the central server sorts the target file systems according to the expected synchronization duration to obtain a file system sequence, and the central server divides the target file systems at adjacent positions into a system group, thereby completing the division of the file system sequence. Target files with a smaller difference in expected synchronization duration are divided into a system group, which is conducive to reducing the total idle time of each thread and improving the efficiency of file synchronization.

[0092] Specifically, for each target file system, the central server determines the synchronization order of the system group to which the target file system belongs as the synchronization order of the target file system.

[0093] In this embodiment, each target file system is sorted according to each expected synchronization duration to obtain a file system sequence, and each target file system is divided into a plurality of system groups according to the position of each target file system in the file system sequence. For each target file system, the synchronization order of the system group to which the target file system belongs is determined as the synchronization order of the target file system, which is beneficial to reducing the total idle time of each thread and improving the file synchronization efficiency.

[0094] In one embodiment, based on each synchronization order, the files to be synchronized are synchronized to each target file system in sequence, including: from each target file system, determining the first file system that has completed file synchronization and the second file system that has not completed file synchronization, and synchronizing the files to be synchronized from the first file system to the second file system.

[0095] The first file system indicates a target file system that has completed file synchronization, and the second file system indicates a target file system that has not completed file synchronization.

[0096] It should be noted that the central server can synchronize the files to be synchronized from the central file system to the target file system, and the central server can also synchronize the files to be synchronized from one target file system to another target file system. More specifically, the central server can synchronize the files to be synchronized from the first file system to the second file system.

[0097] Specifically, the central server first sends the files to be synchronized in the central file system to the first target file system, and then sends the files to be synchronized in the central file system to the second target file system. At the same time, the central server sends the files to be synchronized in the first target file system to the third target file system, and so on. The central server synchronizes the files to be synchronized to each target file system.

[0098] In one of the embodiments, the central server sends the files to be synchronized in the central file system to multiple first target file systems with a synchronization order of one, and the central server then sends the files to be synchronized in the central file system to the target file system with a synchronization order of two. At the same time, the central server sends the files to be synchronized in the target file system with a synchronization order of one to the target file system with a synchronization order of three, and so on. The central server synchronizes the files to be synchronized to each target file system.

[0099] Specifically, the central server determines, from each target file system, a first file system that has completed file synchronization and a second file system that has not completed file synchronization, and synchronizes the to-be-synchronized files from the first file system to the second file system.

[0100] In one embodiment, the central server synchronizes the to-be-synchronized files in the first file system with the same synchronization order (for example, the synchronization order is one) to the second file system with the same synchronization order (for example, the synchronization order is two).

[0101] In this embodiment, a first file system that has completed file synchronization and a second file system that has not completed file synchronization are determined from each target file system, and the files to be synchronized are synchronized from the first file system to the second file system to further reduce the total synchronization time and improve the file synchronization efficiency.

[0102] In one embodiment, a file synchronization method is provided, which is applied to an edge cloud, wherein the edge cloud is connected to a central cloud, wherein the edge cloud includes an edge file system for managing files, and the central cloud includes multiple central file systems for managing files. The method includes: obtaining files to be synchronized stored in the edge file system, and an edge synchronization policy configured for the edge file system, screening multiple target file systems matching the edge synchronization policy in each central file system, determining a synchronization order for each target file system according to system status information of each target file system, and synchronizing the files to be synchronized to each target file system in turn based on each synchronization order.

[0103] The edge synchronization strategy represents the mapping relationship between the edge file system and the central file system.

[0104] Specifically, the edge server obtains the to-be-synchronized files stored in the edge file system and the edge synchronization policy configured for the edge file system.

[0105] In one of the embodiments, in response to a configuration request for a file system, the edge server obtains an edge synchronization policy configured for the edge file system from a page to which the configuration request belongs.

[0106] In one embodiment, the edge server obtains the to-be-synchronized file from a database corresponding to the edge file system, wherein the database is used to persist the to-be-synchronized file and the edge synchronization strategy.

[0107] In one of the embodiments, the edge server obtains the edge synchronization policy configured for the edge file system from a database corresponding to the edge file system.

[0108] There are multiple edge file systems, and the edge synchronization strategy represents a mapping relationship between the edge file system and the central file system.

[0109] The filtering means filtering multiple central file systems to obtain a target file system that matches the edge synchronization policy.

[0110] Specifically, the edge server can parse the mapping relationship between the edge file system and the edge file system to obtain the target system type, target deployment location, etc. of the central file system. The target system type indicates the type of the central file system, such as SFTP file system, MinIO file system, etc. The target deployment location can indicate the physical storage location of the server corresponding to the central cloud to which the central file system belongs.

[0111] In one embodiment, the edge server may screen each central system according to the target system type and the system type of each central file system to obtain a target file system whose system type is the target system type.

[0112] In one embodiment, the edge server may screen each central system according to the target deployment location of the central file system and the respective deployment locations of each central file system to obtain a target file system whose deployment location is the target deployment location.

[0113] In one example, the edge server may jointly screen the central systems according to the target deployment location and target system type of the central file system to obtain a target file system whose deployment location is the target deployment location and whose system type is the target system type.

[0114] After determining the target file system, the synchronization order of each target file system is determined according to the system status information of each target file system. Based on each synchronization order, the files to be synchronized are synchronized to each target file system in sequence, which is similar to the above method and will not be described in detail here.

[0115] In this embodiment, by obtaining the files to be synchronized stored in the edge file system and the edge synchronization strategy configured for the edge file system, multiple target file systems matching the edge synchronization strategy are screened in each central file system to accurately determine the synchronization direction of the files to be synchronized, which is conducive to improving the file synchronization efficiency. According to the system status information of each target file system, the synchronization order of each target file system is determined, and the synchronization order is determined by the above method. Based on each synchronization order, the files to be synchronized are synchronized to each target file system in turn, thereby achieving orderly synchronization of the files to be synchronized and improving the file synchronization efficiency.

[0116] In one embodiment, as shown in FIG3 , a file collaborative management method for a cloud-edge collaborative scenario is provided. The method is applied to a central cloud, the central cloud is connected to a plurality of edge clouds respectively, the central cloud includes a central file system for managing files, and each edge cloud includes an edge file system for managing files. The method includes:

[0117] S302, obtaining files to be synchronized stored in the central file system and a central synchronization policy configured for the central file system.

[0118] Specifically, the central server obtains the to-be-synchronized files stored in the central file system and the central synchronization policy configured for the central file system.

[0119] Exemplarily, referring to the architectural diagram of the central cloud and edge cluster as shown in FIG4 , the central server obtains the files to be synchronized from the central file system (Minio, Ceph, FTP), and obtains the synchronization policy configured for the central file system from the database db.

[0120] Continuing to refer to FIG. 4, in the file collaborative management service, the function of automatically synchronizing files in heterogeneous file systems between the central cloud and the edge cloud (edge ​​clusters A, B, etc.) can be realized. In addition, the function of automatically synchronizing files in heterogeneous file systems between edge clouds can also be realized. The file collaborative management service supports synchronization policy configuration, one-to-many synchronization, multiple protocols to achieve file synchronization of heterogeneous file systems, and secure data encryption transmission.

[0121] Among them, the architecture in Figure 4 includes: WEB management module, synchronization policy module, security pipeline module, synchronization transmission module and database. The WEB management module is used to provide synchronization configuration strategy, and configure the file system synchronization between the central cloud and the edge cloud according to the cloud-edge file collaboration requirements. The synchronization policy module can be used to manage the policy rules of the synchronization channel, such as one-to-one synchronization, one-to-many synchronization, synchronization cycle and other strategies. The security pipeline module is used to implement the data encryption function between the cloud and the edge, and improve the security of cloud-edge data interaction. The synchronization transmission module is used to implement file transfer between various protocols (such as sftp, ftp, s3, https and other protocols) according to the synchronization policy information. The database is used to persist the synchronization strategy.

[0122] S304: Filter multiple target file systems that match the central synchronization policy in each edge file system.

[0123] In FIG. 4 , the SFTP service and Minio both represent edge file systems.

[0124] S306: Determine the expected synchronization duration of each target file system according to the system status information of each target file system.

[0125] S308, sorting the target file systems according to the expected synchronization durations to obtain a file system sequence.

[0126] S310, dividing each target file system into a plurality of system groups according to the position of each target file system in the file system sequence.

[0127] The number of target file systems included in each system group is the same as the number of file synchronization threads in the central cloud.

[0128] S312: For each target file system, determine the synchronization order of the system group to which the target file system belongs as the synchronization order of the target file system.

[0129] S314, obtaining multiple candidate encryption methods in the central cloud.

[0130] S316: Filter target encryption methods that match target file systems from among the candidate encryption methods.

[0131] S318, encrypting the file to be synchronized according to the target encryption method to obtain an encrypted file of the file to be synchronized.

[0132] S320: Determine, from each target file system, a first file system that has completed file synchronization and a second file system that has not completed file synchronization.

[0133] S322: Synchronize the files to be synchronized from the first file system to the second file system.

[0134] Exemplarily, referring to FIG. 4 , the central server may synchronize the files to be synchronized from the first file system (SFTP service in edge cluster A) to the second file system (Minio in edge cluster B).

[0135] In this embodiment, the central cloud is connected to multiple edge clouds respectively. The central cloud includes a central file system for managing files, and each edge cloud includes an edge file system for managing files. By obtaining the files to be synchronized stored in the central file system and the central synchronization policy configured for the central file system, multiple target file systems matching the central synchronization policy are screened in each edge file system to accurately determine the synchronization direction of the files to be synchronized, which is conducive to improving the efficiency of file synchronization. According to the system status information of each target file system, the synchronization order of each target file system is determined. The above method is used to determine the synchronization order. Based on each synchronization order, the files to be synchronized are synchronized to each target file system in sequence, so as to achieve orderly synchronization of the files to be synchronized and improve the efficiency of file synchronization. The method 1. All services of the method of the present invention support the deployment of containerized cloud native environment and adapt to the deployment requirements of the central cloud and edge cloud. 2. A unified WEB management page is provided, and multiple file synchronization policies can be flexibly configured. 3. Multiple protocols are supported to adapt to the synchronization between multiple file systems. 4. The synchronization channel adopts encryption technology to improve the security of file synchronization in cloud-edge scenarios. 5. Synchronization automatically adjusts the synchronization traffic according to the environmental bandwidth of the edge cloud.

[0136] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0137] Based on the same inventive concept, the embodiment of the present application also provides a file synchronization device for implementing the file synchronization method involved above. The implementation solution provided by the device to solve the problem is similar to the implementation solution recorded in the above method, so the specific limitations in one or more file synchronization device embodiments provided below can refer to the limitations on the file synchronization method above, and will not be repeated here.

[0138] In one embodiment, as shown in FIG5 , a file synchronization device is provided, including: an acquisition module, a screening module, an order determination module and a synchronization module, wherein:

[0139] An acquisition module is used to acquire files to be synchronized stored in the central file system and a central synchronization strategy configured for the central file system;

[0140] A screening module, used to screen multiple target file systems matching the central synchronization strategy in each edge file system;

[0141] An order determination module, used to determine the synchronization order of each target file system according to the system status information of each target file system;

[0142] The synchronization module is used to synchronize the files to be synchronized to each target file system in sequence based on each synchronization order.

[0143] In one of the embodiments, the synchronization module is also used to obtain multiple candidate encryption methods in the central cloud; among each candidate encryption method, a target encryption method that matches each target file system is screened; according to the target encryption method, the file to be synchronized is encrypted to obtain an encrypted file of the file to be synchronized; based on each synchronization order, the encrypted file is synchronized to each target file system in turn.

[0144] In one embodiment, the order determination module is further used to determine the expected synchronization duration of each target file system according to the system status information of each target file system; and determine the synchronization order of each target file system according to the expected synchronization duration.

[0145] In one of the embodiments, the order determination module is also used to sort each target file system according to each expected synchronization time to obtain a file system sequence; divide each target file system into multiple system groups according to the position of each target file system in the file system sequence; the number of target file systems included in each system group is the same as the number of file synchronization threads of the central cloud; for each target file system, the synchronization order of the system group to which the target file system belongs is determined as the synchronization order of the target file system.

[0146] In one embodiment, the order determination module is further used to determine, from each target file system, a first file system that has completed file synchronization and a second file system that has not completed file synchronization; and synchronize the files to be synchronized from the first file system to the second file system.

[0147] Each module in the above-mentioned file synchronization device can be implemented in whole or in part by software, hardware or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute the operations corresponding to each module above.

[0148] In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be shown in FIG6. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. The processor, the memory and the input / output interface are connected via a system bus, and the communication interface is connected to the system bus via the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store file data to be synchronized and central synchronization strategy data. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal via a network connection. When the computer program is executed by the processor, a file synchronization method is implemented.

[0149] Those skilled in the art will understand that the structure shown in FIG. 6 is merely a block diagram of a partial structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different arrangement of components.

[0150] In one embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above-mentioned method embodiments when executing the computer program.

[0151] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0152] In one embodiment, a computer program product is provided, including a computer program, which implements the steps in the above method embodiments when executed by a processor.

[0153] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but are not limited to this.

[0154] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0155] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A file synchronization method, characterized in that: Applied to a central cloud, the central cloud is connected to a plurality of edge clouds respectively, the central cloud includes a central file system for managing files, and each of the edge clouds includes an edge file system for managing files, the method includes: Acquire the files to be synchronized stored in the central file system and the central synchronization strategy configured for the central file system; In each of the edge file systems, a plurality of target file systems matching the central synchronization strategy are screened; Determining the synchronization order of each of the target file systems according to the system status information of each of the target file systems; Based on the synchronization orders, the files to be synchronized are synchronized to the target file systems in sequence.

2. The method according to claim 1, characterized in that: The step of sequentially synchronizing the to-be-synchronized files to the target file systems based on the synchronization orders includes: Obtaining multiple candidate encryption methods in the central cloud; Selecting a target encryption method that matches each target file system from among the candidate encryption methods; Encrypting the file to be synchronized according to the target encryption method to obtain an encrypted file of the file to be synchronized; Based on the synchronization orders, the encrypted files are synchronized to the target file systems in sequence.

3. The method according to claim 1, characterized in that Determining the synchronization order of each of the target file systems according to the system status information of each of the target file systems comprises: Determining the expected synchronization duration of each of the target file systems according to the system status information of each of the target file systems; According to the expected synchronization durations, a synchronization order of each of the target file systems is determined.

4. The method according to claim 3, characterized in that Determining the synchronization order of each of the target file systems according to the expected synchronization durations includes: The target file systems are sorted according to the expected synchronization durations to obtain a file system sequence; According to the positions of the target file systems in the file system sequence, the target file systems are divided into a plurality of system groups; the number of target file systems included in each system group is the same as the number of file synchronization threads of the central cloud; For each of the target file systems, the synchronization order of the system group to which the target file system belongs is determined as the synchronization order of the target file system.

5. The method according to claim 3, characterized in that: The step of sequentially synchronizing the to-be-synchronized files to the target file systems based on the synchronization orders includes: Determine, from each of the target file systems, a first file system for which file synchronization has been completed, and a second file system for which file synchronization has not been completed; Synchronize the to-be-synchronized file from the first file system to the second file system.

6. A file synchronization method, characterized in that: Applied to an edge cloud, the edge cloud is connected to a central cloud, the edge cloud includes an edge file system for managing files, and the central cloud includes multiple central file systems for managing files, the method includes: Acquire the to-be-synchronized files stored in the edge file system and the edge synchronization policy configured for the edge file system; In each of the central file systems, a plurality of target file systems matching the edge synchronization strategy are screened; Determining the synchronization order of each of the target file systems according to the system status information of each of the target file systems; Based on the synchronization orders, the files to be synchronized are synchronized to the target file systems in sequence.

7. A file synchronization device, characterized in that: The device comprises: An acquisition module, used to acquire files to be synchronized stored in the central file system, and a central synchronization strategy configured for the central file system; A screening module, used for screening a plurality of target file systems matching the central synchronization strategy in each of the edge file systems; An order determination module, used for determining the synchronization order of each of the target file systems according to the system status information of each of the target file systems; The synchronization module is used to synchronize the to-be-synchronized files to the target file systems in sequence based on the synchronization orders.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 5 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 5 are implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 5 are implemented.

Citation Information

Patent Citations

  • Wireless device synchronization systems and methods

    CN110050473A

  • Application resource deployment method and device, electronic equipment and medium

    CN112925651A

  • File synchronization method and device and storage medium

    CN117389977A