File synchronization methods, computing devices, storage media, and computer program products
By setting tags for the file synchronization channels of the gateway device and optimizing the correspondence between scanning threads and channels, the problem of decreased synchronization rate and repeated scanning when file synchronization channels reference the same directory was solved, achieving efficient and stable file synchronization.
Patent Information
- Application Number
- CN202411741287.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-11-29
- Publication Date
- 2025-10-28
- Estimated Expiration
- 2044-11-29
AI Technical Summary
In network gateway devices, when multiple file synchronization channels reference the same file server listening directory, it can easily lead to a decrease in synchronization rate, unstable synchronization, or even synchronization failure. Furthermore, repeated scanning generates multiple unnecessary file information tables, consuming device performance and failing to determine the accurate file information table.
By setting tags for file synchronization channels, a scanning thread is determined based on channels with the same tag, target file information is generated, the file information to be synchronized is determined, and the file is synchronized to the target server through the target file synchronization channel. This optimizes the correspondence between file synchronization channels and scanning threads, and reduces duplicate scanning and synchronization congestion.
It improves file synchronization efficiency and stability, reduces redundant scanning and the generation of unnecessary file information tables, optimizes resource utilization, and enhances the accuracy and consistency of file synchronization.
Smart Images

Figure CN119583571B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of computer technology, and more specifically to a file synchronization method, a computing device, a storage medium, and a computer program product. Background Technology
[0002] Network isolation devices are divided into internal and external terminals. The internal terminal is configured to monitor the file server monitoring directory in the internal network area, and the external terminal is configured to monitor the file server monitoring directory in the external network area. The file server monitoring directory is referenced through the file synchronization channel configured by the network isolation device. The network isolation device will create a scanning thread to scan the referenced file server monitoring directory in real time, obtain file information, and generate a file information table.
[0003] A single file server listening directory can only be referenced once by one file synchronization channel. This means all files under that listening directory can only be synchronized through a single channel. When a large number of files need to be synchronized, congestion can occur if there is only one synchronization channel, leading to decreased synchronization speed, unstable synchronization, or even synchronization failure. Network isolation devices create a scanning thread for each file synchronization channel. If multiple synchronization channels reference the same file server listening directory, duplicate scans will occur, generating multiple unnecessary file information tables. This not only consumes network isolation device resources but also prevents the device from determining which file information table to use for file synchronization. Summary of the Invention
[0004] To address the aforementioned shortcomings in the prior art, the purpose of this application is to provide a file synchronization method, a computing device, a storage medium, and a computer program product.
[0005] To achieve the above objectives, the first aspect of this application provides a file synchronization method, comprising:
[0006] A scanning thread is determined for each file synchronization channel with the same tag;
[0007] For each scanning thread, target file information is generated based on the file monitoring directory of the source server being monitored by the scanning thread.
[0008] The information of the files to be synchronized is determined based on the target file information and the initial file information, whereby the information of the files to be synchronized includes the file size and file type;
[0009] The target file synchronization channel and the file to be synchronized are determined based on the information of the file to be synchronized, wherein the target file synchronization channel is at least one of the file synchronization channels;
[0010] The files to be synchronized are synchronized to the target server via the target file synchronization channel.
[0011] In this embodiment of the application, before the step of creating a scanning thread based on the tag of the file synchronization channel, the method further includes:
[0012] Configure tags and synchronization strategies for all file synchronization channels based on preset file synchronization requirements. The synchronization strategies include the allowed range of file size and the allowed set of file types.
[0013] In this embodiment of the application, determining the target file synchronization channel among all file synchronization channels corresponding to the scanning thread based on the file information to be synchronized includes:
[0014] Get the file size and / or file type of all files to be synchronized from the file information to be synchronized;
[0015] Based on file size and / or file type, determine the target file synchronization channel among all file synchronization channels corresponding to the scanning thread. The allowed range of file size for the target file synchronization channel includes any one of all file sizes and / or the allowed set of file types includes any one of all file types.
[0016] In this embodiment of the application, the file to be synchronized is synchronized to the target server through the target file synchronization channel, including:
[0017] The target files to be synchronized for each target file synchronization channel are determined based on the allowed range of file size and / or the allowed set of file types for the target file synchronization channel.
[0018] The corresponding target files to be synchronized are synchronized to the target server through each target file synchronization channel.
[0019] In this embodiment of the application, a scanning thread is determined based on the file synchronization channel with the same tag, including:
[0020] Determine whether a target scanning thread exists that corresponds to the target tag, where the target tag is the tag of the file synchronization channel with the same tag;
[0021] In the absence of a target scanning thread, a scanning thread is determined for each file synchronization channel with the same label.
[0022] If a target scanning thread exists, associate the file synchronization channel with the same tag with the target scanning thread.
[0023] In this embodiment of the application, the initial file information includes historical file information of the file monitoring directory. Based on the target file information and the initial file information, the file information to be synchronized is determined, including:
[0024] Determine if there are any discrepancies between the historical file information of the file monitoring directory and the target file information;
[0025] In the presence of discrepancies, the information of the files to be synchronized is determined based on these discrepancies.
[0026] In this embodiment of the application, the initial file information includes target server file information generated based on the target server's file monitoring directory. The file information to be synchronized is determined based on the target file information and the initial file information, including:
[0027] Determine if there are any discrepancies between the target server file information and the target file information;
[0028] In the presence of discrepancies, the information of the files to be synchronized is determined based on these discrepancies.
[0029] A second aspect of this application provides a file synchronization device, comprising:
[0030] The thread determination module is used to determine a scanning thread based on file synchronization channels with the same label.
[0031] The file information generation module is used to generate target file information for each scanning thread, based on the file monitoring directory of the source server monitored by the scanning thread.
[0032] The module for determining the information of files to be synchronized is used to determine the information of files to be synchronized based on the target file information and the initial file information. The information of files to be synchronized includes the file size and the file type.
[0033] The channel determination module is used to determine the target file synchronization channel and the file to be synchronized for the scanning thread based on the file information to be synchronized, wherein the target file synchronization channel is at least one of the file synchronization channels;
[0034] The file synchronization module is used to synchronize files to be synchronized to the target server through the target file synchronization channel.
[0035] A third aspect of this application provides a computing device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the file synchronization method as described in the above embodiments.
[0036] A fourth aspect of this application provides a machine-readable storage medium on which a program or instructions are stored, and when the program or instructions are executed by a processor, a file synchronization method as described in the above embodiments is implemented.
[0037] The fifth aspect of this application provides a computer program product, including a computer program that, when executed by a processor, implements the file synchronization method as described in the above embodiments.
[0038] The above technical solution identifies a scanning thread for each file synchronization channel with the same label. For each scanning thread, it monitors the file listening directory on the source server and generates target file information. Multiple file synchronization channels with the same label can scan the file listening directory using the same scanning thread, reducing duplicate scans and the generation of redundant file information tables. The file information to be synchronized is determined based on the target file information and initial file information, including file size and file type. The target file synchronization channel corresponding to the scanning thread and the file to be synchronized are then determined based on this information. The file to be synchronized is then synchronized to the target server through the target file synchronization channel. By using multiple file synchronization channels with the same label to synchronize all files under the same file listening directory, synchronization congestion is reduced, and file synchronization efficiency and stability are improved.
[0039] Other features and advantages of the embodiments of the present invention will be described in detail in the following detailed description section. Attached Figure Description
[0040] The accompanying drawings are provided to further illustrate embodiments of the present invention and form part of the specification. They are used together with the following detailed description to explain the embodiments of the present invention, but do not constitute a limitation thereof. In the drawings:
[0041] Figure 1 The illustration shows a schematic flowchart of a file synchronization method according to an embodiment of this application;
[0042] Figure 2 The illustration shows a schematic flowchart of a file synchronization method according to another embodiment of this application. Detailed Implementation
[0043] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are only for illustration and explanation of the embodiments of this application and are not intended to limit the embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the scope of protection of this application.
[0044] It should be noted that the acquisition, transmission, storage, use, and processing of data in the technical solution of this application all comply with the relevant provisions of national laws and regulations. In the embodiments of this application, certain existing industry solutions such as software, components, and models may be mentioned. These should be considered exemplary, intended only to illustrate the feasibility of implementing the technical solution of this application, and do not imply that the applicant has already used or necessarily used such solutions.
[0045] It should be noted that if the embodiments of this application involve directional indicators (such as up, down, left, right, front, back, etc.), the directional indicators are only used to explain the relative positional relationship and movement of the components in a certain specific posture (as shown in the figure). If the specific posture changes, the directional indicators will also change accordingly.
[0046] Furthermore, if the embodiments of this application involve descriptions such as "first" or "second," these descriptions are for descriptive purposes only and should not be construed as indicating or implying their relative importance or implicitly specifying the number of technical features indicated. Therefore, features defined with "first" or "second" may explicitly or implicitly include at least one of those features. Additionally, the technical solutions of the various embodiments can be combined with each other, but this must be based on the ability of those skilled in the art to implement them. If the combination of technical solutions is contradictory or impossible to implement, it should be considered that such a combination of technical solutions does not exist and is not within the scope of protection claimed in this application.
[0047] Figure 1 The illustration shows a schematic flowchart of a file synchronization method according to an embodiment of this application. Figure 1 As shown in the figure, this application provides a file synchronization method, which may include the following steps:
[0048] Step 100: Determine a scanning thread for each file synchronization channel with the same tag;
[0049] It should be noted that the file synchronization method provided in this embodiment is mainly applied to the file synchronization function of the gateway device. In one embodiment, the gateway device can adopt a dual-host architecture, consisting of two isolated hosts and an independent hardware switching chip. Data exchanged between the internal and external networks must be transmitted through the gateway's switching chip to achieve physical isolation. When internal network data needs to be transmitted to the external network, the gateway initiates a non-TCP (Transmission Control Protocol) data connection request to the internal network server and issues a write command, closing the gateway's write control switch, stripping all protocols, and writing the original data to the storage medium. Once the data is completely written to the gateway's storage medium, the control switch between the gateway and the internal network server is immediately disconnected. Then, the control switch between the gateway and the external network server is closed, and the gateway initiates a non-TCP data connection request to the external network. Upon receiving the request, the external network server issues a read command to transmit the data from the gateway's storage medium to the external network server. After receiving the data, the external network server repackages the received data according to TCP protocol requirements and delivers it to the application system. The file server listening directories corresponding to the internal and external network servers are monitored by the gateway device. In traditional file synchronization methods, all files under a file monitoring directory can only be synchronized through a single file synchronization channel. The same file monitoring directory can only be referenced once by one file synchronization channel. When a large number of files need to be synchronized, congestion may occur if there is only one file synchronization channel, leading to a decrease in synchronization speed, unstable synchronization, or even synchronization failure. Furthermore, the network gateway device creates a scanning thread for each file synchronization channel. If multiple file synchronization channels reference the same file monitoring directory, duplicate scans will occur, generating multiple unnecessary file information tables.
[0050] To address the aforementioned issues, this embodiment modifies the correspondence between scanning threads and file synchronization channels. By assigning tags to file synchronization channels, the same scanning thread is assigned to channels with the same tag, reducing redundant scans and enabling synchronization of all files under the same file monitoring directory through multiple file synchronization channels with the same tag. The file monitoring directory is used to monitor file changes in a specified directory in real time, including file creation, modification, and deletion events. The file synchronization channel refers to the secure and reliable transmission of files between different network areas using the gateway device. The scanning thread scans the file monitoring directory referenced by the gateway device in real time to obtain file information, including file type, file size, file name, file creation date, and file modification date. When changes occur in the file monitoring directory, the scanning thread can immediately capture these changes.
[0051] refer to Figure 2It should be noted that, in one embodiment, before the step of creating a scanning thread based on the tag of the file synchronization channel, the method further includes:
[0052] Configure tags and synchronization strategies for all file synchronization channels based on preset file synchronization requirements. The synchronization strategies include the allowed range of file size and the allowed set of file types.
[0053] Pre-defined file synchronization requirements refer to a series of specific needs and conditions determined before designing and implementing a file synchronization solution, based on actual business needs and file management requirements. These requirements typically cover all aspects of file synchronization to ensure accurate, efficient, and secure file synchronization across different devices, systems, or locations. Specifically, pre-defined file synchronization requirements may include the synchronization scope, which specifies the files and folders to be synchronized, as well as their source and destination locations; or whether to synchronize the entire directory structure or only specific file types or subdirectories. Pre-defined file synchronization requirements may also include the synchronization direction, which determines whether it is unidirectional synchronization, such as from server to client, or bidirectional synchronization, such as synchronization between server and client. Pre-defined file synchronization requirements may also include the synchronization frequency, which determines whether it is real-time or scheduled synchronization. Furthermore, pre-defined file synchronization requirements may include requirements such as file size limits, file type filtering, security requirements, conflict handling, and network requirements.
[0054] Different file synchronization tasks have different synchronization requirements. In this embodiment, tags and synchronization strategies are configured for the file synchronization channels of the gateway device based on actual preset file synchronization needs. Tags are used to identify and distinguish file synchronization channels, and can be configured based on the type, purpose, or importance of the file synchronization channel. In this embodiment, using tags to represent the purpose of the file synchronization channel is taken as an example. Different tags are used to distinguish file synchronization channels performing different file synchronization tasks; that is, file synchronization channels monitoring the same file monitoring directory are assigned the same tag. Tags are unique and explicit, and can include numbers, letters, characters, or combinations of various data formats. The configuration of synchronization strategies is used to restrict the basic strategies of the file synchronization channel when performing file synchronization tasks, including the allowed file size range and the allowed set of file types. The allowed file size range can set a limit on the size of a single file or the total file size limit for the entire file synchronization channel; the allowed set of file types can be used to identify and filter file types based on file extensions, MIME types, etc. In one embodiment, the synchronization strategy can also include synchronization methods, such as real-time synchronization, scheduled synchronization, and manual synchronization; the synchronization strategy can also include synchronization time, synchronization method, and conflict resolution strategies. These strategies should be configured according to actual needs to ensure the accuracy and efficiency of file synchronization.
[0055] After initial configuration of the file synchronization channels, in practical applications, a scanning thread is assigned to each file synchronization channel with the same tag. A many-to-one association between the file synchronization channels and scanning threads is achieved by setting a one-to-one association configuration between tags and scanning threads. Multiple file synchronization channels under the same tag can scan the file monitoring directory based on the same scanning thread, reducing duplicate scans. It is understandable that the correspondence between file synchronization channels and scanning threads can be dynamically adjusted based on tag updates. The number and configuration of file synchronization channels are dynamically adjusted according to file synchronization needs and load conditions. In one embodiment, a monitoring mechanism can be introduced to evaluate the load and performance of file synchronization channels in real time. When congestion or performance degradation is detected in a file synchronization channel, the system automatically adjusts the configuration of other file synchronization channels, such as increasing the number of file synchronization channels corresponding to that tag or adjusting the synchronization strategy to distribute the load.
[0056] Step 200: For each scanning thread, generate target file information based on the file monitoring directory of the source server monitored by the scanning thread;
[0057] It should be noted that each scanning thread continuously monitors the file monitoring directory on the source server and generates target file information for that monitoring directory. This target file information typically includes the file's name, path, size, type, timestamp, etc.
[0058] Step 300: Determine the file information to be synchronized based on the target file information and the initial file information, wherein the file information to be synchronized includes the file size and file type;
[0059] It should be noted that by comparing the target file information with the initial file information, the system can determine which files need to be synchronized, i.e., the file information to be synchronized. This file information can include the file's name, size, type, etc., for further processing.
[0060] Specifically, in one embodiment, the initial file information includes historical file information of the file monitoring directory, and the file information to be synchronized is determined based on the target file information and the initial file information, including:
[0061] Determine if there are any discrepancies between the historical file information of the file monitoring directory and the target file information;
[0062] In the presence of discrepancies, the information of the files to be synchronized is determined based on these discrepancies.
[0063] It should be noted that the historical file information in the file monitoring directory refers to the historical file information of the source server's file monitoring directory. This historical file information includes file information obtained from the last scan of the source server's file monitoring directory by the scanning thread, or file information obtained from scans at even earlier historical times. The specific scan time for historical file information can be determined based on actual application requirements. The target file information is the file information obtained from the current scan of the source server's file monitoring directory by the scanning thread. By identifying whether there are differences between the historical file information and the target file information, it can be determined whether the source server's file monitoring directory has been updated. If differences exist, the file information corresponding to the differences is used as the file information to be synchronized. In subsequent file synchronization jobs, updates from the source server are synchronized to the target server based on this file information to be synchronized. It can be understood that the source server can be an internal network server or an external network server, corresponding to the target server being an external network server or an internal network server, respectively. File synchronization achieves the synchronous update of files on the source server and the target server.
[0064] Specifically, in one embodiment, the initial file information includes target server file information generated based on the target server's file monitoring directory, and the file information to be synchronized is determined based on the target file information and the initial file information, including:
[0065] Determine if there are any discrepancies between the target server file information and the target file information;
[0066] In the presence of discrepancies, the information of the files to be synchronized is determined based on these discrepancies.
[0067] It should be noted that the target server file information refers to the target file information in the target server's file monitoring directory. This includes the file information obtained by the scanning thread during the current scan of the target server's file monitoring directory, and the file information obtained by the scanning thread during the current scan of the source server's file monitoring directory. By identifying differences between the target server file information and the target file information, it can be determined whether there are differences between the source server's file monitoring directory and the target server's file monitoring directory. If differences exist, the file information corresponding to the differing portion is used as the file information to be synchronized. This allows for the unification of files between the source and target servers during subsequent file synchronization operations. It can be understood that the source server can be an internal network server or an external network server, corresponding to the target server being either an external network server or an internal network server, respectively. File synchronization achieves file unification between the source and target servers.
[0068] Step 400: Determine the target file synchronization channel and the file to be synchronized for the scanning thread based on the file information to be synchronized, wherein the target file synchronization channel is at least one of the file synchronization channels;
[0069] Step 500: Synchronize the file to be synchronized to the target server through the target file synchronization channel.
[0070] It's important to note that the most suitable file synchronization channel should be selected based on the information of the files to be synchronized, such as file size and file type. Understandably, file synchronization channels can be optimized based on factors such as different network protocols, transfer speeds, and security. After determining the target file synchronization channel, the files that need to be synchronized through that channel are selected; these are the files to be synchronized. The selected target file synchronization channel is then used to transfer the files to be synchronized from the source server to the target server. During the transfer, error checking and data verification steps can be performed to ensure the integrity and accuracy of the files. After synchronization is complete, the file status on the target server will be consistent with the file status on the source server, or specific modifications can be made as needed.
[0071] In this embodiment, a scanning thread is determined for each file synchronization channel with the same label. For each scanning thread, target file information is generated by monitoring the file monitoring directory on the source server. Multiple file synchronization channels with the same label can scan the file monitoring directory based on the same scanning thread, reducing duplicate scans and the generation of redundant file information tables. The file information to be synchronized is determined based on the target file information and initial file information, including file size and file type. The target file synchronization channel corresponding to the scanning thread and the file to be synchronized are determined based on the file information to be synchronized. The file to be synchronized is synchronized to the target server through the target file synchronization channel. By synchronizing all files under the same file monitoring directory through multiple file synchronization channels with the same label, synchronization congestion is reduced, and file synchronization efficiency and stability are improved.
[0072] In one embodiment, determining the target file synchronization channel among all file synchronization channels corresponding to the scanning thread based on the file information to be synchronized includes:
[0073] Get the file size and / or file type of all files to be synchronized from the file information to be synchronized;
[0074] Based on file size and / or file type, determine the target file synchronization channel among all file synchronization channels corresponding to the scanning thread. The allowed range of file size for the target file synchronization channel includes any one of all file sizes and / or the allowed set of file types includes any one of all file types.
[0075] In this embodiment, it should be noted that the information of the files to be synchronized includes the filename, file size, file type (e.g., text, image, video), modification time, and other key attributes of each file. The file size and / or file type of the files to be synchronized are analyzed. This determines which files require more transmission resources; for example, large files may require more bandwidth and longer transmission time, and which files may be suitable for specific transmission channels. For example, some file synchronization channels may be specifically designed for transmitting large files or specific types of files. The characteristics of each file synchronization channel are determined, including the file size range supported by each channel, the allowed set of file types, transmission speed, and available bandwidth. This information is usually set during system configuration and may be dynamically adjusted based on network conditions, server performance, and other factors. Based on the file size and / or file type of the files to be synchronized, and the characteristics of the file synchronization channels, the files to be synchronized are assigned to the most suitable file synchronization channel. For example, if a large file needs to be synchronized, a file synchronization channel that supports large file transmission and has sufficient bandwidth is selected as the target file synchronization channel for synchronizing that large file. The target file synchronization channel is the file synchronization channel selected for transmitting the specific file to be synchronized. Based on the attributes of each file to be synchronized and the characteristics of each file synchronization channel, the target file synchronization channels corresponding to all files to be synchronized are determined. Once the target file synchronization channels are determined, the file synchronization operation begins, transferring the files to be synchronized from the source server to the target server, while ensuring the integrity, order, and reliability of the data.
[0076] Understandably, when determining the target file synchronization channel, some special cases can be considered, such as file priority and transmission deadline. Furthermore, intelligent optimization can be performed based on historical data and network conditions; for example, prioritizing file synchronization channels that have performed well in the past, or avoiding file synchronization channels currently experiencing network congestion.
[0077] In this embodiment, by determining the target file synchronization channel among all file synchronization channels corresponding to the scanning thread, the intelligent allocation of file synchronization resources is optimized, thereby improving resource utilization efficiency.
[0078] In one embodiment, synchronizing the file to be synchronized to the target server via the target file synchronization channel includes:
[0079] The target files to be synchronized for each target file synchronization channel are determined based on the allowed range of file size and / or the allowed set of file types for the target file synchronization channel.
[0080] The corresponding target files to be synchronized are synchronized to the target server through each target file synchronization channel.
[0081] In this embodiment, it should be noted that the target files to be synchronized for each target file synchronization channel are selected and determined based on the allowed file size range and / or allowed set of file types for that target file synchronization channel. This means that for each target file synchronization channel, the file size and file type of the files to be synchronized will be further checked to ensure that they meet the requirements of that target file synchronization channel. After determining the target files to be synchronized, synchronization operations are prepared for each target file. This may include setting file metadata, such as permissions and timestamps; preparing necessary encryption or compression operations; determining the file transmission order, considering priority or dependencies if necessary. The corresponding target files to be synchronized are synchronized to the target server through each target file synchronization channel. The transmission progress and error conditions are monitored to ensure file integrity and reliability. If errors are encountered during the synchronization process, error handling attempts can be made, including retrying transmission, logging error information for subsequent analysis, or taking other remedial measures according to the configuration. After the file synchronization is completed, the resources used during the synchronization process are cleaned up and released.
[0082] In this embodiment, by further matching the target file synchronization channel and the target file to be synchronized, the effectiveness and efficiency of file synchronization are improved, and synchronization congestion is reduced.
[0083] refer to Figure 2 In one embodiment, a scanning thread is determined based on file synchronization channels with the same tag, including:
[0084] Determine whether a target scanning thread exists that corresponds to the target tag, where the target tag is the tag of the file synchronization channel with the same tag;
[0085] In the absence of a target scanning thread, a scanning thread is determined for each file synchronization channel with the same label.
[0086] If a target scanning thread exists, associate the file synchronization channel with the same tag with the target scanning thread.
[0087] In this embodiment, it should be noted that a unique tag is assigned to each file synchronization channel during the pre-configuration phase. These tags can be defined based on the characteristics of the file synchronization channel, such as file type, file size range, target server, etc. The target tag is the tag of file synchronization channels with the same tag. After obtaining the file synchronization channel tags, for file synchronization channels with the same tag, it is determined whether a target scanning thread corresponding to the target tag exists. Specifically, this can be achieved by checking an internal data structure, such as a hash table or dictionary, which maps tags to corresponding scanning threads.
[0088] If no target scan thread exists corresponding to the target tag, a new scan thread will need to be created based on the file synchronization channel with the same tag. This includes allocating necessary resources, such as memory and processor time, to the scan thread and initializing it to handle the file synchronization channel associated with the tag. If a target scan thread exists, then file synchronization channels with the same tag will be associated with the target scan thread. This means that these file synchronization channels will be assigned to the target scan thread for processing, thus ensuring the continuity and consistency of synchronization operations. After the scan thread is created and associated with the file synchronization channel, its status can be monitored in real time, such as busy, idle, or error, and the number and priority of scan threads can be dynamically adjusted as needed.
[0089] In one embodiment, detailed information related to the scanning thread and file synchronization channel can also be recorded, including the creation time and destruction time of the scanning thread, the number of files processed, and the synchronization success rate. This information can be used for subsequent performance analysis, troubleshooting, and report generation.
[0090] In this embodiment, creating a new scanning thread or associating it with an existing scanning thread by using a file synchronization channel with the same tag can optimize resource usage and improve synchronization efficiency.
[0091] This invention provides a file synchronization device, comprising:
[0092] The thread determination module is used to determine a scanning thread based on file synchronization channels with the same label.
[0093] The file information generation module is used to generate target file information for each scanning thread, based on the file monitoring directory of the source server monitored by the scanning thread.
[0094] The module for determining the information of files to be synchronized is used to determine the information of files to be synchronized based on the target file information and the initial file information. The information of files to be synchronized includes the file size and the file type.
[0095] The channel determination module is used to determine the target file synchronization channel and the file to be synchronized for the scanning thread based on the file information to be synchronized, wherein the target file synchronization channel is at least one of the file synchronization channels;
[0096] The file synchronization module is used to synchronize files to be synchronized to the target server through the target file synchronization channel.
[0097] The file synchronization device includes a processor and a memory. The thread determination module, file information generation module, file information determination module, channel determination module, and file synchronization module are all stored as program units in the memory. The processor executes the program units stored in the memory to implement the corresponding functions.
[0098] The processor contains a kernel, which retrieves the corresponding program units from memory. One or more kernels can be configured, and file synchronization can be achieved by adjusting kernel parameters.
[0099] The memory may include non-permanent memory in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM, and the memory includes at least one memory chip.
[0100] The file synchronization device provided in this application embodiment can respectively achieve Figure 1 The various processes of the file synchronization method in the method embodiment can achieve the same technical effect, and will not be described again here to avoid repetition.
[0101] This invention provides a computing device, including a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the file synchronization method as described in the above embodiments.
[0102] This invention provides a machine-readable storage medium on which a program or instruction is stored, and when the program or instruction is executed by a processor, it implements a file synchronization method as described in the above embodiments.
[0103] This invention provides a computer program product, including a computer program that, when executed by a processor, implements the file synchronization method described in the above embodiments.
[0104] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.
[0105] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.
[0106] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.
[0107] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.
[0108] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.
[0109] Memory may include non-persistent memory in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, like read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.
[0110] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.
[0111] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.
[0112] The above are merely embodiments of this application and are not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application.
Claims
1. A file synchronization method, characterized in that, include: A scanning thread is determined for each file synchronization channel with the same tag; For each scanning thread, target file information is generated based on the file monitoring directory of the source server monitored by the scanning thread; Based on the target file information and the initial file information, the file information to be synchronized is determined, wherein the file information to be synchronized includes the file size and file type; Based on the information of the file to be synchronized, the target file synchronization channel and the file to be synchronized corresponding to the scanning thread are determined, wherein the target file synchronization channel is at least one of the file synchronization channels; The file to be synchronized is synchronized to the target server through the target file synchronization channel; Before the step of determining a scanning thread for each file synchronization channel with the same tag, the method further includes: Based on preset file synchronization requirements, configure tags and synchronization strategies for all file synchronization channels, wherein the synchronization strategy includes a allowed range of file size and a allowed set of file types; Determining the target file synchronization channel corresponding to the scanning thread based on the file information to be synchronized includes: Obtain the file size and / or file type of all files to be synchronized from the file information to be synchronized; Based on the file size and / or the file type, a target file synchronization channel is determined among all file synchronization channels corresponding to the scanning thread, wherein the allowed range of file size corresponding to the target file synchronization channel includes any one of the file sizes and / or the allowed set of file types includes any one of the file types.
2. The file synchronization method according to claim 1, characterized in that, The step of synchronizing the file to be synchronized to the target server through the target file synchronization channel includes: Based on the allowed file size range and / or allowed file type set of the target file synchronization channel, the target file to be synchronized for each target file synchronization channel is determined respectively. The corresponding target files to be synchronized are synchronized to the target server through each of the target file synchronization channels.
3. The file synchronization method according to claim 1, characterized in that, The method of determining a scanning thread for each file synchronization channel with the same tag includes: Determine whether a target scanning thread exists that corresponds to a target tag, wherein the target tag is a tag of a file synchronization channel with the same tag; In the absence of the target scanning thread, a scanning thread is determined based on the file synchronization channels with the same label; If the target scanning thread exists, associate the file synchronization channel with the same tag with the target scanning thread.
4. The file synchronization method according to claim 1, characterized in that, The initial file information includes historical file information of the file monitoring directory, and the step of determining the file information to be synchronized based on the target file information and the initial file information includes: Determine whether there are any differences between the historical file information of the file monitoring directory and the target file information; In the presence of the aforementioned differences, the file information to be synchronized is determined based on these differences.
5. The file synchronization method according to claim 1, characterized in that, The initial file information includes target server file information generated based on the target server's file monitoring directory. The step of determining the file information to be synchronized based on the target file information and the initial file information includes: Determine whether there is a difference between the target server file information and the target file information; In the presence of the aforementioned differences, the file information to be synchronized is determined based on these differences.
6. A computing device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the file synchronization method according to any one of claims 1 to 5.
7. A machine-readable storage medium on which a program or instructions are stored, characterized in that, When the program or the instructions are executed by the processor, the file synchronization method according to any one of claims 1 to 5 is implemented.
8. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the file synchronization method according to any one of claims 1 to 5.
Citation Information
Patent Citations
Data transmission method, proxy server, storage medium and electronic device
CN113965560A
Data synchronization method and device, electronic equipment and storage medium
CN116028577A