File synchronization method and device suitable for various hardware environments, medium and product

Abstract

The invention discloses a file synchronization method and device suitable for various hardware environments, a medium and a product. The method comprises the following steps: pre-analyzing acquired information of a sending end, and determining an initial core thread count of the sending end and a single sending amount of a sending queue; dynamically adjusting the initial core thread count of the sending end based on the total data volume of the to-be-synchronized file and a preset dynamic adjustment strategy to obtain the target core thread count of the sending end, then creating the same number of Socket sending threads, and sending the to-be-synchronized file to the receiving end based on the Socket sending threads and the single sending volume; acquiring a single receiving quantity threshold value of the receiving end based on the acquired information of the receiving end; and calling a corresponding writing strategy to write the to-be-synchronized file into a disk corresponding to the receiving end based on the single receiving quantity threshold of the receiving end and the actual sending quantity of the to-be-synchronized file. The method can automatically adapt to different hardware environments and load conditions, realizes automatic and flexible transmission adjustment, and is good in compatibility, efficient and stable.

Description

Technical Field

[0001] The present disclosure relates to the technical field of data synchronization, and in particular to a file synchronization method, device, medium, and product that are adaptable to a variety of hardware environments. Background Art

[0002] With the rapid development of the information age, the rise of big data and artificial intelligence (AI), files, as carriers of information or data, play an especially important role in the information age. Nowadays, all walks of life are inseparable from data transmission. There are many types of data transmission servers and complex transmission hardware. The most obvious manifestation is in the implementation of persistent data. In the field of data transmission and persistence technology, the hardware compatibility and data consistency problems existing in the existing technology are particularly prominent. The differences between different types of servers, storage devices, and network hardware often lead to compatibility problems in actual applications, which often cause bottlenecks when synchronizing different batches of data files, making it impossible to achieve efficient and fast synchronization, and even causing damage or loss during the transmission process. At the same time, this diverse hardware environment not only increases the complexity of system configuration and maintenance, but also causes potential compatibility problems when upgrading or replacing hardware. Summary of the invention

[0003] In view of this, the embodiments of the present disclosure provide a file synchronization method, device, medium, and product that are adaptable to a variety of hardware environments, which can solve the problems existing in the prior art such as poor file synchronization transmission efficiency, inability to meet the needs of file transfers of different batches, and inability to simultaneously adapt to efficient data synchronization in multiple hardware environments.

[0004] In a first aspect, an embodiment of the present disclosure provides a file synchronization method adapted to a variety of hardware environments, including:

[0005] Acquire the sender information, wherein the sender information includes the sender disk type, the sender disk storage information, the sender memory information, and the sender CPU information;

[0006] Performing a preliminary analysis based on the sending end information to determine the initial number of core threads of the sending end and the single sending volume of the sending queue;

[0007] Based on the total data volume of the files to be synchronized and a preset dynamic adjustment strategy, dynamically adjust the initial number of core threads of the sending end to obtain the target number of core threads of the sending end;

[0008] Creating the same number of Socket sending threads based on the target core thread number of the sending end, and sending the to-be-synchronized file to the receiving end based on the Socket sending threads and the single sending amount;

[0009] Acquire receiving end information, wherein the receiving end information includes receiving end disk type, receiving end disk storage information, receiving end core thread number, and receiving end CPU information;

[0010] Acquire a single reception quantity threshold of the receiving end based on the receiving end information;

[0011] Based on the single receiving quantity threshold of the receiving end and the actual sending quantity of the to-be-synchronized files, the corresponding writing strategy is called to write the to-be-synchronized files into the disk corresponding to the receiving end.

[0012] Optionally, the performing a pre-analysis based on the sending end information to determine the initial number of core threads of the sending end and the single sending amount of the sending queue includes:

[0013] Based on the sending end disk type, a core thread number setting strategy of a corresponding sending end thread pool is called;

[0014] Setting the initial number of core threads of the sending end based on the core thread number setting strategy;

[0015] The single sending amount of the sending queue is determined based on the sending end memory information and the sending end disk storage information.

[0016] Optionally, the core thread number setting strategy of calling the corresponding sender thread pool based on the sender disk type includes:

[0017] If the sending end disk type is the first type, obtaining the initial number of core threads of the sending end thread pool based on the first strategy;

[0018] If the sending end disk type is the second type, obtaining the initial number of core threads of the sending end thread pool based on the second strategy;

[0019] If the sending end disk type is the third type, the initial number of core threads of the sending end thread pool is obtained based on the third strategy.

[0020] Optionally, the first type is a mechanical disk;

[0021] The obtaining of the initial number of core threads of the sending end thread pool based on the first strategy includes:

[0022] Acquire the CPU size of the sending end based on the sending end CPU information;

[0023] Determine the initial number of core threads of the sending end thread pool based on the sending end CPU size;

[0024] The initial number of core threads is N0: N0=n0, where n0 is the CPU size of the sending end.

[0025] Optionally, the second type is a solid state drive;

[0026] The obtaining of the initial number of core threads of the sending end thread pool based on the second strategy includes:

[0027] Acquire the CPU size of the sending end based on the sending end CPU information;

[0028] Determine the initial number of core threads of the sending end thread pool based on the sending end CPU size;

[0029] The initial number of core threads is N0: N0=n0×n1, where n0 is the CPU size of the sending end, and 4≤n1≤16.

[0030] Optionally, the third type is a RAID disk;

[0031] The obtaining of the initial number of core threads of the sending end thread pool based on the third strategy includes:

[0032] Acquire the CPU size of the sending end based on the sending end CPU information;

[0033] Determine the initial number of core threads of the sending end thread pool based on the sending end CPU size;

[0034] The initial number of core threads is N0: N0=n0×n2, where n0 is the CPU size of the sending end, and 2≤n2≤8.

[0035] Optionally, the single transmission information is Q1: Q1 = F2 ÷ F1 × F0;

[0036] Among them, F2 is the disk storage information of the sending end, F1 is the memory information of the sending end, and F0 is the CPU information of the sending end.

[0037] Optionally, dynamically adjusting the initial number of core threads of the sending end based on the total data volume of the files to be synchronized and a preset dynamic adjustment strategy to obtain the target number of core threads of the sending end includes:

[0038] Determine a data synchronization mode based on the total data volume of the files to be synchronized.

[0039] Based on the data synchronization mode and the sending end disk type, a corresponding dynamic adjustment strategy for the number of core threads on the sending end is called;

[0040] In response to the dynamic adjustment strategy of the number of core threads at the sending end, the initial number of core threads at the sending end is dynamically adjusted to obtain the target number of core threads at the sending end.

[0041] Optionally, determining the data synchronization mode based on the acquired total data volume of the files to be synchronized includes:

[0042] If the total data volume of the files to be synchronized meets the first range, the data synchronization mode is determined to be the high-efficiency mode;

[0043] If the total data volume of the files to be synchronized meets the second range, determining that the data synchronization mode is a stable mode;

[0044] If the total data volume of the files to be synchronized meets the third range, the data synchronization mode is determined to be the general mode;

[0045] If the total data volume of the files to be synchronized meets the fourth range, determining that the data synchronization mode is a safe mode;

[0046] The first range is M1, 1KB≤M1≤2MB;

[0047] The second range is M2, 2MB<M2≤50MB;

[0048] The third range is M3, 50MB<M3≤200MB

[0049] The fourth range is M4, 200MB<M4.

[0050] Optionally, the dynamically adjusting strategy of the number of core threads on the sending end corresponding to the calling of the data synchronization mode and the sending end disk type includes:

[0051] When the data synchronization mode is the high-efficiency mode and the sending end disk type is a mechanical disk, calling the corresponding sending end core thread number dynamic adjustment strategy as the first expansion strategy;

[0052] When the data synchronization mode is the high-efficiency mode and the sending end disk type is a solid-state hard disk, calling the corresponding sending end core thread number dynamic adjustment strategy as the second expansion strategy;

[0053] When the data synchronization mode is the high-efficiency mode and the sending end disk type is a RAID disk, calling the corresponding sending end core thread number dynamic adjustment strategy as the third expansion strategy;

[0054] When the data synchronization mode is the safe mode and the sending end disk type is a mechanical disk, calling the corresponding sending end core thread number dynamic adjustment strategy as the first reduction strategy;

[0055] When the data synchronization mode is the safe mode and the sending end disk type is a solid state drive, calling the corresponding sending end core thread number dynamic adjustment strategy as the second reduction strategy;

[0056] When the data synchronization mode is the safe mode and the sending end disk type is a RAID disk, the corresponding sending end core thread number dynamic adjustment strategy is called as the third reduction strategy.

[0057] Optionally, dynamically adjusting the initial number of core threads of the sending end in response to the dynamic adjustment strategy of the number of core threads of the sending end to obtain the target number of core threads of the sending end includes:

[0058] When the data synchronization mode is the high-efficiency mode and the sending end disk type is a mechanical disk, the initial number of core threads of the sending end is increased by H1 times, and the increased initial number of core threads of the sending end is used as the target number of core threads of the sending end; 2≤H1≤3;

[0059] When the data synchronization mode is the high-efficiency mode and the sending end disk type is a solid-state hard disk, the initial number of core threads of the sending end is increased by H2 times, and the increased initial number of core threads of the sending end is used as the target number of core threads of the sending end; 8≤H2≤9;

[0060] When the data synchronization mode is the high-efficiency mode and the sending end disk type is a RAID disk, the initial number of core threads of the sending end is increased by H3 times, and the increased initial number of core threads of the sending end is used as the target number of core threads of the sending end; 4≤H3≤6;

[0061] When the data synchronization mode is the safe mode and the sending end disk type is a mechanical disk, the initial number of core threads of the sending end is reduced by H1 times, and the reduced initial number of core threads of the sending end is used as the target number of core threads of the sending end; 2≤H1≤3;

[0062] When the data synchronization mode is the safe mode and the sending end disk type is a solid state drive, the initial number of core threads of the sending end is reduced by H2 times, and the reduced initial number of core threads of the sending end is used as the target number of core threads of the sending end; 8≤H2≤9;

[0063] When the data synchronization mode is the safety mode and the sending end disk type is a RAID disk, the initial number of core threads of the sending end is reduced by H3 times, and the reduced initial number of core threads of the sending end is used as the target number of core threads of the sending end; 4≤H3≤6.

[0064] Optionally, the acquiring the threshold value of the number of single receptions by the receiving end based on the receiving end information includes:

[0065] Based on the CPU information of the receiving end and the hardware performance information of the disk where the file is to be stored, the receiving end disk storage information and the number of core threads of the receiving end are obtained;

[0066] Based on the acquired memory information of the receiving end and the disk storage information of the receiving end, a single reception quantity threshold of the receiving end is acquired.

[0067] Optionally, calling a corresponding write strategy to write the files to be synchronized to a disk corresponding to the receiving end based on the single reception quantity threshold of the receiving end and the actual number of sent files to be synchronized includes:

[0068] If the actual number of files to be synchronized is not greater than the threshold of the number of files to be synchronized at a time, the size of each file actually sent is obtained, and based on the size of each file and the number of core threads of the receiving end, the corresponding write strategy is called to write the files to be synchronized into the disk corresponding to the receiving end;

[0069] If the actual number of files to be synchronized is greater than the threshold of the number of files to be received at a single time by the receiving end, the files to be synchronized that do not exceed the threshold of the number of files to be received at a single time by the receiving end are recorded as a first group of files, and the files to be synchronized that exceed the threshold of the number of files to be received at a single time by the receiving end are recorded as a second group of files, and the second group of files are stored in the queue to be synchronized;

[0070] Obtaining the size of each file in the first group of files, and based on the size of each file and the number of core threads of the receiving end, calling a corresponding write strategy to write the first group of files into a disk corresponding to the receiving end;

[0071] After all the first group of files are written to the corresponding disk, the size of each file in the second group of files is obtained, and based on the size of each file and the number of core threads of the receiving end, the corresponding write strategy is called to write the second group of files to the corresponding disk of the receiving end.

[0072] Optionally, based on the size of each file and the number of core threads of the receiving end, calling a corresponding write strategy to write the file to be synchronized to a disk corresponding to the receiving end includes:

[0073] Record files whose file sizes exceed a preset size as first-class files, and record files whose file sizes do not exceed a preset size as second-class files, and obtain the total number of the first-class files and the second-class files respectively;

[0074] If the total number of the first type of files and the second type of files are both less than the number of core threads of the receiving end, calling the first writing strategy to write the files to be synchronized into the disk corresponding to the receiving end;

[0075] If the total number of the first type of files and the second type of files is greater than the number of core threads of the receiving end, calling the second writing strategy to write the files to be synchronized into the disk corresponding to the receiving end;

[0076] If the total number of the first type of files and the second type of files are both equal to the number of core threads of the receiving end, calling the third writing strategy to write the files to be synchronized into the disk corresponding to the receiving end;

[0077] If the total number of the first type of files is greater than the number of core threads of the receiving end, and the total number of the second type of files is equal to the number of core threads of the receiving end, calling the fourth writing strategy to write the files to be synchronized to the disk corresponding to the receiving end;

[0078] If the total number of the first type of files is greater than the number of core threads of the receiving end, and the total number of the second type of files is less than the number of core threads of the receiving end, calling the fifth writing strategy to write the files to be synchronized to the disk corresponding to the receiving end;

[0079] If the total number of the first type of files is equal to the number of core threads of the receiving end, and the total number of the second type of files is greater than the number of core threads of the receiving end, calling the sixth writing strategy to write the files to be synchronized to the disk corresponding to the receiving end;

[0080] If the total number of the first type of files is equal to the number of core threads of the receiving end, and the total number of the second type of files is less than the number of core threads of the receiving end, calling the seventh writing strategy to write the files to be synchronized to the disk corresponding to the receiving end;

[0081] If the total number of the first type of files is less than the number of core threads of the receiving end, and the total number of the second type of files is equal to the number of core threads of the receiving end, calling the eighth writing strategy to write the files to be synchronized to the disk corresponding to the receiving end;

[0082] If the total number of the first type of files is less than the number of core threads of the receiving end, and the total number of the second type of files is greater than the number of core threads of the receiving end, the ninth writing strategy is called to write the files to be synchronized into the disk corresponding to the receiving end.

[0083] Optionally, the method for obtaining the initial number of core threads of the receiving end includes:

[0084] Acquire the CPU size of the receiving end based on the CPU information of the receiving end;

[0085] Determine the type of the receiving end disk based on the hardware performance information of the disk where the file is to be stored;

[0086] The initial number of core threads of the receiving end is determined based on the CPU size of the receiving end and the disk type of the receiving end.

[0087] Optionally, when the receiving end disk type is a mechanical disk, the initial number of core threads of the receiving end is consistent with the size of the receiving end CPU;

[0088] When the receiving end disk type is a solid state drive, the initial number of core threads of the receiving end is M1 times the size of the receiving end CPU;

[0089] When the receiving end disk type is a RAID disk, the initial number of core threads of the receiving end is M2 times the size of the receiving end CPU;

[0090] M1≥M2.

[0091] Optionally, the single receiving quantity threshold of the receiving end is Q2: Q2 = J2 ÷ J1 × J0;

[0092] Among them, J2 is the receiving end disk storage information, J1 is the receiving end memory information, and J0 is the receiving end CPU information.

[0093] In a second aspect, the present disclosure also provides a file synchronization system that is adaptable to a variety of hardware environments, including:

[0094] A sending end information acquisition module, used to acquire sending end information, wherein the sending end information includes sending end disk type, sending end disk storage information, sending end memory information, and sending end CPU information;

[0095] A pre-analysis module, used to perform pre-analysis based on the sending end information to determine the initial number of core threads of the sending end and the single sending amount of the sending queue;

[0096] The dynamic adjustment module is used to dynamically adjust the initial core thread number of the sending end based on the total data volume of the file to be synchronized and the preset dynamic adjustment strategy to obtain the target core thread number of the sending end YY+242255P

[0097] Number of trips;

[0098] A thread creation module, used to create the same number of Socket sending threads based on the target core thread number of the sending end, and send the to-be-synchronized file to the receiving end based on the Socket sending threads and the single sending amount;

[0099] A receiving end information acquisition module is used to acquire receiving end information, wherein the receiving end information includes receiving end disk type, receiving end disk storage information, receiving end core thread number, and receiving end CPU information;

[0100] A receiving end single reception quantity threshold acquisition module, used to acquire the receiving end single reception quantity threshold based on the receiving end information;

[0101] The dynamic writing module is used to call the corresponding writing strategy to write the to-be-synchronized files into the disk corresponding to the receiving end based on the single receiving quantity threshold of the receiving end and the actual sending quantity of the to-be-synchronized files.

[0102] In a third aspect, the embodiment of the present disclosure further provides a file sending method adapted to various hardware environments, which is applicable to file sending at a sending end in file synchronization in various hardware environments, including:

[0103] Acquire the sender information, wherein the sender information includes the sender disk type, the sender disk storage information, the sender memory information, and the sender CPU information;

[0104] Performing a preliminary analysis based on the sending end information to determine the initial number of core threads of the sending end and the single sending volume of the sending queue;

[0105] Based on the total data volume of the files to be synchronized and a preset dynamic adjustment strategy, dynamically adjust the initial number of core threads of the sending end to obtain the target number of core threads of the sending end;

[0106] The same number of Socket sending threads is created based on the target core thread number of the sending end, and the files to be synchronized are sent to the receiving end based on the Socket sending threads and the single sending amount.

[0107] In a fourth aspect, the embodiments of the present disclosure further provide a file receiving method adapted to a variety of hardware environments, which is applicable to file receiving at a receiving end in file synchronization in a variety of hardware environments, including:

[0108] Acquire receiving end information, wherein the receiving end information includes receiving end disk type, receiving end disk storage information, receiving end core thread number, and receiving end CPU information;

[0109] Acquire a single reception quantity threshold of the receiving end based on the receiving end information;

[0110] Based on the single receiving quantity threshold of the receiving end and the actual sending quantity of the to-be-synchronized files, the corresponding writing strategy is called to write the to-be-synchronized files into the disk corresponding to the receiving end.

[0111] In a fifth aspect, the embodiments of the present disclosure further provide a computer device, which adopts the following technical solution:

[0112] The computer device comprises:

[0113] at least one processor; and,

[0114] a memory communicatively connected to the at least one processor; wherein,

[0115] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute any of the above-mentioned file synchronization methods adapted to multiple hardware environments, or the above-mentioned file sending methods adapted to multiple hardware environments, or the above-mentioned file receiving methods adapted to multiple hardware environments.

[0116] In a sixth aspect, an embodiment of the present disclosure further provides a computer-readable storage medium, which stores computer instructions, and the computer instructions are used to enable a computer to execute any of the above-mentioned file synchronization methods that are adapted to multiple hardware environments, or the above-mentioned file sending methods that are adapted to multiple hardware environments, or the above-mentioned file receiving methods that are adapted to multiple hardware environments.

[0117] In a seventh aspect, an embodiment of the present disclosure further provides a computer program product, comprising a computer program / instruction, which implements the steps of any of the above methods when executed by a processor.

[0118] The file synchronization method disclosed in the present application is adaptable to a variety of hardware environments. By acquiring detailed hardware information of the sending end and the receiving end, it can optimize the configuration according to different hardware environments and reduce compatibility issues caused by hardware differences. After pre-analyzing the sending end information, it can automatically determine the appropriate number of core threads and the sending amount, simplifying the configuration process. According to the total data volume of the file to be synchronized, the number of core threads of the sending end is dynamically adjusted, which can effectively improve the data transmission speed and avoid performance bottlenecks. By creating the same number of Socket sending threads as the target number of core threads, high-concurrency data transmission can be achieved and the file synchronization time can be shortened. Dynamic adjustment of the sending strategy can reduce the delay of data synchronization, thereby reducing the possibility of inconsistency in data during transmission. According to the actual sending amount and the receiving capacity of the receiving end, the appropriate write strategy is automatically selected to ensure the integrity and consistency of the data. Based on the acquired hardware information, the system can select a suitable encryption strategy to balance performance and security and reduce the burden of the encryption process on the system. By ensuring that each step takes security into consideration, the risk of data leakage or tampering during transmission can be effectively reduced. The automatic configuration and dynamic adjustment strategies in this application reduce the need for manual intervention and alleviate the workload of operation and maintenance personnel; through real-time monitoring and dynamic adjustment, the resources of the sending and receiving ends can be more efficiently utilized to improve overall performance.

[0119] The above description is only an overview of the technical solution of the present invention. In order to more clearly understand the technical means of the present invention, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present invention more obvious and easy to understand, the following preferred embodiments are specifically cited and described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0120] In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0121] Figure 1 A flowchart of a file synchronization method adapted to a variety of hardware environments provided by an embodiment of the present disclosure.

[0122] Figure 2 A flowchart of a method for determining the initial number of core threads on the sending end and the single sending amount of the sending queue provided in an embodiment of the present disclosure.

[0123] Figure 3 A flowchart of a method for obtaining the initial number of core threads of a sending-end thread pool based on a first strategy provided in an embodiment of the present disclosure.

[0124] Figure 4 A flowchart of a method for obtaining the initial number of core threads of a sending-end thread pool based on a second strategy provided in an embodiment of the present disclosure.

[0125] Figure 5 A flowchart of a method for obtaining the initial number of core threads of a sending-end thread pool based on a third strategy provided in an embodiment of the present disclosure.

[0126] Figure 6 A flowchart of a method for obtaining the target core thread number of a sending end provided in an embodiment of the present disclosure.

[0127] Figure 7 A flowchart of a method for obtaining a single reception quantity threshold at a receiving end provided in an embodiment of the present disclosure.

[0128] Figure 8 A flowchart of a method for writing a file to be synchronized to a disk corresponding to a receiving end provided in an embodiment of the present disclosure.

[0129] Fig. 9 A flowchart of a method for calling a write strategy provided in an embodiment of the present disclosure.

[0130] Fig.10 A schematic diagram of the structure of a computer device provided in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0131] The embodiments of the present disclosure are described in detail below with reference to the accompanying drawings.

[0132] It should be clear that the following embodiments of the present disclosure are described by specific specific examples, and those skilled in the art can easily understand other advantages and effects of the present disclosure from the contents disclosed in this specification. Obviously, the described embodiments are only a part of the embodiments of the present disclosure, rather than all of the embodiments. The present disclosure can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present disclosure. It should be noted that the following embodiments and features in the embodiments can be combined with each other in the absence of conflict. Based on the embodiments in the present disclosure, all other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of the present disclosure.

[0133] It should be noted that various aspects of the embodiments within the scope of the appended claims are described below. It should be apparent that the aspects described herein may be embodied in a wide variety of forms, and any specific structure and / or function described herein is merely illustrative. Based on the present disclosure, it should be understood by those skilled in the art that an aspect described herein may be implemented independently of any other aspect, and two or more of these aspects may be combined in various ways. For example, any number of aspects described herein may be used to implement the device and / or practice the method. In addition, other structures and / or functionalities other than one or more of the aspects described herein may be used to implement this device and / or practice this method.

[0134] It should also be noted that the illustrations provided in the following embodiments are only schematic illustrations of the basic concept of the present disclosure. The drawings only show components related to the present disclosure rather than being drawn according to the number, shape and size of components in actual implementation. In actual implementation, the type, quantity and proportion of each component may be changed arbitrarily, and the component layout may also be more complicated.

[0135] Additionally, in the following description, specific details are provided to facilitate a thorough understanding of the examples. However, it will be understood by those skilled in the art that the aspects described may be practiced without these specific details.

[0136] Reference Figure 1 The first aspect of the present application discloses a file synchronization method adapted to a variety of hardware environments, which specifically includes the following steps:

[0137] S100, obtaining information of the sending end, where the sending end information includes a sending end disk type, sending end disk storage information, sending end memory information, and sending end CPU information.

[0138] Specifically, based on the acquired hardware performance information of the disk where the file to be synchronized is located, the type of the sending end disk and the storage information of the sending end disk are determined.

[0139] For example, when the sending end is an external server, the external server mainly reads files and reads persistent files, that is, files to be synchronized, from the mechanical disk, solid-state disk or RAID disk of the hardware architecture. In this process, the disk information of the files to be synchronized is obtained, including the disk type, storage capacity, and CPU information of the PC.

[0140] This step ensures a comprehensive understanding of the hardware performance of the sender, thus providing a basis for subsequent configuration and optimization and avoiding performance bottlenecks caused by hardware limitations.

[0141] S200, performing a preliminary analysis based on the information of the sending end to determine the initial number of core threads of the sending end and the single sending amount of the sending queue.

[0142] specifically refer to Figure 2 The method for determining the initial number of core threads at the sending end and the single sending amount of the sending queue specifically includes the following steps:

[0143] S210, calling a core thread number setting strategy of a corresponding sending-end thread pool based on the sending-end disk type.

[0144] Different disk types have different read and write speeds. Automatically setting the number of threads according to the disk type can fully utilize the performance of the hardware and ensure efficient data transmission. Reasonable configuration of the number of threads can reduce waiting time, increase data processing speed, and thus reduce transmission delays.

[0145] If the sending end disk type is the first type, the initial number of core threads of the sending end thread pool is obtained based on the first strategy.

[0146] The first type is a mechanical disk, and refer to Figure 3 The method for obtaining the initial number of core threads of the sending end thread pool based on the first strategy specifically includes:

[0147] A111, obtain the CPU size of the sending end based on the CPU information of the sending end.

[0148] The obtained CPU size can accurately reflect the processing capacity of the sender, provide a basis for the subsequent setting of the number of threads, and can flexibly adapt to different hardware environments according to the actual CPU configuration to ensure the rationality of the system configuration.

[0149] A112, determining the initial number of core threads of the thread pool of the sender based on the CPU size of the sender;

[0150] The initial number of core threads is N0: N0=n0, where n0 is the CPU size of the sending end, that is, for a mechanical disk, the number of core threads of the thread pool is set to be consistent with the number of CPUs.

[0151] A reasonable initial number of core threads can maximize the use of CPU resources, improve data processing capabilities, and reduce thread switching overhead. By directly setting the number of threads to the CPU size, the configuration process is simple and clear, reducing complex adjustments and trial-and-error processes.

[0152] If the sending end disk type is the second type, the initial number of core threads of the sending end thread pool is obtained based on the second strategy.

[0153] The second type is a solid state drive, and refer to Figure 4 The method for obtaining the initial number of core threads of the sending end thread pool based on the second strategy specifically includes:

[0154] A211, obtaining the CPU size of the sending end based on the CPU information of the sending end;

[0155] A212, determining the initial number of core threads of the sender thread pool based on the sender CPU size;

[0156] The initial number of core threads is N0: N0=n0×n1, where n0 is the CPU size of the sender, that is, the initial number of core threads of the sender thread pool is configured to be n1 times the CPU size of the sender.

[0157] In this embodiment, preferably 4≤n1≤16.

[0158] In this embodiment, by introducing the multiplier factor n1, the number of threads can be dynamically adjusted according to different workload types (CPU intensive or IO intensive), so as to adapt to various application scenarios more flexibly; reasonable setting of the number of threads can make full use of CPU resources, avoid idle or overloaded resources, and improve the overall performance and throughput of the system; for specific types of workloads (such as IO intensive), increasing the number of threads can reduce the time that threads wait for IO, thereby improving the response speed of the system and task processing efficiency; although an additional multiplier factor is introduced, by presetting a reasonable n1 value, the configuration process of the number of threads can be simplified and frequent manual adjustments can be avoided.

[0159] If the sending end disk type is the third type, the initial number of core threads of the sending end thread pool is obtained based on the third strategy.

[0160] The third type is RAID disk, refer to Figure 5 The method for obtaining the initial number of core threads of the sending end thread pool based on the third strategy specifically includes:

[0161] A311, obtains the CPU size of the sending end based on the CPU information of the sending end;

[0162] A312, determining the initial number of core threads of the sender thread pool based on the sender CPU size;

[0163] The initial number of core threads is N0: N0=n0×n2, where n0 is the CPU size of the sender, that is, the initial number of core threads of the sender thread pool is n2 times the CPU size of the sender.

[0164] In this embodiment, preferably, 2≤n2≤8.

[0165] In this embodiment, the number of threads is calculated based on the CPU size and dynamic factors, which can flexibly respond to different types of application loads and ensure optimal system performance; by optimizing thread configuration, it can reduce latency and increase response speed, thereby improving user experience; it can reasonably allocate system resources under different operating conditions, improve resource utilization efficiency, and reduce costs.

[0166] S220, setting an initial number of core threads at the sending end based on a core thread number setting strategy.

[0167] Assuming that the number of threads of the sender is determined to be 8 in step S210, according to the current CPU load and memory usage (eg, the CPU load is low and the memory usage is 40%), it is finally decided to set the initial number of core threads of the sender to 8.

[0168] The number of threads can be flexibly set according to the actual usage of the CPU and memory, which improves the adaptability of the system and avoids resource waste; a reasonable configuration of the number of core threads can ensure load balancing and avoid overloading of a single thread, thereby improving the overall processing capability.

[0169] S230, determining a single sending amount of the sending queue based on the sending end memory information and the sending end disk storage information.

[0170] The single transmission information is Q1: Q1 = F2 ÷ F1 × F0;

[0171] Among them, F2 is the disk storage information of the sender, F1 is the memory information of the sender, and F0 is the CPU information of the sender.

[0172] By comprehensively considering memory and storage information, memory overflow or disk write pressure caused by an overly large sending queue can be effectively avoided. Reasonable sending volume configuration can improve data transmission efficiency, reduce the frequency of network transmission, and thus improve the overall transmission speed.

[0173] In this embodiment, S210-S230 discloses a method for determining the initial number of core threads on the sending end and the single sending amount of the sending queue. Through a comprehensive analysis of the sending end information, the system can optimize the configuration according to specific hardware conditions to ensure that available resources are fully utilized during the transmission process; automated configuration of the number of core threads and sending amount reduces the need for human intervention, making the system easier to manage and maintain; reasonably configured number of threads and sending amount can significantly increase data transmission speed, reduce latency, and enhance user experience; the solution can adaptively adjust in a variety of hardware environments to ensure good performance in different usage scenarios; through dynamic adjustment and reasonable configuration, the risk of system overload and data loss is reduced, and the stability and reliability of the file synchronization process are improved.

[0174] S300, dynamically adjusting the initial number of core threads on the sending end based on the total data volume of the files to be synchronized and a preset dynamic adjustment strategy, to obtain a target number of core threads on the sending end.

[0175] Specifically, refer to Figure 6 , the method for obtaining the target core thread number of the sending end specifically includes the following steps:

[0176] S310: Determine a data synchronization mode based on the acquired total data volume (ie, total size) of the files to be synchronized.

[0177] By identifying the data synchronization mode, you can select the corresponding optimization strategy according to different load types to ensure that the data synchronization process is efficient and stable, can quickly adapt to different data volume requirements, and ensure that the task can be effectively completed under high load conditions.

[0178] Specifically, if the total data volume of the files to be synchronized meets the first range, the data synchronization mode is determined to be the high-efficiency mode.

[0179] In the embodiment, the first range is M1, 1KB≤M1≤2MB.

[0180] If the total data volume of the files to be synchronized meets the second range, the data synchronization mode is determined to be a stable mode.

[0181] In the embodiment, the second range is M2, 2MB<M2≤50MB.

[0182] If the total data volume of the files to be synchronized meets the third range, the data synchronization mode is determined to be the general mode.

[0183] In the embodiment, the third range is M3, 50MB<M3≤200MB.

[0184] If the total data volume of the files to be synchronized meets the fourth range, the data synchronization mode is determined to be the safe mode.

[0185] In the embodiment, the fourth range is M4, 200MB<M4.

[0186] S320, based on the data synchronization mode and the sender disk type, a corresponding dynamic adjustment strategy for the number of core threads on the sender is called.

[0187] Dynamically adjust the number of threads according to the data synchronization mode and disk performance to ensure full utilization of hardware capabilities and improve synchronization efficiency. Reasonable thread number configuration can reduce data transmission delays and improve overall response speed.

[0188] Specifically, when the data synchronization mode is the high-efficiency mode and the sender disk type is a mechanical disk, the corresponding sender core thread number dynamic adjustment strategy is called as the first expansion strategy;

[0189] When the data synchronization mode is the high-efficiency mode and the sender disk type is a solid-state drive, the corresponding sender core thread number dynamic adjustment strategy is called as the second expansion strategy;

[0190] When the data synchronization mode is the high-efficiency mode and the sender disk type is a RAID disk, the corresponding sender core thread number dynamic adjustment strategy is called as the third expansion strategy;

[0191] When the data synchronization mode is the safe mode and the sender disk type is a mechanical disk, the corresponding sender core thread number dynamic adjustment strategy is called as the first reduction strategy;

[0192] When the data synchronization mode is the safe mode and the sender disk type is a solid state drive, the corresponding sender core thread number dynamic adjustment strategy is called as the second reduction strategy;

[0193] When the data synchronization mode is the safe mode and the sending end disk type is a RAID disk, the corresponding sending end core thread number dynamic adjustment strategy is called as the third reduction strategy.

[0194] S330, in response to the dynamic adjustment strategy of the number of core threads at the sending end, dynamically adjust the initial number of core threads at the sending end to obtain the target number of core threads at the sending end.

[0195] By dynamically adjusting the number of core threads, it is possible to maintain optimal performance during actual operation and improve the efficiency of data synchronization. It can also be adjusted according to the actual data volume and hardware performance during operation to ensure that the system is always in an efficient working state.

[0196] Specifically, when the data synchronization mode is the high-efficiency mode and the disk type of the sending end is a mechanical disk, the first expansion strategy is executed, that is, the initial number of core threads of the sending end is expanded by H1 times, and the expanded initial number of core threads of the sending end is used as the target number of core threads of the sending end; 2≤H1≤3;

[0197] When the data synchronization mode is the high-efficiency mode and the sender disk type is a solid-state drive, the second expansion strategy is executed, that is, the initial core thread number of the sender is expanded by H2 times, and the expanded initial core thread number of the sender is used as the target core thread number of the sender; 8≤H2≤9;

[0198] When the data synchronization mode is the high-efficiency mode and the sender disk type is a RAID disk, the third expansion strategy is executed, that is, the initial core thread number of the sender is expanded by H3 times, and the expanded initial core thread number of the sender is used as the target core thread number of the sender; 4≤H3≤6;

[0199] When the data synchronization mode is the safe mode and the sender disk type is a mechanical disk, the first reduction strategy is executed, that is, the initial core thread number of the sender is reduced by H1 times, and the reduced initial core thread number of the sender is used as the target core thread number of the sender; 2≤H1≤3;

[0200] When the data synchronization mode is the safe mode and the sender disk type is a solid state drive, the second reduction strategy is executed, that is, the initial core thread number of the sender is reduced by H2 times, and the reduced initial core thread number of the sender is used as the target core thread number of the sender; 8≤H2≤9;

[0201] When the data synchronization mode is the safety mode and the sending end disk type is a RAID disk, the third reduction strategy is executed, that is, the initial core thread number of the sending end is reduced by H3 times, and the reduced initial core thread number of the sending end is used as the target core thread number of the sending end; 4≤H3≤6.

[0202] When the data synchronization mode is the stable mode or the general mode, the initial number of core threads on the sending end is used as the target number of core threads on the sending end.

[0203] In another scheme, for the data synchronization mode being the high-efficiency mode, when the sending end disk type is a mechanical disk, the initial number of core threads of the sending end can be expanded by 7-8 times, and the expanded initial number of core threads of the sending end is used as the target number of core threads of the sending end; when the sending end disk type is a solid-state drive, the initial number of core threads of the sending end can be expanded by 8-9 times, and the expanded initial number of core threads of the sending end is used as the target number of core threads of the sending end; when the sending end disk type is a RAID disk, the initial number of core threads of the sending end can be expanded by 4-6 times, and the expanded initial number of core threads of the sending end is used as the target number of core threads of the sending end.

[0204] For the data synchronization mode being the stable mode, when the sending end disk type is a mechanical disk, the initial number of core threads of the sending end can be expanded by 4-6 times, and the expanded initial number of core threads of the sending end is used as the target number of core threads of the sending end; when the sending end disk type is a solid state drive, the initial number of core threads of the sending end can be expanded by 6-7 times, and the expanded initial number of core threads of the sending end is used as the target number of core threads of the sending end; when the sending end disk type is a RAID disk, the initial number of core threads of the sending end can be expanded by 2-3 times, and the expanded initial number of core threads of the sending end is used as the target number of core threads of the sending end.

[0205] For the general data synchronization mode, when the sending end disk type is a mechanical disk, the initial number of core threads of the sending end can be increased by 2-3 times, and the increased initial number of core threads of the sending end is used as the target number of core threads of the sending end; when the sending end disk type is a solid state drive, the initial number of core threads of the sending end can be increased by 4-5 times, and the increased initial number of core threads of the sending end is used as the target number of core threads of the sending end; when the sending end disk type is a RAID disk, the initial number of core threads of the sending end can be increased by 1 times, and the increased initial number of core threads of the sending end is used as the target number of core threads of the sending end.

[0206] For the data synchronization mode being the safe mode, when the sending end disk type is a mechanical disk, the initial number of core threads of the sending end can be reduced by 2-3 times, and the enlarged initial number of core threads of the sending end can be used as the target number of core threads of the sending end; when the sending end disk type is a solid state drive, the initial number of core threads of the sending end can be reduced by 2-3 times, and the enlarged initial number of core threads of the sending end can be used as the target number of core threads of the sending end; when the sending end disk type is a RAID disk, the initial number of core threads of the sending end can be reduced by 2-3 times, and the enlarged initial number of core threads of the sending end can be used as the target number of core threads of the sending end.

[0207] S400: Create the same number of Socket sending threads based on the target core thread number of the sending end, and send the to-be-synchronized file to the receiving end based on the Socket sending threads and the single sending amount.

[0208] Create the same number of socket sending threads as the target core threads, and send the to-be-synchronized file segments to the receiving end according to the amount of each segment sent. This can effectively improve the concurrent transmission capability, shorten the synchronization time, and make the file transfer process more efficient.

[0209] S500, obtaining receiving end information, where the receiving end information includes receiving end disk type, receiving end disk storage information, receiving end core thread number, and receiving end CPU information.

[0210] The method for obtaining the initial number of core threads at the receiving end includes:

[0211] Get the CPU size of the receiving end based on the CPU information of the receiving end;

[0212] Determine the type of the receiving end disk based on the hardware performance information of the disk where the file is to be stored;

[0213] The initial number of core threads on the receiving end is determined based on the receiving end CPU size and the receiving end disk type.

[0214] When the receiving end disk type is a mechanical disk, the initial number of core threads on the receiving end is consistent with the size of YY+242255PCPU on the receiving end;

[0215] When the receiving end disk type is a solid state drive, the initial number of core threads at the receiving end is M1 times the size of the receiving end CPU, preferably 16≥M1≥4.

[0216] When the receiving end disk type is a RAID disk, the initial core thread number of the receiving end is M2 times the size of the receiving end CPU, preferably 8≥M2≥2.

[0217] M1≥M2.

[0218] By obtaining the receiving end information, transmission failure or delay caused by insufficient receiving end performance can be effectively avoided.

[0219] S600, obtaining a single reception quantity threshold of the receiving end based on the receiving end information.

[0220] Specific reference Figure 7 , the method for obtaining the single receiving quantity threshold of the receiving end specifically includes:

[0221] S610, based on the CPU information of the receiving end and the hardware performance information of the disk where the file is to be stored, obtain the disk storage information of the receiving end and the number of core threads of the receiving end.

[0222] Obtaining the CPU and disk information of the receiving end can fully understand the hardware performance of the system, provide a scientific basis for the subsequent setting of the receiving quantity threshold, and dynamically adjust the system parameters according to the hardware performance to ensure the rationality of resource allocation.

[0223] S620, based on the acquired memory information and disk storage information of the receiving end, acquire a single reception quantity threshold of the receiving end.

[0224] The single receiving quantity threshold of the receiving end is Q2: Q2=J2÷J1×J0;

[0225] Among them, J2 is the receiving end disk storage information, J1 is the receiving end memory information, and J0 is the receiving end CPU information.

[0226] By reasonably setting the threshold for the number of data received at a time, you can ensure a smooth data receiving process, avoid memory or disk bottlenecks caused by receiving too much data at one time, avoid system pressure caused by receiving too much data at one time, reduce potential delays and errors, and improve stability.

[0227] S700: Based on the single reception quantity threshold of the receiving end and the actual number of files to be synchronized, a corresponding write strategy is called to write the files to be synchronized into a disk corresponding to the receiving end.

[0228] Specific reference Figure 8 The method for writing the file to be synchronized into the disk corresponding to the receiving end specifically includes:

[0229] If the actual number of files to be synchronized is not greater than the threshold of the number of files to be received at a time on the receiving end, obtain the size of each file actually sent, and based on the size of each file and the number of core threads on the receiving end, call the corresponding write strategy to write the files to be synchronized to the disk corresponding to the receiving end;

[0230] If the actual number of files to be synchronized is greater than the threshold of the number of files to be received at the receiving end at a single time, the files to be synchronized that do not exceed the threshold of the number of files to be received at the receiving end at a single time are recorded as the first group of files, and the files to be synchronized that exceed the threshold of the number of files to be received at the receiving end at a single time are recorded as the second group of files, and the second group of files are stored in the queue to be synchronized;

[0231] Obtain the size of each file in the first group of files, and based on the size of each file and the number of core threads on the receiving end, call the corresponding write strategy to write the first group of files into the disk corresponding to the receiving end;

[0232] After all the first set of files are written to the corresponding disk, the size of each file in the second set of files is obtained, and based on the size of each file and the number of core threads on the receiving end, the corresponding write strategy is called to write the second set of files to the corresponding disk on the receiving end.

[0233] Based on the threshold of the number of files to be synchronized received at the receiving end at one time and the actual number of files to be synchronized sent, the corresponding write strategy is called to write the files to be synchronized to the receiving end disk. Through clear steps and strategies, the data synchronization process can be optimized. Specifically, if the actual number of files to be synchronized sent is not greater than the threshold of the number of files to be synchronized received at the receiving end at one time, when the actual number of files sent is less than or equal to the threshold, the size of each file is obtained, and based on the number of core threads on the receiving end, the corresponding write strategy is called, which can reduce the processing complexity, write small-scale data quickly and efficiently, avoid resource waste, and ensure fast data transmission; if the actual number of files to be synchronized sent is greater than the threshold of the number of files to be synchronized received at the receiving end at one time, the files that do not exceed the threshold are marked as the first group, and the files that exceed the threshold are marked as the second group, and the second group of files are stored in The queue to be synchronized optimizes resource allocation through group processing, reduces competition during the writing process, controls the number of files received each time, and avoids excessive system load; obtains the size of each file in the first group of files, calls the corresponding write strategy, obtains the file size for the first group of files, and adopts a suitable write strategy for writing according to the number of core threads, and uses multi-threaded parallel writing to improve file transfer speed, process sequential writing, and reduce the risk of data inconsistency; after the first group of files are all written, process the second group of files, that is, after the first group of files are all written, obtain the size of the second group of files, and call the corresponding write strategy to write them, ensuring that the second group is started after the first group of files is processed, reducing the system burden, and the write strategy can be flexibly adjusted according to the current system status and load.

[0234] In this embodiment, reasonable threshold control and group processing enable data to be written to the receiving end disk efficiently and quickly, dynamically adjust write strategies and thread usage, maximize the use of system resources, avoid idle resources, control the number of files received each time, reduce the risk of crash or performance degradation due to overload, ensure stable system operation, and dynamically adjust the write strategy according to the actual number of sends and the capabilities of the receiving end, so that the system can flexibly adapt to different workloads and environments. The automated grouping and writing process reduces manual intervention, simplifies system management and maintenance, and improves work efficiency. Through group processing and orderly writing, the risk of data loss or damage is reduced, and the consistency and integrity of the data are ensured.

[0235] In summary, through scientific step design and dynamic adjustment, this solution can improve the efficiency, stability and resource utilization of data transmission in many aspects, ensuring that the receiving end can efficiently process the files to be synchronized.

[0236] Further, refer to Fig. 9 , the writing strategy calling method specifically includes the following schemes:

[0237] B100, record files whose sizes exceed a preset size as first-class files, and record files whose sizes do not exceed a preset size as second-class files, and obtain the total number of first-class files and second-class files respectively.

[0238] Among them, the first type of files refers to large files, and the second type of files refers to small files.

[0239] B200: If the total number of first-class files and second-class files is less than the number of core threads on the receiving end, call the first writing strategy to write the files to be synchronized to the disk corresponding to the receiving end.

[0240] Specifically, when the total number of Category I files, the total number of Category II files, and the sum of Category I and Category II files are all less than the number of core threads on the receiving end, the first write strategy includes: evenly distributing the files to be synchronized to each thread, and each thread is responsible for writing a certain number of files, that is, writing them to the corresponding disk.

[0241] When the total number of Class I files and the total number of Class II files are both less than the number of core threads on the receiving end, and the sum of Class I files and Class II files is not less than the number of core threads on the receiving end, the first writing strategy includes: preferentially allocating Class I files for writing, and then allocating the writing of Class II files.

[0242] B300: If the total number of first-class files and second-class files is greater than the number of core threads on the receiving end, the second writing strategy is called to write the files to be synchronized to the disk corresponding to the receiving end.

[0243] Specifically, the number of first-class files exceeding the number of core threads at the receiving end is obtained and stored in the receiving queue. The first-class files of the same number are preferentially processed by the core threads at the receiving end, and then the first-class files in the receiving queue are processed, and finally the second-class files are written.

[0244] For writing of Category II files, obtain the quotient of the total number of Category II files and the number of core threads on the receiving end. If the quotient is an integer and not greater than a preset threshold, all Category II files are evenly distributed and each core thread on the receiving end processes the same amount of Category II files. If the quotient is an integer and greater than a preset threshold, each core thread on the receiving end processes the same amount of Category II files as the preset threshold, and stores the excess Category II files in the receiving queue before processing the Category II files in the receiving queue.

[0245] B400, if the total number of first-class files and second-class files is equal to the number of core threads of the receiving end, call the third writing strategy to write the files to be synchronized to the disk corresponding to the receiving end.

[0246] Specifically, the third writing strategy includes: preferably writing all the first-category files, and then writing all the second-category files.

[0247] B500, if the total number of the first type of files is greater than the number of core threads of the receiving end, and the total number of the second type of files is equal to the number of core threads of the receiving end, call the fourth writing strategy to write the files to be synchronized to the disk corresponding to the receiving end.

[0248] Specifically, the fourth write strategy includes: obtaining a type of file that exceeds the number of core threads on the receiving end, and storing it in the receiving queue, preferentially processing the same number of type of files through the core thread on the receiving end, and then processing the type of files in the receiving queue, and finally performing YY+242255P

[0249] File writing for type 2 files.

[0250] B600, if the total number of the first type of files is greater than the number of core threads of the receiving end, and the total number of the second type of files is less than the number of core threads of the receiving end, call the fifth writing strategy to write the files to be synchronized to the disk corresponding to the receiving end.

[0251] Specifically, the fifth writing strategy includes: obtaining a number of Class I files exceeding the number of core threads on the receiving end, and storing them in a receiving queue, preferentially processing the same number of Class I files through the core threads on the receiving end, then processing the Class I files in the receiving queue, and finally arbitrarily selecting an equal number of core threads on the receiving end to write Class II files.

[0252] B700, if the total number of the first type of files is equal to the number of core threads of the receiving end, and the total number of the second type of files is greater than the number of core threads of the receiving end, call the sixth writing strategy to write the files to be synchronized to the disk corresponding to the receiving end.

[0253] Specifically, the sixth write strategy includes: giving priority to writing one type of files; then, obtaining the quotient of the total number of the second type of files and the number of core threads on the receiving end; if the quotient is an integer divisible value and is not greater than a preset threshold, all the second type of files are evenly distributed, and each core thread on the receiving end processes the same amount of second type of files; if the quotient is an integer divisible value and is greater than the preset threshold, each core thread on the receiving end processes the same amount of second type of files as the preset threshold, stores the excess second type of files in the receiving queue, and then processes the second type of files in the receiving queue.

[0254] B800, if the total number of the first type of files is equal to the number of core threads of the receiving end, and the total number of the second type of files is less than the number of core threads of the receiving end, call the seventh writing strategy to write the files to be synchronized to the disk corresponding to the receiving end.

[0255] Specifically, the seventh writing strategy includes: giving priority to processing all the first-category files, and then arbitrarily selecting an equal number of receiving-end core threads to write the second-category files.

[0256] B900, if the total number of the first category of files is less than the number of core threads on the receiving end, and the total number of the second category of files is equal to the number of core threads on the receiving end, call the eighth write strategy to write the files to be synchronized to the disk corresponding to the receiving end.

[0257] Specifically, the eighth writing strategy includes: writing all the first-category files first, and then writing the second-category files.

[0258] B1000, if the total number of the first category of files is less than the number of core threads on the receiving end, and the total number of the second category of files is greater than the number of core threads on the receiving end, call the ninth write strategy to write the files to be synchronized to the disk corresponding to the receiving end.

[0259] Specifically, the ninth write strategy includes: giving priority to writing all Class I files; then obtaining the quotient of the total number of Class II files and the number of core threads on the receiving end; if the quotient is an integer divisible value and is not greater than a preset threshold, all Class II files are evenly distributed, and each receiving end core thread processes the same amount of Class II files; if the quotient is an integer divisible value and is greater than a preset threshold, each receiving end core thread processes the same amount of Class II files as the preset threshold, stores the excess Class II files in the receiving queue, and then processes the Class II files in the receiving queue.

[0260] In this embodiment, by classifying files by size, it helps to accurately control write strategies and improve processing efficiency. Different write strategies can be optimized for different resource conditions and file characteristics to improve write efficiency. By reasonably allocating threads and tasks, resource competition can be reduced and overall performance can be improved. Strategies can be flexibly called according to actual conditions to adapt to changing workloads and ensure that the system can still run smoothly under high loads.

[0261] The solution disclosed in this embodiment improves the efficiency of file writing and shortens data synchronization time by intelligently allocating and using core threads. Through reasonable writing strategies, it reduces the risk of system crash due to overload and ensures high availability of the system. It dynamically adjusts the allocation of writing tasks and core threads to ensure full utilization of system resources and avoids idle resources. Through orderly processing and reasonable strategies, it reduces inconsistencies in the data transmission process and improves data integrity. Automated strategy calls reduce manual intervention and improve the convenience and efficiency of system management.

[0262] In summary, this solution can improve the efficiency, stability and resource utilization of file writing in many aspects through precise classification management and flexible policy calling, and provide reliable guarantee for data synchronization at the receiving end.

[0263] The file synchronization method disclosed in the present application is adaptable to a variety of hardware environments. By acquiring detailed hardware information of the sending end and the receiving end, it can optimize the configuration according to different hardware environments and reduce compatibility issues caused by hardware differences. After pre-analyzing the sending end information, it can automatically determine the appropriate number of core threads and the sending amount, simplifying the configuration process. According to the total data volume of the file to be synchronized, the number of core threads of the sending end is dynamically adjusted, which can effectively improve the data transmission speed and avoid performance bottlenecks. By creating the same number of Socket sending threads as the target number of core threads, high-concurrency data transmission can be achieved and the file synchronization time can be shortened. Dynamic adjustment of the sending strategy can reduce the delay of data synchronization, thereby reducing the possibility of inconsistency in data during transmission. According to the actual sending amount and the receiving capacity of the receiving end, the appropriate write strategy is automatically selected to ensure the integrity and consistency of the data. Based on the acquired hardware information, the system can select a suitable encryption strategy to balance performance and security and reduce the burden of the encryption process on the system. By ensuring that each step takes security into consideration, the risk of data leakage or tampering during transmission can be effectively reduced. The automatic configuration and dynamic adjustment strategies in this application reduce the need for manual intervention and alleviate the workload of operation and maintenance personnel; through real-time monitoring and dynamic adjustment, the resources of the sending and receiving ends can be more efficiently utilized to improve overall performance.

[0264] At the same time, the solution can be flexibly applied in a variety of hardware environments, suitable for different types of servers, storage devices and network settings, enhancing the versatility of the system. When the system upgrades or replaces hardware, the automatic configuration and optimization capabilities can effectively reduce compatibility issues and ensure a smooth transition. The fast and stable file synchronization process can improve user satisfaction and reduce business interruption time. It is suitable for implementation in large-scale distributed systems and can efficiently support large data transmission and multi-task parallel processing.

[0265] In summary, the file synchronization method of the present application not only solves many problems in the prior art, but also optimizes the overall management and maintenance process on the basis of improving efficiency, security and compatibility, thereby greatly enhancing the availability and reliability of the system.

[0266] In the prior art, hardware compatibility issues arise from differences in different servers, storage devices, and network hardware, which can lead to complex system configuration, difficult maintenance, and compatibility issues during hardware upgrades. The solution disclosed in the present application can accurately analyze and evaluate each hardware environment by obtaining detailed hardware information of the sender and receiver (such as disk type, storage information, memory information, CPU information, etc.). In this way, targeted configuration can be made before file synchronization to ensure compatibility under different hardware environments; pre-analysis is performed based on the obtained hardware information to determine the initial configuration and single sending volume; and different configurations and strategies are adapted in different hardware environments by dynamically adjusting strategies, thereby reducing compatibility issues between hardware.

[0267] With regard to the problem of data inconsistency caused by data synchronization delay in distributed systems existing in the prior art, the solution disclosed in the present application dynamically adjusts the number of core threads according to the total data volume of the files to be synchronized and the dynamic adjustment strategy, which can better control the synchronization speed and concurrency, thereby reducing synchronization delays; by reasonably setting the single sending volume and synchronization strategy, data loss or damage during the transmission process can be reduced, and dynamic adjustment of the number of threads and sending strategy can also help optimize the synchronization process and reduce data inconsistency problems caused by delays.

[0268] With regard to the problems existing in the prior art, such as bandwidth limitation, delay and throughput, which are the main performance bottlenecks in the data transmission process and directly affect the real-time and overall performance of the system, the solution disclosed in the present application pre-analyzes the information of the sending end, determines the appropriate number of threads and the single sending amount of the sending queue, effectively utilizes the bandwidth and processing power, and improves the data transmission rate; dynamically adjusts the number of core threads according to the total data volume and dynamic adjustment strategy to adapt to the changes in bandwidth and delay and optimize the data transmission performance; creates the same number of Socket sending threads as the target number of core threads, improves the data transmission rate by concurrent processing, and alleviates the problems of bandwidth limitation and delay.

[0269] The solution disclosed in this application can automatically adapt to different hardware environments and load conditions through pre-analysis and dynamic adjustment strategies, reducing the complexity of manual configuration and maintenance; in the process of file synchronization, by optimizing configuration and synchronization strategies, the system's automated management level is improved, manual intervention is reduced, and management efficiency is improved. The file synchronization method of this application can effectively solve compatibility problems, data consistency problems, performance bottlenecks, encryption and security problems, and management and maintenance complexity in the prior art by dynamically adjusting the number of threads, reasonably configuring the sending volume, and optimizing strategies based on hardware information. This flexible and efficient synchronization solution improves the overall performance, reliability, and security of the system, enabling it to achieve efficient and stable file synchronization in different hardware environments.

[0270] The second aspect of the present application discloses a file synchronization system adapted to a variety of hardware environments, including:

[0271] The sender information acquisition module is used to acquire the sender information, including the sender disk type, the sender disk storage information, the sender memory information, and the sender CPU information;

[0272] A pre-analysis module is used to perform pre-analysis based on the information of the sending end, and determine the number of initial core threads of the sending end and the single sending volume of the sending queue;

[0273] A dynamic adjustment module is used to dynamically adjust the initial number of core threads on the sending end based on the total data volume of the files to be synchronized and a preset dynamic adjustment strategy to obtain the target number of core threads on the sending end;

[0274] The thread creation module is used to create the same number of Socket sending threads based on the target core thread number of the sending end, and send the files to be synchronized to the receiving end based on the Socket sending threads and the single sending amount;

[0275] The receiving end information acquisition module is used to acquire the receiving end information, including the receiving end disk type, the receiving end disk storage information, the receiving end core thread number, and the receiving end CPU information;

[0276] A receiving end single reception quantity threshold acquisition module is used to acquire the receiving end single reception quantity threshold based on the receiving end information;

[0277] The dynamic writing module is used to call the corresponding writing strategy to write the files to be synchronized to the disk corresponding to the receiving end based on the single receiving quantity threshold of the receiving end and the actual sending quantity of the files to be synchronized.

[0278] The third aspect of the present application discloses a file sending method adapted to various hardware environments, which is applicable to file sending at a sending end in file synchronization in various hardware environments, including:

[0279] Get the sender information, including the sender disk type, sender disk storage information, sender memory information, and sender CPU information;

[0280] Perform preliminary analysis based on the sender information to determine the initial number of core threads on the sender and the single sending volume of the send queue;

[0281] Based on the total data volume of the files to be synchronized and the preset dynamic adjustment strategy, the initial number of core threads on the sending end is dynamically adjusted to obtain the target number of core threads on the sending end;

[0282] Create the same number of Socket sending threads based on the target core thread number of the sending end, and send the files to be synchronized to the receiving end based on the Socket sending threads and the single sending amount.

[0283] A fourth aspect of the present application discloses a file receiving method adapted to a variety of hardware environments, which is applicable to file receiving at a receiving end in file synchronization in a variety of hardware environments, and includes:

[0284] Get the receiving end information, including the receiving end disk type, receiving end disk storage information, receiving end core thread number, and receiving end CPU information;

[0285] Obtaining a single receiving quantity threshold of the receiving end based on the receiving end information;

[0286] Based on the single receiving quantity threshold of the receiving end and the actual number of files to be synchronized, the corresponding write strategy is called to write the files to be synchronized into the corresponding disk of the receiving end.

[0287] It should be noted that the specific schemes in the file synchronization method adapted to multiple hardware environments disclosed in the first aspect of this application are all applicable to the file synchronization system adapted to multiple hardware environments disclosed in the second aspect of this application, the file sending method adapted to multiple hardware environments disclosed in the third aspect of this application, and the file receiving method adapted to multiple hardware environments disclosed in the fourth aspect of this application, and therefore will not be described one by one.

[0288] The computer device according to the embodiment of the present disclosure includes a memory and a processor. The memory is used to store non-temporary computer-readable instructions. Specifically, the memory may include one or more computer program products, and the computer program product may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory may, for example, include a random access memory (RAM) and / or a cache memory (cache), etc. The non-volatile memory may, for example, include a read-only memory (ROM), a hard disk, a flash memory, etc.

[0289] The processor may be a central processing unit (CPU) or other forms of processing units with data processing capabilities and / or instruction execution capabilities, and may control other components in the computer device to perform desired functions. In one embodiment of the present disclosure, the processor is used to run the computer-readable instructions stored in the memory, so that the computer device executes all or part of the steps of the file synchronization method adapted to multiple hardware environments, the file sending method adapted to multiple hardware environments, or the file receiving method adapted to multiple hardware environments described in the aforementioned embodiments of the present disclosure.

[0290] Those skilled in the art should be able to understand that in order to solve the technical problem of how to obtain a good user experience, the present embodiment may also include well-known structures such as a communication bus and an interface, and these well-known structures should also be included in the protection scope of the present disclosure.

[0291] like Fig.10 A schematic diagram of the structure of a computer device provided in an embodiment of the present disclosure is shown, which is a schematic diagram of the structure of a computer device suitable for implementing the embodiment of the present disclosure. Fig.10 The computer device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0292] like Fig.10 As shown, the computer device may include a processor (e.g., a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) or a program loaded from a storage device into a random access memory (RAM). In the RAM, various programs and data required for the operation of the computer device are also stored. The processor, ROM, and RAM are connected to each other via a bus. An input / output (I / O) interface is also connected to the bus.

[0293] Typically, the following devices can be connected to the I / O interface: input devices such as sensors or visual information acquisition devices; output devices such as display screens; storage devices such as tapes, hard disks, etc.; and communication devices. The communication device can allow the computer device to communicate with other devices (such as edge computing devices) wirelessly or by wire to exchange data. Fig.10 A computer device having various devices is shown, but it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed instead.

[0294] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a non-transitory computer-readable medium, and the computer program contains a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device, or installed from a storage device, or installed from a ROM. When the computer program is executed by a processor, all or part of the steps of the file synchronization method adapted to multiple hardware environments, or the file sending method adapted to multiple hardware environments, or the file receiving method adapted to multiple hardware environments of the embodiment of the present disclosure are executed.

[0295] For detailed description of this embodiment, reference may be made to the corresponding descriptions in the aforementioned embodiments, which will not be repeated here.

[0296] According to the computer-readable storage medium of the embodiment of the present disclosure, non-transitory computer-readable instructions are stored thereon. When the non-transitory computer-readable instructions are executed by the processor, all or part of the steps of the file synchronization method adapted to multiple hardware environments, the file sending method adapted to multiple hardware environments, or the file receiving method adapted to multiple hardware environments described in the aforementioned embodiments of the present disclosure are executed.

[0297] The above-mentioned computer-readable storage media include, but are not limited to: optical storage media (e.g., CD-ROM and DVD), magneto-optical storage media (e.g., MO), magnetic storage media (e.g., magnetic tape or mobile hard disk), media with built-in rewritable non-volatile memory (e.g., memory card) and media with built-in ROM (e.g., ROM box).

[0298] For detailed description of this embodiment, reference may be made to the corresponding descriptions in the aforementioned embodiments, which will not be repeated here.

[0299] The basic principles of the present disclosure are described above in conjunction with specific embodiments. However, it should be noted that the advantages, strengths, effects, etc. mentioned in the present disclosure are only examples and not limitations, and it cannot be considered that these advantages, strengths, effects, etc. are required by each embodiment of the present disclosure. In addition, the specific details disclosed above are only for the purpose of illustration and ease of understanding, and are not limitations. The above details do not limit the present disclosure to the necessity of adopting the above specific details to be implemented.

[0300] In the present disclosure, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply that there is any such actual relationship or order between these entities or operations. The block diagrams of the devices, devices, equipment, and systems involved in the present disclosure are only illustrative examples and are not intended to require or imply that they must be connected, arranged, and configured in the manner shown in the block diagram. As will be appreciated by those skilled in the art, these devices, devices, equipment, and systems can be connected, arranged, and configured in any manner. Words such as "including", "comprising", "having", etc. are open words, referring to "including but not limited to", and can be used interchangeably with them. The words "or" and "and" used here refer to the words "and / or" and can be used interchangeably with them, unless the context clearly indicates otherwise. The words "such as" used here refer to the phrase "such as but not limited to", and can be used interchangeably with them.

[0301] Additionally, as used herein, "or" used in a list of items beginning with "at least one" indicates a separate list, so that, for example, a list of "at least one of A, B, or C" means A or B or C, or AB or AC or BC, or ABC (i.e., A and B and C). Furthermore, the word "exemplary" does not mean that the example described is preferred or better than other examples.

[0302] It should also be noted that in the system and method of the present disclosure, each component or each step can be decomposed and / or recombined. Such decomposition and / or recombination should be regarded as equivalent solutions of the present disclosure.

[0303] Various changes, substitutions, and modifications of the techniques described herein may be made without departing from the teachings defined by the appended claims. Furthermore, the scope of the claims of the present disclosure is not limited to the specific aspects of the processes, machines, manufactures, compositions of events, means, methods, and actions described above. Currently existing or later to be developed processes, machines, manufactures, compositions of events, means, methods, or actions that perform substantially the same functions or achieve substantially the same results as the corresponding aspects described herein may be utilized. Thus, the appended claims include such processes, machines, manufactures, compositions of events, means, methods, or actions within their scope.

[0304] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use the present disclosure. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other aspects without departing from the scope of the present disclosure. Therefore, the present disclosure is not intended to be limited to the aspects shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

[0305] The above description has been given for the purpose of illustration and description. In addition, this description is not intended to limit the embodiments of the present disclosure to the forms disclosed herein. Although multiple example aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, changes, additions and sub-combinations thereof.

Claims

1. A file synchronization method adapted to various hardware environments, characterized in that: include: Acquire the sender information, wherein the sender information includes the sender disk type, the sender disk storage information, the sender memory information, and the sender CPU information; Performing a preliminary analysis based on the sending end information to determine the initial number of core threads of the sending end and the single sending volume of the sending queue; Based on the total data volume of the files to be synchronized and a preset dynamic adjustment strategy, dynamically adjust the initial number of core threads of the sending end to obtain the target number of core threads of the sending end; Creating the same number of Socket sending threads based on the target core thread number of the sending end, and sending the to-be-synchronized file to the receiving end based on the Socket sending threads and the single sending amount; Acquire receiving end information, wherein the receiving end information includes receiving end disk type, receiving end disk storage information, receiving end core thread number, and receiving end CPU information; Acquire a single reception quantity threshold of the receiving end based on the receiving end information; Based on the single receiving quantity threshold of the receiving end and the actual sending quantity of the to-be-synchronized files, the corresponding writing strategy is called to write the to-be-synchronized files into the disk corresponding to the receiving end.

2. The file synchronization method adapted to various hardware environments according to claim 1, characterized in that: The pre-analysis based on the sending end information to determine the initial number of core threads of the sending end and the single sending amount of the sending queue includes: Based on the sending end disk type, a core thread number setting strategy of a corresponding sending end thread pool is called; Setting the initial number of core threads of the sending end based on the core thread number setting strategy; Determine a single sending amount of a sending queue based on the sending end memory information and the sending end disk storage information; If the sending end disk type is the first type, obtaining the initial number of core threads of the sending end thread pool based on the first strategy; If the sending end disk type is the second type, obtaining the initial number of core threads of the sending end thread pool based on the second strategy; If the sending end disk type is the third type, the initial number of core threads of the sending end thread pool is obtained based on the third strategy.

3. The file synchronization method adapted to various hardware environments according to claim 2, characterized in that: The first type is a mechanical disk; The obtaining of the initial number of core threads of the sending end thread pool based on the first strategy includes: Acquire the CPU size of the sending end based on the sending end CPU information; Determine the initial number of core threads of the sending end thread pool based on the sending end CPU size; The initial number of core threads is N0: N0=n0, where n0 is the CPU size of the sending end.

4. The file synchronization method adapted to various hardware environments according to claim 2, characterized in that: The second type is a solid state drive; The obtaining of the initial number of core threads of the sending end thread pool based on the second strategy includes: Acquire the CPU size of the sending end based on the sending end CPU information; Determine the initial number of core threads of the sending end thread pool based on the sending end CPU size; The initial number of core threads is N0: N0=n0×n1, where n0 is the CPU size of the sending end, and 4≤n1≤16.

5. The file synchronization method adapted to various hardware environments according to claim 2, characterized in that: The third type is a RAID disk; The obtaining of the initial number of core threads of the sending end thread pool based on the third strategy includes: Acquire the CPU size of the sending end based on the sending end CPU information; Determine the initial number of core threads of the sending end thread pool based on the sending end CPU size; The initial number of core threads is N0: N0=n0×n2, where n0 is the CPU size of the sending end, and 2≤n2≤8.

6. The file synchronization method adapted to various hardware environments according to claim 2, characterized in that: The single transmission information is Q1: Q1 = F2 ÷ F1 × F0; Among them, F2 is the disk storage information of the sending end, F1 is the memory information of the sending end, and F0 is the CPU information of the sending end.

7. The file synchronization method adapted to various hardware environments according to claim 1, characterized in that: The method of dynamically adjusting the initial number of core threads of the sending end based on the total data volume of the files to be synchronized and the preset dynamic adjustment strategy to obtain the target number of core threads of the sending end includes: Determine a data synchronization mode based on the total data volume of the files to be synchronized. Based on the data synchronization mode and the sending end disk type, a corresponding dynamic adjustment strategy for the number of core threads on the sending end is called; In response to the dynamic adjustment strategy of the number of core threads at the sending end, the initial number of core threads at the sending end is dynamically adjusted to obtain the target number of core threads at the sending end.

8. The file synchronization method adapted to various hardware environments according to claim 7, characterized in that: The data synchronization mode YY+242255P is determined based on the total data volume of the files to be synchronized. formula, including: If the total data volume of the files to be synchronized meets the first range, the data synchronization mode is determined to be the high-efficiency mode; If the total data volume of the files to be synchronized meets the second range, determining that the data synchronization mode is a stable mode; If the total data volume of the files to be synchronized meets the third range, the data synchronization mode is determined to be the general mode; If the total data volume of the files to be synchronized meets the fourth range, determining that the data synchronization mode is a safe mode; The first range is M1, 1KB≤M1≤2MB; The second range is M2, 2MB<M2≤50MB; The third range is M3, 50MB<M3≤200MB The fourth range is M4, 200MB<M4.

9. The file synchronization method adapted to various hardware environments according to claim 8, characterized in that: The dynamically adjusting strategy of the number of core threads of the sending end corresponding to the calling of the data synchronization mode and the sending end disk type includes: When the data synchronization mode is the high-efficiency mode and the sending end disk type is a mechanical disk, calling the corresponding sending end core thread number dynamic adjustment strategy as the first expansion strategy; When the data synchronization mode is the high-efficiency mode and the sending end disk type is a solid-state hard disk, calling the corresponding sending end core thread number dynamic adjustment strategy as the second expansion strategy; When the data synchronization mode is the high-efficiency mode and the sending end disk type is a RAID disk, calling the corresponding sending end core thread number dynamic adjustment strategy as the third expansion strategy; When the data synchronization mode is the safe mode and the sending end disk type is a mechanical disk, calling the corresponding sending end core thread number dynamic adjustment strategy as the first reduction strategy; When the data synchronization mode is the safe mode and the sending end disk type is a solid state drive, calling the corresponding sending end core thread number dynamic adjustment strategy as the second reduction strategy; When the data synchronization mode is the safe mode and the sending end disk type is a RAID disk, the corresponding sending end core thread number dynamic adjustment strategy is called as the third reduction strategy.

10. The file synchronization method adapted to various hardware environments according to claim 9, characterized in that: The dynamically adjusting the initial number of core threads of the sending end in response to the dynamic adjustment strategy of the number of core threads of the sending end to obtain the target number of core threads of the sending end includes: When the data synchronization mode is the high-efficiency mode and the sending end disk type is a mechanical disk, the initial number of core threads of the sending end is increased by H1 times, and the increased initial number of core threads of the sending end is used as the target number of core threads of the sending end; 2≤H1≤3; When the data synchronization mode is the high-efficiency mode and the sending end disk type is a solid-state hard disk, the initial number of core threads of the sending end is increased by H2 times, and the increased initial number of core threads of the sending end is used as the target number of core threads of the sending end; 8≤H2≤9; When the data synchronization mode is the high-efficiency mode and the sending end disk type is a RAID disk, the initial number of core threads of the sending end is increased by H3 times, and the increased initial number of core threads of the sending end is used as the target number of core threads of the sending end; 4≤H3≤6; When the data synchronization mode is the safe mode and the sending end disk type is a mechanical disk, the initial number of core threads of the sending end is reduced by H1 times, and the reduced initial number of core threads of the sending end is used as the target number of core threads of the sending end; 2≤H1≤3; When the data synchronization mode is the safe mode and the sending end disk type is a solid state drive, the initial number of core threads of the sending end is reduced by H2 times, and the reduced initial number of core threads of the sending end is used as the target number of core threads of the sending end; 8≤H2≤9; When the data synchronization mode is the safety mode and the sending end disk type is a RAID disk, the initial number of core threads of the sending end is reduced by H3 times, and the reduced initial number of core threads of the sending end is used as the target number of core threads of the sending end; 4≤H3≤6.

11. The file synchronization method adapted to various hardware environments according to claim 1, characterized in that: The acquiring the threshold value of the number of single receptions by the receiving end based on the receiving end information includes: Based on the CPU information of the receiving end and the hardware performance information of the disk where the file is to be stored, the receiving end disk storage information and the number of core threads of the receiving end are obtained; Based on the acquired receiving end memory information and the receiving end disk storage information, a single receiving quantity threshold of the receiving end is acquired; The single receiving quantity threshold of the receiving end is Q2: Q2=J2÷J1×J0; wherein J2 is the disk storage information of the receiving end, J1 is the memory information of the receiving end, and J0 is the CPU information of the receiving end.

12. The file synchronization method adapted to various hardware environments according to claim 11, characterized in that: The calling of a corresponding write strategy to write the to-be-synchronized file into a disk corresponding to the receiving end based on the single-time receiving quantity threshold of the receiving end and the actual sending quantity of the to-be-synchronized file includes: If the actual number of files to be synchronized is not greater than the threshold of the number of files received at the receiving end at a time, obtain the size of each file actually sent, and based on the size of each file and the number of core threads at the receiving end, call the corresponding write strategy to write the files to be synchronized to the corresponding disk of the receiving end. plate; If the actual number of files to be synchronized is greater than the threshold of the number of files to be received at a single time by the receiving end, the files to be synchronized that do not exceed the threshold of the number of files to be received at a single time by the receiving end are recorded as a first group of files, and the files to be synchronized that exceed the threshold of the number of files to be received at a single time by the receiving end are recorded as a second group of files, and the second group of files are stored in the queue to be synchronized; Obtaining the size of each file in the first group of files, and based on the size of each file and the number of core threads of the receiving end, calling a corresponding write strategy to write the first group of files into a disk corresponding to the receiving end; After all the first group of files are written to the corresponding disk, the size of each file in the second group of files is obtained, and based on the size of each file and the number of core threads of the receiving end, the corresponding write strategy is called to write the second group of files to the corresponding disk of the receiving end.

13. The file synchronization method adapted to various hardware environments according to claim 12, characterized in that: The method of calling a corresponding write strategy based on the size of each file and the number of core threads of the receiving end to write the to-be-synchronized file to a disk corresponding to the receiving end includes: Record files whose file sizes exceed a preset size as first-class files, and record files whose file sizes do not exceed a preset size as second-class files, and obtain the total number of the first-class files and the second-class files respectively; If the total number of the first type of files and the second type of files are both less than the number of core threads of the receiving end, calling the first writing strategy to write the files to be synchronized into the disk corresponding to the receiving end; If the total number of the first type of files and the second type of files is greater than the number of core threads of the receiving end, calling the second writing strategy to write the files to be synchronized into the disk corresponding to the receiving end; If the total number of the first type of files and the second type of files are both equal to the number of core threads of the receiving end, calling the third writing strategy to write the files to be synchronized into the disk corresponding to the receiving end; If the total number of the first type of files is greater than the number of core threads of the receiving end, and the total number of the second type of files is equal to the number of core threads of the receiving end, calling the fourth writing strategy to write the files to be synchronized to the disk corresponding to the receiving end; If the total number of the first type of files is greater than the number of core threads of the receiving end, and the total number of the second type of files is less than the number of core threads of the receiving end, calling the fifth writing strategy to write the files to be synchronized to the disk corresponding to the receiving end; If the total number of the first type of files is equal to the number of core threads of the receiving end, and the total number of the second type of files is greater than the number of core threads of the receiving end, calling the sixth writing strategy to write the files to be synchronized to the disk corresponding to the receiving end; If the total number of the first type of files is equal to the number of core threads of the receiving end, and the total number of the second type of files is less than the number of core threads of the receiving end, calling the seventh writing strategy to write the files to be synchronized to the disk corresponding to the receiving end; If the total number of the first type of files is less than the number of core threads of the receiving end, and the total number of the second type of files is equal to the number of core threads of the receiving end, calling the eighth writing strategy to write the files to be synchronized to the disk corresponding to the receiving end; If the total number of the first type of files is less than the number of core threads of the receiving end, and the total number of the second type of files is greater than the number of core threads of the receiving end, the ninth writing strategy is called to write the files to be synchronized into the disk corresponding to the receiving end.

14. The file synchronization method adapted to various hardware environments according to claim 11, characterized in that: The method for obtaining the initial number of core threads of the receiving end includes: Acquire the CPU size of the receiving end based on the CPU information of the receiving end; Determine the type of the receiving end disk based on the hardware performance information of the disk where the file is to be stored; Determine the initial number of core threads of the receiving end based on the CPU size of the receiving end and the disk type of the receiving end; When the receiving end disk type is a mechanical disk, the initial number of core threads of the receiving end is consistent with the size of the receiving end CPU; When the receiving end disk type is a solid state drive, the initial number of core threads of the receiving end is M1 times the size of the receiving end CPU; When the receiving end disk type is a RAID disk, the initial number of core threads of the receiving end is M2 times the size of the receiving end CPU; M1≥M2.

15. A method for sending files in multiple hardware environments, applicable to sending files at a sending end in file synchronization in multiple hardware environments, characterized in that: include: Acquire the sender information, wherein the sender information includes the sender disk type, the sender disk storage information, the sender memory information, and the sender CPU information; Performing a preliminary analysis based on the sending end information to determine the initial number of core threads of the sending end and the single sending volume of the sending queue; Based on the total data volume of the files to be synchronized and a preset dynamic adjustment strategy, dynamically adjust the initial number of core threads of the sending end to obtain the target number of core threads of the sending end; The same number of Socket sending threads is created based on the target core thread number of the sending end, and the files to be synchronized are sent to the receiving end based on the Socket sending threads and the single sending amount.

16. A file receiving method for multiple hardware environments, suitable for receiving files at a receiving end in file synchronization in multiple hardware environments, characterized in that: include: Acquire receiving end information, wherein the receiving end information includes receiving end disk type, receiving end disk storage information, receiving end core thread number, and receiving end CPU information; Acquire a single reception quantity threshold of the receiving end based on the receiving end information; Based on the single receiving quantity threshold of the receiving end and the actual sending quantity of the to-be-synchronized files, the corresponding writing strategy is called to write the to-be-synchronized files into the disk corresponding to the receiving end.

17. A computer device, characterized in that: The computer device comprises: at least one processor; and, a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the file synchronization method adapted to multiple hardware environments as described in any one of claims 1-14, or the file sending method adapted to multiple hardware environments as described in claim 15, or the file receiving method adapted to multiple hardware environments as described in claim 16.

18. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, which are used to enable a computer to execute the file synchronization method adapted to multiple hardware environments as described in any one of claims 1-14, or the file sending method adapted to multiple hardware environments as described in claim 15, or the file receiving method adapted to multiple hardware environments as described in claim 16.

19. A computer program product comprising computer instructions, characterized in that When the computer instruction is executed by the processor, the file synchronization method adapted to multiple hardware environments described in any one of claims 1 to 14 or the file synchronization method adapted to multiple hardware environments described in claim 15 is implemented. The method for sending a file, or the steps of the method for receiving a file adapted to a variety of hardware environments as described in claim 16.

Images