Model file synchronization method, server, and storage medium
By using fragmentation processing and configuration file generation, combined with object storage and caching devices, efficient concurrent transmission of model files across regions was achieved, solving the problem of single-link bandwidth limitation and improving transmission speed and efficiency.
Patent Information
- Application Number
- CN202610384355.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-03-26
- Publication Date
- 2026-06-12
Smart Images
Figure CN122204871A_ABST
Abstract
Description
Technical Field
[0001] This disclosure relates to the field of data synchronization technology, and in particular to a model file synchronization method, server, and storage medium. Background Technology
[0002] Currently, large models are widely used in AI training, inference processes, and downstream engineering scenarios. These large models typically need to be shared and distributed across regions.
[0003] Currently, cross-regional data synchronization typically relies on object storage services provided by cloud service providers. While object storage services support cross-regional synchronization mechanisms, they face the limitation of bandwidth limits for single-link transmission, which in turn restricts transmission speed.
[0004] Therefore, there is an urgent need for a model file synchronization method that can transmit model files concurrently and improve overall transmission efficiency. Summary of the Invention
[0005] To solve the above-mentioned technical problems, or at least partially solve them, this disclosure provides a model file synchronization method, a server, and a storage medium.
[0006] A first aspect of this disclosure provides a model file synchronization method applied to a first server, the method comprising: The first device acquires the model file to be uploaded, and the model file to be uploaded is divided into multiple fragment files according to a preset size, and the file checksum of the fragment files is calculated; wherein, the first device is configured in the first server; the first device is the device for acquiring the model file to be uploaded; the first server is the source device for the model file to be uploaded; The first device generates a configuration file based on the file information of the model file to be uploaded, the file verification code of the segmented file, and the address information of the segmented file; The configuration file and the plurality of fragmented files are uploaded to a first object storage device via a first device, and the plurality of fragmented files are transmitted to a second object storage device in a second server via the first object storage device; wherein, the second server is the target device of the model file to be uploaded; the second server and the first server are different devices.
[0007] A second aspect of this disclosure provides a model file synchronization method applied to a second server, the method comprising: The third device retrieves the configuration file from the second object storage device at a preset frequency, parses the configuration file, and obtains the file checksums of multiple fragment files and the address information of the multiple fragment files; wherein, the second server is the target device of the model file to be uploaded; the third device is configured in the second server; the third device is the receiving device of the model file to be uploaded; In response to a segmented download request initiated by the third device, the multiple segmented files are retrieved from the second cache device at a preset frequency according to the segmented download request and the address information of the segmented files; wherein the second cache device is deployed within a preset geographical location range of the third device; The multiple fragment files are verified based on their file checksums to obtain a third verification result.
[0008] A third aspect of this disclosure provides a model file synchronization apparatus applied to a first server, the apparatus comprising: The processing module is used to acquire the model file to be uploaded through the first device, and to divide the model file to be uploaded into multiple fragment files according to a preset size, and to calculate the file verification code of the fragment files; wherein, the first device is configured in the first server; the first device is the device for acquiring the model file to be uploaded; the first server is the source device for the model file to be uploaded; The first generation module is used to generate a configuration file by means of a first device based on the file information of the model file to be uploaded, the file verification code of the segment file and the address information of the segment file; The transmission module is used to upload the configuration file and the plurality of fragmented files to a first object storage device via a first device, and to transmit the plurality of fragmented files to a second object storage device in a second server via the first object storage device; wherein, the second server is the target device of the model file to be uploaded; the second server and the first server are different devices.
[0009] A fourth aspect of this disclosure provides a model file synchronization apparatus applied to a second server, the apparatus comprising: The first acquisition module is used to acquire a configuration file from the second object storage device at a preset frequency through a third device, and parse the configuration file to obtain the file checksum of multiple fragment files and the address information of the multiple fragment files; wherein, the second server is the target device of the model file to be uploaded; the third device is configured in the second server; the third device is the receiving device of the model file to be uploaded; The second acquisition module is used to respond to the segmented download request initiated by the third device, and acquire the plurality of segmented files from the second cache device at a preset frequency according to the segmented download request and the address information of the segmented files; wherein the second cache device is deployed within a preset geographical location range of the third device; The verification module is used to verify the multiple fragment files according to the file checksum of the fragment files to obtain a third verification result.
[0010] A fifth aspect of this disclosure provides a server comprising a processor and a memory, wherein the memory stores a computer program that, when executed by the processor, performs the methods described in the first or second aspect.
[0011] A sixth aspect of this disclosure provides a computer-readable storage medium storing a computer program that, when executed by a processor, can implement the methods of the first or second aspect described above.
[0012] This disclosure provides a model file synchronization method, server, and storage medium. The method includes: acquiring a model file to be uploaded via a first device, and splitting the model file to be uploaded into multiple fragment files according to a preset size, and calculating the file checksum of each fragment file; generating a configuration file via the first device based on the file information of the model file to be uploaded, the file checksums of the fragment files, and the address information of the fragment files; uploading the configuration file and the multiple fragment files to a first object storage device via the first device, and transmitting the multiple fragment files to a second object storage device in a second server via the first object storage device. This technical solution enables concurrent transmission of model files, improving overall transmission efficiency. Attached Figure Description
[0013] The accompanying drawings, which are incorporated in and form a part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure.
[0014] To more clearly illustrate the technical solutions in the embodiments of this disclosure or the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0015] Figure 1 This is a flowchart illustrating a model file synchronization method provided in an embodiment of this disclosure; Figure 2This is a schematic diagram illustrating the transmission of model files between different servers according to an embodiment of this disclosure; Figure 3 This is a flowchart illustrating a model file synchronization method provided in an embodiment of this disclosure; Figure 4 This is a flowchart illustrating a model file synchronization method provided in an embodiment of this disclosure; Figure 5 This is a schematic diagram of the structure of a model file synchronization device provided in an embodiment of this disclosure; Figure 6 This is a schematic diagram of the structure of a model file synchronization device provided in an embodiment of this disclosure; Figure 7 This is a schematic diagram of the structure of a server according to an embodiment of this disclosure. Detailed Implementation
[0016] To better understand the above-mentioned objectives, features, and advantages of this disclosure, the solutions disclosed herein will be further described below. It should be noted that, unless otherwise specified, the embodiments and features described herein can be combined with each other.
[0017] Numerous specific details are set forth in the following description in order to provide a full understanding of this disclosure, but this disclosure may also be implemented in other ways different from those described herein; obviously, the embodiments in the specification are only some, and not all, of the embodiments of this disclosure.
[0018] Figure 1 This is a flowchart illustrating a model file synchronization method provided in this embodiment of the disclosure. The method is applied to a first server and can be executed by a server. This server can be executively understood as a device such as a mobile phone, tablet computer, laptop computer, desktop computer, or smart TV. Figure 1 As shown, the method provided in this embodiment includes the following steps: S101. Obtain the model file to be uploaded through the first device, and divide the model file to be uploaded into multiple fragment files according to the preset size, and calculate the file check code of the fragment files.
[0019] In this embodiment, please refer to Figure 2 This diagram illustrates the transfer of model files between different servers. Figure 2As can be seen, the model file to be uploaded is transmitted from the first server to the second server. The first server includes a first device, a second device, a first object storage device, and a first cache device. The second server includes a third device, a second object storage device, and a second cache device. Specifically, the first device is configured within the first server; it is the device that acquires the model file to be uploaded; the first server is the source device for the model file to be uploaded. The second device is configured within the first server; it is the receiving device for the model file to be uploaded; the first cache device is deployed within a preset geographical location range of the second device; and the second server is the target device for the model file to be uploaded. The third device is configured within the second server; it is the receiving device for the model file to be uploaded; and the second cache device is deployed within a preset geographical location range of the third device.
[0020] In one example, the model file to be uploaded is a model file used for training or inference. The preset size can be 4MB. After the first device obtains the model file to be uploaded, the total size of the model file to be uploaded is divided into multiple fragment files according to the preset size. Specifically, it can be divided into average fragments, or it can be divided into fragments with the preset size as the maximum value, resulting in multiple fragment files. For example, if the model file to be uploaded is M, after being divided into fragments, it will result in multiple fragment files A, B, and C.
[0021] Furthermore, calculate the file checksum for file fragment A, file fragment B, and file fragment C. Specifically, the file checksum can be a CRC32 value. CRC32 is a 32-bit algorithm for cyclic redundancy check, a commonly used error detection technique in data communication, which verifies data integrity through a generator polynomial.
[0022] S102. The first device generates a configuration file based on the file information of the model file to be uploaded, the file checksum of the segmented file, and the address information of the segmented file.
[0023] In one example, the file information of the model file to be uploaded includes the model file name, the total size of the model file, and the address information of the model file. In this embodiment, the configuration file is generated from the file information of the model file to be uploaded, the file checksum of the fragment files, and the address information of the fragment files. This configuration file can be a Manifest file. The address information of the fragment files refers to the address information of each individual fragment file. For example, the address information of fragment file A, the address information of fragment file B, and the address information of fragment file C.
[0024] S103. Upload the configuration file and multiple fragment files to the first object storage device through the first device, and transfer the multiple fragment files to the second object storage device in the second server through the first object storage device.
[0025] In one example, the first object storage device is used to store configuration files and multiple fragmented files, and the first object storage device is also used to transfer multiple fragmented files with the second object storage device. The advantage of this setup is that since fragmented files are transferred between the first and second object storage devices, it is not limited by the bandwidth of the transfer, and because multiple fragmented files are transferred in parallel, the transfer efficiency is high.
[0026] This disclosure provides a model file synchronization method, which includes: acquiring a model file to be uploaded via a first device, and splitting the model file to be uploaded into multiple fragment files according to a preset size, and calculating the file checksum of the fragment files; generating a configuration file via the first device based on the file information of the model file to be uploaded, the file checksums of the fragment files, and the address information of the fragment files; uploading the configuration file and the multiple fragment files to a first object storage device via the first device, and transmitting the multiple fragment files to a second object storage device in a second server via the first object storage device. This technical solution enables concurrent transmission of model files, improving overall transmission efficiency.
[0027] Figure 3 A flowchart illustrating a model file synchronization method provided in an embodiment of this disclosure is shown. This model file synchronization method is applied to a first server. This embodiment of the disclosure is an optimization based on the above embodiments, and can be combined with various optional solutions from one or more of the above embodiments.
[0028] like Figure 3 As shown, the model file synchronization method may include the following steps: S301. Obtain the model file to be uploaded through the first device, and divide the model file to be uploaded into multiple fragment files according to the preset size, and calculate the file check code of the fragment files.
[0029] In one example, this step can be found in step S101, and will not be repeated here.
[0030] S302. The first device generates a configuration file based on the file information of the model file to be uploaded, the file checksum of the segmented file, and the address information of the segmented file.
[0031] In one example, this step can be found in step S102, and will not be repeated here.
[0032] S303. Store the configuration file in the first object storage device according to the address information corresponding to the configuration file.
[0033] In one example, the address information corresponding to the configuration file refers to the location where the configuration file is stored in the first object storage device.
[0034] S304. Store multiple fragmented files in the first object storage device according to the address information of the fragmented files.
[0035] In one example, the address information of the fragmented file refers to the address information corresponding to each fragmented file, and each fragmented file is stored in the first object storage device according to the address information of the corresponding fragmented file.
[0036] S305. Obtain the fragment priority corresponding to multiple fragment files.
[0037] In one example, each file fragment has a corresponding fragment priority, which is used to characterize the transmission priority of the file fragment. For example, file fragment A has a fragment priority of 1, file fragment B has a fragment priority of 2, and file fragment C has a fragment priority of 2. Among them, the transmission priority of fragment priority 1 is greater than the transmission priority of fragment priority 2.
[0038] S306. Based on the fragmentation priority, transmit multiple fragmented files sequentially to the second object storage device in the second server.
[0039] For example, fragment file A has a fragmentation priority of 1, fragment file B has a fragmentation priority of 2, and fragment file C has a fragmentation priority of 2. The transmission priority of fragment file A with a fragmentation priority of 1 is higher than that of fragment file C with a fragmentation priority of 2. Specifically, fragment file A can be transmitted to the second object storage device in the second server first, and then fragment files B and C can be transmitted to the second object storage device in the second server.
[0040] In one example, the method also includes: The second device retrieves the configuration file from the first object storage device at a preset frequency, parses the configuration file, and obtains the file checksum and address information of the fragmented file. In response to a fragment download request initiated by the second device, multiple fragment files are retrieved from the first cache device at a preset frequency according to the fragment download request and the address information of the fragment files; The first verification result is obtained by verifying multiple fragment files based on their file checksums.
[0041] In one example, the preset frequency could be once per minute. The second device queries the first object storage device at the preset frequency. If a configuration file exists, it retrieves the configuration file, parses it, and obtains the file checksum and address information of the fragment files.
[0042] In some specific embodiments, in response to a fragment download request initiated by the second device, after receiving the fragment download request, multiple fragment files are retrieved from the first cache device according to the address information of the fragment files at a preset frequency. Then, the associated fragment files are verified according to the file checksum of each fragment file to obtain a first verification result. The first verification result is used to characterize the integrity of the fragment files.
[0043] In one example, multiple fragment files are retrieved from the first cache device at a preset frequency based on the fragment download request and the address information of the fragment files, including: Based on the fragment download request, multiple fragment files are detected from the first cache device. If they exist, multiple fragment files are retrieved from the first cache device at a preset frequency. If it does not exist, the corresponding fragment file is retrieved from the first object storage device through the first cache device based on the address information of the fragment file.
[0044] In one example, multiple fragment files are first detected from the first cache device based on the fragment download request. If multiple fragment files exist in the first cache device, it means that the first cache device has obtained the corresponding fragment files from the first object storage device.
[0045] If multiple fragment files do not exist in the first cache device, the corresponding fragment file needs to be retrieved from the first object storage device based on the address information of the fragment file.
[0046] In one example, after verifying multiple fragment files based on their file checksums and obtaining the first verification result, the method further includes: If the first verification result indicates that multiple fragment files are complete, then the target model file is generated based on the multiple fragment files; Calculate the target checksum of the model file to be uploaded based on the file checksum of the fragmented files; The target model file is verified based on the target checksum to obtain the second verification result; If the second verification result indicates that the target model file is complete, a normal notification message is sent. If the second verification result indicates that the target model file is incomplete, an exception notification message is sent.
[0047] In one example, if the first verification result indicates that multiple fragment files are complete, then the completeness of the target model file composed of these fragment files needs to be determined. Specifically, the target checksum of the model file to be uploaded is calculated based on the file checksums of the fragment files. This target checksum represents the overall checksum of the model file to be uploaded. The current checksum of the target model file is calculated, and then compared with the target checksum to obtain the second verification result. If the second verification result indicates that the target model file is complete, a normal notification message is sent to remind the first server that the target model file is complete. If the second verification result indicates that the target model file is incomplete, an exception notification message is sent to remind the first server that the target model file is incomplete.
[0048] This disclosure provides a model file synchronization method, which includes: storing a configuration file in a first object storage device according to the address information corresponding to the configuration file; storing multiple fragment files in the first object storage device according to the address information of the fragment files; obtaining the fragment priority corresponding to the multiple fragment files; and then transmitting the multiple fragment files sequentially to a second object storage device in a second server according to the fragment priority. This technical solution improves the cross-server transmission speed of model files.
[0049] Figure 4 A flowchart illustrating a model file synchronization method provided in an embodiment of this disclosure is shown. This model file synchronization method is applied to a second server. This embodiment of the disclosure is an optimization based on the above embodiments, and can be combined with various optional solutions from one or more of the above embodiments.
[0050] like Figure 4 As shown, the model file synchronization method may include the following steps: S401. The third device retrieves the configuration file from the second object storage device at a preset frequency, parses the configuration file, and obtains the file checksum of multiple fragment files and the address information of multiple fragment files.
[0051] In one example, the second server includes a third device, a second object storage device, and a second cache device. This step can be found in step S306 and will not be repeated here.
[0052] Step S306 is used to describe the model file synchronization process in the first server, while in this embodiment, it is used to describe the model file synchronization process in the second server.
[0053] S402. In response to a fragment download request initiated by a third device, obtain multiple fragment files from the second cache device at a preset frequency according to the fragment download request and the address information of the fragment files.
[0054] In one example, this step can be found in step S306, and will not be repeated here.
[0055] S403. Verify multiple fragment files based on their file checksums to obtain a third verification result.
[0056] In one example, after verifying multiple file fragments based on their file checksums and obtaining a third verification result, the method further includes: If the third verification result indicates that multiple fragment files are complete, then a target model file is generated based on the multiple fragment files; the target verification code of the model file to be uploaded is calculated based on the file verification codes of the fragment files; the target model file is verified based on the target verification code to obtain the fourth verification result; if the fourth verification result indicates that the target model file is complete, a normal notification message is sent; if the fourth verification result indicates that the target model file is incomplete, an exception notification message is sent.
[0057] This disclosure provides a model file synchronization method. The method first retrieves a configuration file from a second object storage device at a preset frequency using a third device, and parses the configuration file to obtain file checksums and address information for multiple fragment files. In response to a fragment download request initiated by the third device, multiple fragment files are retrieved from a second cache device at a preset frequency based on the download request and the fragment file address information. Finally, the multiple fragment files are verified using their file checksums to obtain a third verification result. This technical solution reduces the bandwidth pressure on object storage by using cache devices within a region, and the cache and download client are deployed nearby or on the same machine, significantly reducing download latency.
[0058] Figure 5 This is a schematic diagram of a model file synchronization device provided in an embodiment of this disclosure, applied to a first server. This model file synchronization device can be understood as the aforementioned server or a functional module within the aforementioned server. Figure 5 As shown, the model file synchronization device 50 includes: The processing module 501 is used to obtain the model file to be uploaded through the first device, and to divide the model file to be uploaded into multiple fragment files according to a preset size, and to calculate the file verification code of the fragment files. The first generation module 502 is used to generate a configuration file by the first device based on the file information of the model file to be uploaded, the file verification code of the fragment file and the address information of the fragment file; The transmission module 503 is used to upload the configuration file and multiple fragment files to the first object storage device through the first device, and to transmit the multiple fragment files to the second object storage device in the second server through the first object storage device.
[0059] In one example, the transmission module 503 is used to: store the configuration file in the first object storage device according to the address information corresponding to the configuration file; and store multiple fragment files in the first object storage device according to the address information of the fragment files.
[0060] In one example, the transmission module 503 is used to: obtain the fragment priority corresponding to multiple fragment files; and transmit the multiple fragment files sequentially to the second object storage device in the second server according to the fragment priority.
[0061] In one example, the model file synchronization device 50 includes: The parsing module 504 is used to: obtain a configuration file from the first object storage device at a preset frequency through the second device, and parse the configuration file to obtain the file checksum and address information of the segmented files; respond to the segmented download request initiated by the second device, obtain multiple segmented files from the first cache device at a preset frequency according to the segmented download request and the address information of the segmented files; and verify the multiple segmented files according to the file checksum of the segmented files to obtain a first verification result.
[0062] In one example, parsing module 504 is used for: Based on the segmented download request, multiple segmented files are detected from the first cache device. If they exist, multiple segmented files are retrieved from the first cache device at a preset frequency. If they do not exist, the corresponding segmented files are retrieved from the first object storage device based on the address information of the segmented files through the first cache device.
[0063] In one example, the model file synchronization device 50 includes: The second generation module 505 is used to generate a target model file based on the multiple fragment files if the first verification result indicates that the multiple fragment files are complete; calculate the target verification code of the model file to be uploaded based on the file verification code of the fragment files; verify the target model file based on the target verification code to obtain a second verification result; send a normal notification message if the second verification result indicates that the target model file is complete; and send an exception notification message if the second verification result indicates that the target model file is incomplete.
[0064] The apparatus provided in this embodiment can execute the methods of any of the above embodiments, and its execution method and beneficial effects are similar, so they will not be described again here.
[0065] Figure 6This is a schematic diagram of a model file synchronization device provided in an embodiment of this disclosure, applied to a second server. This model file synchronization device can be understood as the aforementioned server or a functional module within the aforementioned server. Figure 6 As shown, the model file synchronization device 60 includes: The first acquisition module 601 is used to acquire the configuration file from the second object storage device through the third device at a preset frequency, and parse the configuration file to obtain the file check code of multiple fragment files and the address information of multiple fragment files; The second acquisition module 602 is used to respond to the segment download request initiated by the third device and acquire multiple segment files from the second cache device at a preset frequency according to the segment download request and the address information of the segment files; The verification module 603 is used to verify multiple fragment files based on the file checksum of the fragment files and obtain a third verification result.
[0066] In one example, the model file synchronization device 60 includes: The third generation module 604 is used to generate a target model file based on the multiple fragment files if the third verification result indicates that the multiple fragment files are complete. The calculation module 605 is used to calculate the target check code of the model file to be uploaded based on the file check code of the fragmented file; The determination module 606 is used to verify the target model file based on the target check code and obtain the fourth verification result; The first sending module 607 is used to send a normal notification message if the fourth verification result indicates that the target model file is complete. The second sending module 608 is used to send an exception notification message if the fourth verification result indicates that the target model file is incomplete.
[0067] The apparatus provided in this embodiment can execute the methods of any of the above embodiments, and its execution method and beneficial effects are similar, so they will not be described again here.
[0068] This disclosure also provides a server, which includes: a memory storing a computer program; and a processor for executing the computer program, wherein when the computer program is executed by the processor, it can implement the methods of any of the above embodiments.
[0069] Example, Figure 7 This is a schematic diagram of the structure of a server according to an embodiment of this disclosure. See below for details. Figure 7The diagram illustrates a structural schematic suitable for implementing the server 1000 in the embodiments of this disclosure. The server 1000 in the embodiments of this disclosure may include, but is not limited to, mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), in-vehicle terminals (e.g., in-vehicle navigation terminals), and fixed terminals such as digital TVs and desktop computers. Figure 7 The server shown is merely an example and should not be construed as limiting the functionality and scope of use of the embodiments disclosed herein.
[0070] like Figure 7 As shown, server 1000 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 1001, which can perform various appropriate actions and processes according to a program stored in read-only memory (ROM) 1002 or a program loaded from storage device 1008 into random access memory (RAM) 1003. RAM 1003 also stores various programs and data required for the operation of server 1000. Processing device 1001, ROM 1002, and RAM 1003 are interconnected via bus 1004. Input / output (I / O) interface 1005 is also connected to bus 1004.
[0071] Typically, the following devices can be connected to I / O interface 1005: input devices 1006 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 1007 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 1008 including, for example, magnetic tapes, hard disks, etc.; and communication devices 1009. Communication device 1009 allows server 1000 to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 7 A server 1000 with various devices is shown, but it should be understood that it is not required to implement or have all of the devices shown. More or fewer devices may be implemented or have alternatively.
[0072] In particular, according to embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this disclosure include a computer program product comprising a computer program carried on a non-transitory computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication device 1009, or installed from storage device 1008, or installed from ROM 1002. When the computer program is executed by processing device 1001, it performs the functions defined in the methods of embodiments of this disclosure.
[0073] It should be noted that the computer-readable medium described in this disclosure can be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this disclosure, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In this disclosure, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium can be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wires, optical fibers, RF (radio frequency), etc., or any suitable combination thereof.
[0074] In some implementations, clients and servers can communicate using any currently known or future-developed network protocol such as HTTP (Hypertext Transfer Protocol) and can interconnect with digital data communication (e.g., communication networks) of any form or medium. Examples of communication networks include local area networks (“LANs”), wide area networks (“WANs”), the Internet (e.g., the Internet of Things), and peer-to-peer networks (e.g., ad hoc peer-to-peer networks), as well as any currently known or future-developed networks.
[0075] The aforementioned computer-readable medium may be included in the aforementioned server; or it may exist independently and not assembled into the server.
[0076] The aforementioned computer-readable medium carries one or more programs. When the server executes the aforementioned one or more programs, the server causes the server to: obtain the model file to be uploaded through the first device, and divide the model file to be uploaded into multiple fragment files according to a preset size, and calculate the file checksum of the fragment files; generate a configuration file through the first device based on the file information of the model file to be uploaded, the file checksum of the fragment files, and the address information of the fragment files; upload the configuration file and the multiple fragment files to the first object storage device through the first device, and transmit the multiple fragment files to the second object storage device in the second server through the first object storage device.
[0077] Computer program code for performing the operations of this disclosure can be written in one or more programming languages or a combination thereof, including but not limited to object-oriented programming languages such as Java, Smalltalk, and C++, as well as conventional procedural programming languages such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).
[0078] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.
[0079] The units described in the embodiments of this disclosure can be implemented in software or hardware. The names of the units are not, in some cases, intended to limit the specific unit.
[0080] The functions described above in this document can be performed at least in part by one or more hardware logic components. For example, exemplary types of hardware logic components that can be used, without limitation, include: field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), system-on-a-chip (SoCs), complex programmable logic devices (CPLDs), and so on.
[0081] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.
[0082] This disclosure also provides a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, it can implement the methods of any of the above embodiments. The execution method and beneficial effects are similar, and will not be described again here.
[0083] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes the element.
[0084] The above are merely specific embodiments of this disclosure, enabling those skilled in the art to understand or implement this disclosure. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this disclosure. Therefore, this disclosure is not to be limited to these embodiments, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.
Claims
1. A method for synchronizing model files, characterized in that, Applied to a first server, the method includes: The first device acquires the model file to be uploaded, and the model file to be uploaded is divided into multiple fragment files according to a preset size, and the file checksum of the fragment files is calculated; wherein, the first device is configured in the first server; the first device is the device for acquiring the model file to be uploaded; the first server is the source device for the model file to be uploaded; The first device generates a configuration file based on the file information of the model file to be uploaded, the file verification code of the segmented file, and the address information of the segmented file; The configuration file and the plurality of fragmented files are uploaded to a first object storage device via a first device, and the plurality of fragmented files are transmitted to a second object storage device in a second server via the first object storage device; wherein, the second server is the target device of the model file to be uploaded; the second server and the first server are different devices.
2. The method according to claim 1, characterized in that, Uploading the configuration file and the plurality of fragmented files to the first object storage device via the first device includes: The configuration file is stored in the first object storage device according to the address information corresponding to the configuration file; The multiple fragmented files are stored in the first object storage device according to the address information of the fragmented files.
3. The method according to claim 1, characterized in that, The step of transferring the plurality of fragmented files to the second object storage device in the second server via the first object storage device includes: Obtain the fragment priority corresponding to the multiple fragment files; The plurality of fragmented files are sequentially transmitted to the second object storage device in the second server according to the fragmentation priority.
4. The method according to claim 1, characterized in that, The method further includes: The configuration file is retrieved from the first object storage device by the second device at a preset frequency, and the configuration file is parsed to obtain the file checksum of the fragment file and the address information of the fragment file; wherein, the second device is configured in the first server; the second device is the receiving device for the model file to be uploaded; In response to a segmented download request initiated by the second device, the plurality of segmented files are retrieved from the first cache device at a preset frequency according to the segmented download request and the address information of the segmented files; wherein, the first cache device is deployed within a preset geographical location range of the second device; The multiple fragment files are verified based on their file checksums to obtain a first verification result.
5. The method according to claim 4, characterized in that, The step of retrieving the plurality of fragment files from the first cache device at a preset frequency according to the fragment download request and the address information of the fragment files includes: According to the segmented download request, the plurality of segmented files are detected from the first cache device. If they exist, the plurality of segmented files are retrieved from the first cache device at a preset frequency. If it does not exist, the corresponding segment file is obtained from the first object storage device through the first cache device according to the address information of the segment file.
6. The method according to claim 4, characterized in that, After verifying the plurality of fragmented files according to the file checksum of the fragmented files to obtain a first verification result, the method further includes: If the first verification result indicates that the multiple fragment files are complete, then a target model file is generated based on the multiple fragment files; Calculate the target checksum of the model file to be uploaded based on the file checksum of the segmented file; The target model file is verified based on the target verification code to obtain a second verification result; If the second verification result indicates that the target model file is complete, a normal notification message is sent. If the second verification result indicates that the target model file is incomplete, an exception notification message is sent.
7. A method for synchronizing model files, characterized in that, Applied to a second server, the method includes: The third device retrieves the configuration file from the second object storage device at a preset frequency, parses the configuration file, and obtains the file checksums of multiple fragment files and the address information of the multiple fragment files; wherein, the second server is the target device of the model file to be uploaded; the third device is configured in the second server; the third device is the receiving device of the model file to be uploaded; In response to a segmented download request initiated by the third device, the multiple segmented files are retrieved from the second cache device at a preset frequency according to the segmented download request and the address information of the segmented files; wherein the second cache device is deployed within a preset geographical location range of the third device; The multiple fragment files are verified based on their file checksums to obtain a third verification result.
8. The method according to claim 7, characterized in that, After verifying the plurality of fragmented files according to the file checksum of the fragmented files to obtain a third verification result, the method further includes: If the third verification result indicates that the multiple fragment files are complete, then a target model file is generated based on the multiple fragment files; Calculate the target checksum of the model file to be uploaded based on the file checksum of the fragmented files; The target model file is verified based on the target verification code to obtain a fourth verification result; If the fourth verification result indicates that the target model file is complete, a normal notification message is sent. If the fourth verification result indicates that the target model file is incomplete, an exception notification message is sent.
9. A server, characterized in that, include: A processor and a memory, wherein the memory stores a computer program that, when executed by the processor, performs the method of any one of claims 1-6 or any one of claims 7-8.
10. A computer-readable storage medium, characterized in that, The storage medium stores a computer program that, when executed by a processor, implements the method as claimed in any one of claims 1-6 or any one of claims 7-8.