Remote synchronization system and method and block transmission subsystem

By setting up a reliable forwarding device and a retransmitted block transmission subsystem in the remote synchronization system, the data transmission stability problem in different network environments is solved, the stable completion of Git clone operations is achieved, and the reliability of data transmission is ensured. Sex and efficiency.

CN119052260AActive Publication Date: 2024-11-29CCB FINTECH CO LTD
View PDF 8 Cites 0 Cited by

Patent Information

Application Number
CN202410984121.8
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-22
Publication Date
2024-11-29
Estimated Expiration
2044-07-22

AI Technical Summary

Technical Problem

When the network environments of the two source code warehouses are in different, the stability of data transmission between the two is poor, resulting in the interruption of Git clone operation due to network problems and the failure to retransmit breakpoints, affecting the stability of data transmission.

Method used

Design a remote synchronization system, by setting up reliable forwarding devices in the first and second network environments, and using a retransmitted block transmission subsystem, Git object data blocks are transmitted one by one, and when transmission fails Retransmission ensures the stability of data transmission.

Benefits of technology

Without changing the client and server, the stable completion of Git clone operations in unreliable networks is achieved, the problem of data transmission interruption caused by network instability is solved, and the reliability and efficiency of data transmission is improved.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119052260A_ABST
    Figure CN119052260A_ABST
Patent Text Reader

Abstract

The invention provides a remote synchronization system and method and a block transmission subsystem. Relates to the technical field of big data. The remote synchronization system comprises a first forwarding device, a second forwarding device and a block transmission subsystem, and the first forwarding device, a first source code warehouse and a part used for communicating with the first forwarding device in the block transmission subsystem are in a first network environment. The second forwarding device, the second source code warehouse and a part, used for communicating with the second forwarding device, in the block transmission subsystem are in a second network environment, and the first network environment is different from the second network environment. And the block transmission subsystem is used for transmitting the plurality of Git object data blocks to the first forwarding equipment one by one, and is also used for retransmitting the Git object data blocks which fail in transmission. In the remote synchronization system, stable data transmission between the client (the first source code warehouse) and the server (the second source code warehouse) is realized.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of big data technology, and in particular to a remote synchronization system, method and block transmission subsystem. Background Art

[0002] Git is a distributed version control system that can handle project version management efficiently and quickly. With Git, each developer can have a complete code repository locally, which not only speeds up the operation, but also allows development and version control without a network connection.

[0003] In the process of using Git, Git clone is a basic and commonly used command, which plays a vital role. With this command, developers can get a complete copy of the code from the remote repository and copy it to the local computer. This process is not just a simple file copy, it also includes obtaining all historical commit records, branches, and tag information of the remote repository. Locally, Git clone will create a new directory, which is a mirror of the remote repository and contains the complete history and structure of the project. In addition, the command will automatically set up the remote tracking branch, allowing developers to easily interact with the remote repository, such as pulling updates or pushing changes.

[0004] However, when the network environments of the two source code repositories are different, the stability of data transmission between the two is poor. Summary of the invention

[0005] The present application provides a remote synchronization system, method and block transmission subsystem to solve the problem in the prior art that when two source code repositories are located in different network environments, the stability of data transmission between the two is poor.

[0006] In a first aspect, the present application provides a remote synchronization system, comprising:

[0007] a first forwarding device, a second forwarding device, and a block transmission subsystem, wherein the first forwarding device, the first source code warehouse, and a portion of the block transmission subsystem used for communicating with the first forwarding device are in a first network environment, and the second forwarding device, the second source code warehouse, and a portion of the block transmission subsystem used for communicating with the second forwarding device are in a second network environment, and the first network environment and the second network environment are different;

[0008] The first forwarding device is used to forward data exchanged between the first source code repository and the block transmission subsystem, and the second forwarding device is used to forward data exchanged between the second source code repository and the block transmission subsystem;

[0009] The block transmission subsystem is used to transmit multiple Git object data blocks to the first forwarding device one by one, and is also used to retransmit the Git object data blocks that failed to transmit. The multiple Git object data blocks belong to target Git object data, and the target Git object data is the Git object data corresponding to the remote repository update request sent by the first source code repository to the second source code repository.

[0010] In one possible design, the block transmission subsystem includes:

[0011] A first transmission unit and a second transmission unit, the first transmission unit is in the first network environment, and the second transmission unit is in the second network environment;

[0012] The first transmission unit is used to send a first control message to the second transmission unit, where the first control message is used to indicate that the target Git object data will be transmitted in a data block manner;

[0013] The first transmission unit is further used to continuously send data packets to the second transmission unit, each data packet including the Git object data block;

[0014] The first transmission unit is further configured to send a second control message to the second transmission unit after all Git object data blocks have been sent, wherein the second control message is used to indicate that the transmission of the target Git object data has been completed;

[0015] The second transmission unit is used to send a response message to the first transmission unit after receiving the first control message, the second control message or the data message;

[0016] The second transmission unit is further configured to transmit the received Git object data block to the first forwarding device in a data stream manner.

[0017] In a possible design, the first transmission unit is specifically configured to:

[0018] For any data message, start timing after sending the data message to the first forwarding device to obtain the accumulated duration;

[0019] If the accumulated time reaches the preset time and the target response message is not obtained, the data message is resent to the first forwarding device, and the target response message is a response message corresponding to the data message.

[0020] In a possible design, the preset duration is pre-set according to a preset data length, and the preset data length is used to divide the target Git object data into multiple Git object data blocks.

[0021] In a possible design, the first forwarding device is further configured to:

[0022] Obtaining a target data block identifier corresponding to the read target Git object data block sent by the first source code repository;

[0023] The target data block identifier is sent to the block transmission subsystem.

[0024] In a possible design, the block transmission subsystem is further used for:

[0025] Obtaining a target data block identifier sent by the first forwarding device;

[0026] If the target Git object data block corresponding to the target data block identifier exists in the Git object data blocks that have not been transmitted, the transmission process of the target Git object data block is canceled.

[0027] In a second aspect, the present application provides a remote synchronization method, which is applied to a block transmission subsystem in a remote synchronization system as described in the first aspect and various possible designs of the first aspect, and the method includes:

[0028] Acquire multiple Git object data blocks corresponding to target Git object data, where the target Git object data is Git object data corresponding to a remote repository update request sent by the first source code repository to the second source code repository;

[0029] Transmitting the plurality of Git object data blocks to the first forwarding device one by one;

[0030] When the transmission of any Git object data block fails, the failed data block is retransmitted.

[0031] In a possible design, the block transmission subsystem includes a first transmission unit and a second transmission unit, and transmitting the plurality of Git object data blocks one by one to the first forwarding device includes:

[0032] Sending a first control message to the second transmission unit through the first transmission unit, where the first control message is used to indicate that the target Git object data will be transmitted in a data block manner;

[0033] Continuously sending data packets to the first forwarding device through the first transmission unit and the second transmission unit, each data packet including the Git object data block;

[0034] After all Git object data blocks are sent, a second control message is sent to the second transmission unit through the first transmission unit, where the second control message is used to indicate that the transmission of the target Git object data is completed.

[0035] In a possible design, when any Git object data block fails to be transmitted, retransmitting the failed data block includes:

[0036] For any data message, start timing after the data message is sent to the second transmission unit through the first transmission unit, and obtain the accumulated duration;

[0037] If the accumulated time reaches the preset time and the first transmission unit does not obtain the target response message, the data message is resent to the second transmission unit, and the target response message is a response message corresponding to the data message.

[0038] In a possible design, the preset duration is pre-set according to a preset data length, and the preset data length is used to divide the target Git object data into multiple Git object data blocks.

[0039] In one possible design, the method further includes:

[0040] Obtaining a target data block identifier, where the target data block identifier is a data block identifier corresponding to a target Git object data block that has been read by the first source code repository;

[0041] If the target Git object data block corresponding to the target data block identifier exists in the Git object data blocks that have not been transmitted, the transmission process of the target Git object data block is canceled.

[0042] In a third aspect, the present application provides a remote synchronization device, which is applied to a block transmission subsystem in a remote synchronization system as described in the first aspect and various possible designs of the first aspect, and the device includes:

[0043] An acquisition module, configured to acquire multiple Git object data blocks corresponding to target Git object data, wherein the target Git object data is Git object data corresponding to a remote repository update request sent by the first source code repository to the second source code repository;

[0044] A transmission module, used for transmitting the plurality of Git object data blocks to the first forwarding device one by one;

[0045] The transmission module is also used to retransmit the failed data block when the transmission of any Git object data block fails.

[0046] In one possible design, the transmission module includes: a first transmission unit and a second transmission unit;

[0047] The first transmission unit is used to send a first control message to the second transmission unit, where the first control message is used to indicate that the target Git object data will be transmitted in a data block manner;

[0048] The first transmission unit is further used to continuously send data packets to the second transmission unit, each data packet including the Git object data block;

[0049] The first transmission unit is further configured to send a second control message to the second transmission unit after all Git object data blocks have been sent, wherein the second control message is used to indicate that the transmission of the target Git object data has been completed;

[0050] The second transmission unit is used to send a response message to the first transmission unit after receiving the first control message, the second control message or the data message;

[0051] The second transmission unit is further configured to transmit the received Git object data block to the first forwarding device in a data stream manner.

[0052] In a possible design, the first transmission unit is specifically configured to:

[0053] For any data message, start timing after sending the data message to the second transmission unit to obtain the accumulated duration;

[0054] If the accumulated time reaches the preset time and the target response message is not obtained, the data message is resent to the second transmission unit, and the target response message is a response message corresponding to the data message.

[0055] In a possible design, the preset duration is pre-set according to a preset data length, and the preset data length is used to divide the target Git object data into multiple Git object data blocks.

[0056] In a possible design, the remote synchronization device further includes:

[0057] The acquisition module is further used to acquire a target data block identifier, where the target data block identifier is a data block identifier corresponding to a target Git object data block that has been read by the first source code repository;

[0058] The canceling module is used to cancel the transmission processing of the target Git object data block if the target Git object data block corresponding to the target data block identifier exists in the Git object data blocks that have not been transmitted.

[0059] In a fourth aspect, an embodiment of the present application provides a block transmission subsystem, comprising: a transceiver, a processor, and a memory communicatively connected to the processor;

[0060] The memory stores computer-executable instructions;

[0061] The transceiver is used to communicate with the first forwarding device and the second forwarding device in the remote synchronization system described in the first aspect and various possible designs of the first aspect;

[0062] The processor executes the computer-executable instructions stored in the memory to implement the remote synchronization method described in various possible designs of the first aspect and the second aspect above.

[0063] In a fifth aspect, an embodiment of the present application provides a computer-readable storage medium, in which computer execution instructions are stored. When the computer execution instructions are executed by a processor, they are used to implement the remote synchronization method described in the second aspect and various possible designs of the second aspect.

[0064] In a sixth aspect, an embodiment of the present application provides a computer program product, including a computer program, which, when executed by a processor, implements the remote synchronization method described in the second aspect and various possible designs of the second aspect.

[0065] The present application provides a remote synchronization system, method and block transmission subsystem, the remote synchronization system includes a first forwarding device, a second forwarding device and a block transmission subsystem, the first forwarding device, the first source code warehouse and the part of the block transmission subsystem used for communicating with the first forwarding device are in a first network environment, the second forwarding device, the second source code warehouse and the part of the block transmission subsystem used for communicating with the second forwarding device are in a second network environment, and the first network environment and the second network environment are different. The first forwarding device is used to forward the data interacted between the first source code warehouse and the block transmission subsystem, and the second forwarding device is used to forward the data interacted between the second source code warehouse and the block transmission subsystem; the block transmission subsystem is used to transmit multiple Git object data blocks to the first forwarding device one by one, and is also used to retransmit the Git object data blocks that failed to be transmitted, the multiple Git object data blocks belong to the target Git object data, and the target Git object data is the Git object data corresponding to the remote warehouse update request sent by the first source code warehouse to the second source code warehouse. In this remote synchronization system, the entire transmission process is divided into two reliable networks (the first network environment and the second network environment) and an unreliable network (the network where the block transmission subsystem is located). The data transmission between the two first forwarding devices and the second forwarding device is realized through the retransmittable block transmission subsystem to solve the transmission interruption problem of the unreliable network. This solution can achieve the stable completion of the Gitclone operation on the unreliable network without changing the client (the first source code warehouse) and the server (the second source code warehouse). BRIEF DESCRIPTION OF THE DRAWINGS

[0066] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0067] Figure 1 A schematic diagram of the structure of a remote synchronization system provided in an embodiment of the present application;

[0068] Figure 2 A flowchart of a first embodiment of a remote synchronization method provided in an embodiment of the present application;

[0069] Figure 3 A flowchart of a second embodiment of a remote synchronization method provided in an embodiment of the present application;

[0070] Figure 4 A flowchart of a third embodiment of the remote synchronization method provided in an embodiment of the present application;

[0071] Figure 5 A flowchart of a fourth embodiment of a remote synchronization method provided in an embodiment of the present application;

[0072] Figure 6 A flowchart of a fifth embodiment of a remote synchronization method provided in an embodiment of the present application;

[0073] Figure 7 A flowchart of a sixth embodiment of a remote synchronization method provided in an embodiment of the present application;

[0074] Figure 8 A schematic diagram of the structure of a remote synchronization device provided in an embodiment of the present application;

[0075] Fig. 9 A schematic diagram of the structure of the block transmission subsystem provided in an embodiment of the present application.

[0076] The above drawings have shown clear embodiments of the present application, which will be described in more detail later. These drawings and text descriptions are not intended to limit the scope of the present application in any way, but to illustrate the concept of the present application to those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION

[0077] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application. Instead, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.

[0078] In the technical solution of this application, the collection, storage, use, processing, transmission, provision and disclosure of information such as financial data or user data involved shall comply with the provisions of relevant laws and regulations and shall not violate public order and good morals.

[0079] It should be noted that in the embodiments of the present application, certain software, components, models and other existing solutions in the industry may be mentioned, and they should be regarded as exemplary. Their purpose is only to illustrate the feasibility of implementing the technical solution of the present application, but it does not mean that the applicant has or will necessarily use the solution.

[0080] Before introducing the present application, the application background involved in the present application is explained.

[0081] In the field of software development, source code management is a crucial link, which involves code version control, collaborative development, history tracking and many other aspects. As a distributed version control system, Git has become a leading tool in the field of source code management due to its excellent design, extensive community support, and efficient, flexible and powerful branch management capabilities. The design philosophy of Git is that every developer can have a complete code repository locally, which not only speeds up the operation, but also allows development and version control without a network connection.

[0082] Source code repository technologies based on Git, such as GitHub, Gitee, and GitLab, provide users with different levels of services and have been widely used in various industries. As the largest code hosting platform, GitHub has a large user base and open source projects, and is a model of open source culture and collaborative development. Gitee is also a code hosting service. Unlike GitHub, it provides Chinese users with a more user-friendly interface and faster access speed. GitLab allows users to deploy privately, which means that enterprises can build GitLab services on their own servers, thereby achieving private management of code and more advanced security control.

[0083] In the process of using Git, Git clone is a basic and commonly used command, which allows developers to copy a complete copy of the code from a remote repository to the local. This process can be completed through a variety of protocols, the most common of which are Secure Shell (SSH), Hypertext Transfer Protocol (HTTP), and Hypertext Transfer Protocol Secure (HTTPS). The SSH protocol provides secure encrypted transmission and is suitable for use in private networks, while HTTP and HTTPS protocols are more universal and easy to access in public networks. The Git clone operation is not just a simple file copy, it also copies all the history and branch structure of the remote repository, so that the local repository is synchronized with the remote repository, laying the foundation for subsequent development and collaboration.

[0084] Specifically, when Git clones, if the HTTP protocol is used, the fetch method of the HTTPS protocol can be used to obtain data from the remote repository. Under this communication protocol framework, data transmission adopts the Transfer-Encoding: chunk encoding mechanism, which allows data to be transmitted in chunks when the exact content length is unknown. Specifically, the Git internal data protocol specifies Content-Type: application / x-Git-upload-pack-result. This Multipurpose Internet Mail Extensions (MIME) type identifies the transmitted data as a Git-specific upload-pack result in binary format. Therefore, the Git cloning operation performed through the HTTP protocol essentially utilizes the variable-length transmission mode of Transfer-Encoding: chunked to transmit binary data based on the Git internal data protocol.

[0085] However, in some scenarios, two source code repositories are deployed far apart in geographical locations and in different network environments. When transferring data between different network environments, there are serious stability issues. Under unstable network transmission conditions, the response results of Git fetch often experience transmission interruptions. In addition, since the storage space of the source code repository is very large, it is almost impossible to transfer the entire project at once for certain large source code projects.

[0086] In summary, the existing technology has the problem of poor transmission stability.

[0087] Based on the above technical problems, the invention of the present application is as follows: In the Git clone operation, the operation that fails due to network problems is generally derived from the remote repository update request (the request is a POST instruction of command=fetch). The fetch operation part in the instruction cannot retransmit the excessively large transmission data at the breakpoint, resulting in poor stability of data transmission. The present application provides a remote synchronization system, which is based on the basic protocol of Git and the http transmission protocol version, divides the entire transmission process into two (the first network environment and the second network environment) and an unreliable network, establishes the first forwarding device and the second forwarding device in the reliable network at the first source code warehouse end and the second source code warehouse end respectively, and realizes data transmission between the two first forwarding devices and the second forwarding device through a retransmittable block transmission subsystem to solve the transmission interruption problem of the unreliable network.

[0088] The technical solution of the present application and how the technical solution of the present application solves the above-mentioned technical problems are described in detail below with specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present application will be described below in conjunction with the accompanying drawings.

[0089] Figure 1 This is a schematic diagram of the structure of the remote synchronization system provided in the embodiment of the present application. Figure 1 As shown, the remote synchronization system 10 includes: a first forwarding device 101 , a second forwarding device 102 , and a block transmission subsystem 103 .

[0090] Among them, the first forwarding device 101, the first source code warehouse and the part of the block transmission subsystem used for communicating with the first forwarding device are in a first network environment, the second forwarding device 102, the second source code warehouse and the part of the block transmission subsystem used for communicating with the second forwarding device are in a second network environment, and the first network environment and the second network environment are different.

[0091] It should be understood that the operations related to the first source code repository can be implemented by the client component of the first source code repository operation, and the operations related to the second source code repository can be implemented by any standard Git server component.

[0092] In actual applications, the first source code warehouse and the second source code warehouse are generally deployed far apart, resulting in different network environments. For example, the first source code warehouse is deployed in City A, and the second source code warehouse is deployed in City B, which are far apart. For another example, the first source code warehouse is deployed in Country C, and the second source code warehouse is deployed in Country D.

[0093] Furthermore, the first forwarding device 101 is used to forward data exchanged between the first source code repository and the block transmission subsystem 103 , and the second forwarding device 102 is used to forward data exchanged between the second source code repository and the block transmission subsystem 103 .

[0094] Specifically, the first forwarding device 101 is used to forward the remote repository update request to the second forwarding device 102 .

[0095] In actual applications, the remote repository update request may be a POST instruction of command=fetch, and the remote repository update request is used to synchronize the target Git object data corresponding to the target tag in the second source code repository to the first source code repository.

[0096] It should be understood that the first forwarding device 101 is configured with the base address of the second source code repository and the address switching rule. When receiving the remote repository update request, the first forwarding device 101 converts the request address of the remote repository update request and converts its own address to the address of the second source code repository.

[0097] Exemplarily, after receiving the remote repository update request, the first forwarding device 101 converts address 1 (the address of the first forwarding device 101) in the remote repository update request into address 2 (the address of the second source code repository), and forwards the converted remote repository update request to the second source code repository.

[0098] Exemplarily, the target Git object data includes a commit object (English: Commit Object), a tree object (English: Tree Object) or a file object (English: Blob Object), etc. The relevant explanations of the commit object, tree object and file object are as follows:

[0099] The commit object is a snapshot of each commit in the second source repository, which contains the commit author, committer, commit date, commit message, and a pointer to the commit's parent commit (if any). The commit object also contains a pointer to a tree object that represents the directory structure at the time of the commit.

[0100] The tree object represents the directory structure at a specific time point in the second source code repository. It records the names of files and subdirectories, permissions, and pointers to file objects or subtree objects. Each tree object corresponds to a directory in the second source code repository, including top-level directories and subdirectories.

[0101] The file object represents the content of the file in the second source code repository. Each file object contains the actual data of a file, but does not contain the file name or directory structure information. The file object is identified by calculating the hash value of the file content.

[0102] Furthermore, the first forwarding device 101 is further configured to obtain target Git object data sent by the block transmission subsystem 103, and forward the target Git object data to the first source code repository.

[0103] In a possible implementation, the first forwarding device 101 receives the target Git object data stream sent by the block transmission subsystem 103, and the target Git object data includes multiple Git object data blocks corresponding to the target Git object data. Further, the first forwarding device 101 forwards the target Git object data stream to the first source code repository. Correspondingly, the first source code repository receives and reads the target Git object data stream.

[0104] Exemplarily, the first forwarding device 101 uses the form of Transfer-Encoding: chunked to respond to the first source code repository with the target Git object data stream returned by the chunked transmission subsystem 103 in the form of chunks.

[0105] Optionally, the first source code repository parses the block data according to the Git protocol, and completes the final local persistence and parsing, as well as subsequent checkout operations. It should be understood that checkout is the process of checking out the target Git object data to the local working directory.

[0106] Furthermore, the first source code repository can also determine the target Git object data block that has been read, and send the target data block identifier corresponding to the target Git object data block that has been read to the first forwarding device 101. The first forwarding device 101 receives the target data block identifier sent by the first source code repository, and forwards it to the block transmission subsystem 103. Correspondingly, the block transmission subsystem 103 obtains the target data block identifier.

[0107] In this implementation, the block transmission subsystem 103 obtains the target data block identifier corresponding to the target Git object data block that has been read by the first source code repository, so as to subsequently determine whether the above-mentioned target Git object data block exists in the Git object data blocks that have not been transmitted, thereby avoiding repeated transmission of the same Git object data block and ensuring transmission efficiency.

[0108] The second forwarding device 102 is used to receive the remote repository update request sent by the first forwarding device 101 and forward it to the second source code repository. Furthermore, the second forwarding device 102 is also used to obtain the target Git object data sent by the second source code repository and forward the target Git object data to the block transmission subsystem 103.

[0109] In a possible implementation, the second source code repository segments the target Git object data according to a preset segmentation rule (eg, a preset data length), obtains multiple Git object data blocks, and then transmits the multiple Git object data to the block transmission subsystem 103 .

[0110] The block transmission subsystem 103 is used to transmit multiple Git object data blocks to the first forwarding device 101 one by one.

[0111] In a specific implementation, the block transmission subsystem 103 includes a first transmission unit and a second transmission unit, the first transmission unit is in a first network environment, and the second transmission unit is in a second network environment.

[0112] The first transmission unit is used to send a first control message to the second transmission unit, where the first control message is used to indicate that the target Git object data will be transmitted in a data block manner.

[0113] The first transmission unit is further configured to continuously send data packets to the second transmission unit, each data packet including a Git object data block.

[0114] The first transmission unit is further configured to send a second control message to the second transmission unit after all Git object data blocks have been sent, where the second control message is used to indicate that the transmission of the target Git object data has been completed.

[0115] The second transmission unit is configured to send a response message to the first transmission unit after receiving the first control message, the second control message or the data message.

[0116] The second transmission unit is further configured to transmit the received Git object data block to the first forwarding device 101 in a data stream manner.

[0117] In the above implementation, the first transmission unit first sends a first control message to the second transmission unit to inform the second transmission unit that the target Git object data will be transmitted to it in the form of data blocks. Then, the first transmission unit continuously sends Git object data blocks to the second transmission unit until all Git object data blocks are sent. Finally, the first transmission unit sends a second control message to the second transmission unit to inform the second transmission unit that the target Git object data has been transmitted and there is no need to wait any longer. After receiving the message (first control message, second control message or data message), the second transmission unit will return a corresponding response message to the first transmission unit to enable the first transmission unit to understand the transmission status of the message.

[0118] In actual applications, the first control message includes a chunked data transmission (Transfer-Encoding) flag, a chunked transmission start flag, and a transmission task unique flag. After that, when the first transmission unit and the second transmission unit send data messages, control messages, and response messages, they need to verify whether the transmission task unique flags are consistent with each other, which will not be repeated later.

[0119] Furthermore, the second transmission unit reads a data segment of a certain length (i.e., reads a Git object data block), which is generally a fixed length. When the data ends and is less than the fixed length, the actual transmittable length is used as the standard, and then the second transmission unit sends a data message to the second transmission unit according to the read Git object data block.

[0120] At the same time, when sending the data message to the second transmission unit, it is also necessary to synchronously send a third control message, and the third control message is used to indicate the length and starting offset of the next sent data segment.

[0121] The third control message and the data message may be transmitted simultaneously in one request, or divided into two independent messages for transmission.

[0122] For example, under the premise that HTTP is the basic transmission protocol, the third control message can be encapsulated as HTTP request header information, and the data message can be encapsulated as request message body information.

[0123] When the WebSocket protocol is used, the third control message and the data message can be split into two independent packages.

[0124] Furthermore, the second control message includes: the overall sent data length (the data length of the target Git object data) and a hash check value.

[0125] After receiving each message, the second transmission unit will respond to the received message with a response message (eg, an acknowledgment (ack) message) to the second transmission unit to confirm successful reception of the message.

[0126] Similarly, when the first transmission unit sends multiple data messages to the second transmission unit, a parallel mechanism can be adopted to send multiple data messages at the same time with a certain degree of parallelism, and the sending of the next data message does not need to wait for the completion of the previous data message.

[0127] However, the third control message and data message of the same Git object data should be ensured to have strict sequence or adopt the mechanism of merging and arriving the header and message body of the HTTP protocol at the same time.

[0128] For the second transmission unit, it is necessary to use a cache buffering mechanism to reorder the Git object data blocks that arrive out of order, and form a target Git object data stream in order and return it to the first forwarding device 101 .

[0129] In a possible implementation, to ensure the validity of the target Git object data transmission, the first transmission unit and the second transmission unit adopt necessary multi-link transmission and retransmission mechanisms. That is, the block transmission subsystem 103 is also used to retransmit the Git object data blocks that failed to be transmitted.

[0130] Specifically, further, the first transmission unit is specifically used to:

[0131] For any data message, the timing starts after the data message is sent to the first forwarding device 101, and the accumulated duration is obtained. If the accumulated duration reaches the preset duration and the target response message is not obtained, the data message is resent to the first forwarding device 101, and the target response message is the response message corresponding to the data message.

[0132] In the prior art, when the network environments of two source code repositories are different, the data transmission between them is unstable, which leads to data transmission failure. In the above implementation, by setting a preset time for the data message, if no corresponding response message is received within the preset time, it means that the data message transmission has failed, and the data message is retransmitted, which solves the data transmission failure caused by network instability.

[0133] Since the interaction between the first forwarding device 101 and the first source code repository is a single HTTP request response, a long data reading interval will result in a read timeout interruption exception. Therefore, it is very important for the block transmission subsystem 103 to maintain the timeliness of sending Git object data blocks.

[0134] Therefore, it is necessary to pre-set the preset duration according to the actual situation to avoid the preset duration being too short, resulting in too frequent retransmission operations, or the preset duration being too long, affecting the timeliness of sending Git object data blocks.

[0135] Specifically, the preset time length is pre-set according to a preset data length, and the preset data length is used to divide the target Git object data into a plurality of Git object data blocks.

[0136] In practical applications, the maximum time T required to transmit the Git object data block is determined according to the size of the Git object data block. T should be much smaller than the preset duration to ensure pseudo-continuity of normal transmission during interruption.

[0137] Furthermore, when a message is sent but no response is received within M T times (preset duration), or the transmission time exceeds M T (preset duration), the Git object data block should be resent immediately to ensure that a read timeout event is avoided in the first source code repository.

[0138] In the above implementation, the preset duration of the timeout retransmission mechanism is set according to the size of the Git object data block, thereby ensuring the accuracy of the timeout retransmission mechanism.

[0139] In a possible implementation, the block transmission subsystem 103 is further configured to obtain the target data block identifier sent by the first forwarding device 101. If the untransmitted Git object data blocks contain a target Git object data block corresponding to the target data block identifier, the transmission process of the target Git object data block is canceled.

[0140] In the above implementation, by determining the target Git object data block that has been read by the first source code repository, repeated transmission of the same Git object data block is avoided, thereby improving the transmission efficiency and saving the time required for transmission.

[0141] The remote synchronization system provided by the embodiment of the present application includes a first forwarding device, a second forwarding device and a block transmission subsystem, wherein the first forwarding device and the first source code warehouse are in a first network environment, and the second forwarding device and the second source code warehouse are in a second network environment, and the first network environment and the second network environment are different. The first forwarding device is used to forward the data exchanged between the first source code warehouse and the block transmission subsystem, and the second forwarding device is used to forward the data exchanged between the second source code warehouse and the block transmission subsystem; the block transmission subsystem is used to transmit multiple Git object data blocks to the first forwarding device one by one, and is also used to retransmit the Git object data blocks that failed to be transmitted, and the multiple Git object data blocks belong to the target Git object data, and the target Git object data is the Git object data corresponding to the remote warehouse update request sent by the first source code warehouse to the second source code warehouse. In this technical solution, the first forwarding device and the first source code warehouse are set in the same first network environment, which ensures the stability of the first transmission device and the first source code warehouse during data transmission. Similarly, the second forwarding device and the second source code warehouse are set in the same second network environment, which ensures the stability of the second transmission device and the second source code warehouse during data transmission. Furthermore, the block transmission subsystem has a retransmission feature, which enables data transmission between the block transmission subsystem and the second forwarding device to solve the problem of data transmission interruption caused by an unreliable network and ensure smooth data transmission.

[0142] Next, Figure 1 The remote synchronization method performed by the block transmission subsystem shown is explained in detail.

[0143] Figure 2 This is a flow chart of the first embodiment of the remote synchronization method provided in the present application. Figure 2 As shown, the remote synchronization method can be implemented by the following steps:

[0144] S21. Obtain multiple Git object data blocks corresponding to the target Git object data.

[0145] The target Git object data is the Git object data corresponding to the remote repository update request sent by the first source code repository to the second source code repository.

[0146] In a specific implementation, the first source code repository sends a remote repository update request to the first forwarding device, and after receiving the remote repository update request, the first forwarding device forwards it to the second forwarding device. After receiving the remote repository update request, the second forwarding device forwards it to the second source code repository. The second source code repository determines the target Git object data corresponding to the target tag based on the target tag carried in the remote repository update request. Further, according to a preset segmentation rule, the target Git object data is segmented into multiple Git object data blocks and sent to the block transmission subsystem. Correspondingly, for the block transmission subsystem, multiple Git object data blocks corresponding to the target Git object data sent by the second forwarding device are received.

[0147] Exemplarily, the target Git object data includes a commit object, a tree object, or a file object, etc. It should be understood that the target Git object data may also include other types of content, which will not be described in detail here.

[0148] S22: Transmit the multiple Git object data blocks to the first forwarding device one by one.

[0149] It should be understood that the implementation method and principle of this step can refer to Figure 1 and Figure 3 The contents of the illustrated embodiments will not be described in detail here.

[0150] S23. When the transmission of any Git object data block fails, retransmit the data block that failed to be transmitted.

[0151] To ensure the validity of the target Git object data transmission, the data blocks that failed to be transmitted need to be retransmitted.

[0152] In one possible implementation, for any data message, timing starts after the data message is sent to the second transmission unit through the first transmission unit, and the accumulated duration is obtained; if the accumulated duration reaches a preset duration and the first transmission unit does not obtain the target response message, the data message is resent to the second transmission unit, and the target response message is the response message corresponding to the data message.

[0153] In the above implementation, by setting a preset time for the data message, if no corresponding response message is received within the preset time, it means that the data message transmission has failed, so it is retransmitted, solving the data transmission failure caused by network instability.

[0154] Optionally, the preset time length is pre-set according to a preset data length, and the preset data length is used to divide the target Git object data into a plurality of Git object data blocks.

[0155] By presetting the preset time length according to the preset data length, it is avoided that the preset time length is set too small, resulting in too frequent retransmission operations, or the preset time length is too long, which affects the timeliness of sending Git object data blocks, thereby improving the effect of data retransmission.

[0156] In the remote synchronization method provided by the embodiment of the present application, the block transmission subsystem obtains multiple Git object data blocks corresponding to the target Git object data, transmits the multiple Git object data blocks one by one to the first forwarding device, and retransmits the data block that failed to be transmitted when the transmission of any Git object data block fails. The target Git object data is the Git object data corresponding to the remote repository update request sent by the first source code repository to the second source code repository. In this technical solution, the Git object data blocks are transmitted in blocks through the block transmission subsystem, and when any Git object data block fails to be transmitted, it is retransmitted, thereby ensuring the stability of data transmission between the first source code repository and the second source code repository.

[0157] based on Figure 2 The remote synchronization method shown in the figure specifically explains the implementation process of S23.

[0158] Figure 3 The flowchart of the remote synchronization method embodiment 2 provided in the present application embodiment is as follows. Figure 3 As shown, the block transmission subsystem includes a first transmission unit and a second transmission unit, and S23 can be implemented by the following steps:

[0159] S31. Send a first control message to a second transmission unit through a first transmission unit.

[0160] The first control message is used to indicate that the target Git object data will be transmitted in the form of data blocks.

[0161] The first control message includes a chunked data transmission flag, a chunked transmission start flag, and a transmission task unique flag. Afterwards, when the first transmission unit and the second transmission unit send data messages, control messages, and response messages, they need to verify whether the transmission task unique flags are consistent with each other, which will not be repeated later.

[0162] S32. Continuously send data packets to the first forwarding device through the first transmission unit and the second transmission unit.

[0163] Each data message includes a Git object data block.

[0164] When a data message is sent from the first transmission unit to the second transmission unit, a third control message needs to be sent synchronously. The third control message is used to indicate the length and starting offset of the next data segment to be sent.

[0165] The third control message and the data message may be transmitted simultaneously in one request, or divided into two independent messages for transmission.

[0166] For example, under the premise that HTTP is the basic transmission protocol, the third control message can be encapsulated as HTTP request header information, and the data message can be encapsulated as request message body information.

[0167] When the WebSocket protocol is used, the third control message and the data message can be split into two independent packages.

[0168] Similarly, when the first transmission unit sends multiple data messages to the second transmission unit, a parallel mechanism can be adopted to send multiple data messages at the same time with a certain degree of parallelism, and the sending of the next data message does not need to wait for the completion of the previous data message.

[0169] However, the third control message and data message of the same Git object data should be ensured to have strict sequence or adopt the mechanism of merging and arriving the header and message body of the HTTP protocol at the same time.

[0170] S33: After all Git object data blocks are sent, send a second control message to the second transmission unit through the first transmission unit.

[0171] The second control message is used to indicate that the target Git object data has been transmitted.

[0172] Exemplarily, the second control message includes: the overall sent data length (the data length of the target Git object data) and a hash check value.

[0173] Optionally, the second transmission unit needs to utilize a cache buffering mechanism to reorder the Git object data blocks that arrive out of order, and form a target Git object data stream in order to return it to the first forwarding device.

[0174] In the above embodiment, the first control message is first sent from the first transmission unit to the second transmission unit to inform the second transmission unit that the target Git object data will be transmitted to it in the form of data blocks. Then, the Git object data blocks are continuously sent from the first transmission unit to the second transmission unit until all the Git object data blocks are sent. Finally, the second control message is sent from the first transmission unit to the second transmission unit to inform the second transmission unit that the target Git object data has been transmitted and there is no need to wait any longer. After receiving the message (first control message, second control message or data message), the second transmission unit will return a corresponding response message to the first transmission unit to enable the first transmission unit to understand the transmission status of the message.

[0175] Optionally, during the transmission process, a target data block identifier may also be obtained, where the target data block identifier is a data block identifier corresponding to the target Git object data block that has been read by the first source code repository. If a target Git object data block corresponding to the target data block identifier exists in the Git object data blocks that have not been transmitted, the transmission process of the target Git object data block is canceled.

[0176] In the above implementation, by determining the target Git object data block that has been read by the first source code repository, repeated transmission of the same Git object data block is avoided, thereby improving the transmission efficiency and saving the time required for transmission.

[0177] based on Figure 2 and Figure 3 The remote synchronization method proposed in the illustrated embodiment will be explained below through several specific embodiments.

[0178] In actual applications, users initiate the Git clone command from Git. In the actual execution, there are three specific requests, namely, Git upload package acquisition request, reference information acquisition request, and remote repository update request.

[0179] Request 1: Git upload package acquisition request

[0180] In actual applications, the Git upload package acquisition request may be service=git-upload-pack, which is used to query basic information of the second source code repository.

[0181] The first forwarding device packages the address, method (GET) and header information of the request message in an agreed format and sends them to the second forwarding device. The agreed format may be JavaScript Object Notation (JSON) format or other encoding formats.

[0182] During the transmission process, the block transmission subsystem can be used, that is, the first forwarding device sends the data to the block transmission subsystem, and then the block transmission subsystem forwards it to the second forwarding device. However, because the message is small, the message is generally transmitted as a whole.

[0183] The second forwarding device forwards the received request information (first request message) to the second source code repository, and obtains a response from the second source code repository.

[0184] The second forwarding device packages the response content in an agreed format, and returns it to the first forwarding device based on the block transmission component, and then converts it into an HTTP response and returns it to the first source code repository.

[0185] Figure 4 This is a flow chart of the third embodiment of the remote synchronization method provided in the present application. Figure 4 As shown, the remote synchronization method can be implemented by the following steps:

[0186] S401: The first source code repository receives a Git clone command.

[0187] Among them, the Git clone command is Git clone.

[0188] S402: The first source code repository performs a local file directory operation according to a Git clone command.

[0189] S403: The first source code repository sends a Git upload package acquisition request to the first forwarding device.

[0190] S404: The first forwarding device performs address translation on the Git upload package acquisition request.

[0191] S405: The first forwarding device encapsulates the converted Git upload package acquisition request to generate a first request message.

[0192] S406: The first forwarding device forwards the first request message to the first transmission unit.

[0193] S407: The first transmission unit forwards the first request message to the second transmission unit.

[0194] S408. The second transmission unit forwards the first request message to the second forwarding device.

[0195] S409: The second forwarding device unpacks the first request message.

[0196] S410: The second forwarding device initiates a first server request to the second source code repository according to the first request message.

[0197] S411. The second source code repository generates first response data corresponding to the first server request.

[0198] S412: The second source code warehouse sends first response data to the second forwarding device.

[0199] S413: The second forwarding device generates a first response message corresponding to the first response data.

[0200] S414. The second forwarding device sends a first response message to the second transmission unit.

[0201] S415. The second transmission unit sends a first response message to the first transmission unit.

[0202] S416. The first transmission unit sends a first response message to the first forwarding device.

[0203] S417: The first forwarding device parses the first response message.

[0204] S418. The first forwarding device generates a first HTTP response result corresponding to the first response message.

[0205] S419: The first forwarding device sends a first HTTP response result to the first source code repository.

[0206] S420: The first source code repository performs local operations.

[0207] Request 2: Request to obtain reference information

[0208] In actual applications, the reference information acquisition request may be a POST instruction of command=ls-refs, which is used to query information such as branches and tags in the second source code repository.

[0209] The first forwarding device packages the address, method (GET) and header information of the request message in an agreed format and sends them to the second forwarding device. The agreed format may be JavaScript Object Notation (JSON) format or other encoding formats.

[0210] During the transmission process, the block transmission subsystem can be used, that is, the first forwarding device sends the data to the block transmission subsystem, and then the block transmission subsystem forwards it to the second forwarding device. However, because the message is small, the message is generally transmitted as a whole.

[0211] The second forwarding device forwards the received request information (first request message) to the second source code repository, and obtains a response from the second source code repository.

[0212] The second forwarding device packages the response content in an agreed format, and returns it to the first forwarding device based on the block transmission component, and then converts it into an HTTP response and returns it to the first source code repository.

[0213] Figure 5 This is a flow chart of the fourth embodiment of the remote synchronization method provided in the present application. Figure 5 As shown, the remote synchronization method can be implemented by the following steps:

[0214] S501: A first source code repository sends a reference information acquisition request to a first forwarding device.

[0215] S502: The first forwarding device performs address translation on the reference information acquisition request.

[0216] S503: The first forwarding device encapsulates the converted reference information acquisition request to generate a second request message.

[0217] S504: The first forwarding device forwards the second request message to the first transmission unit.

[0218] S505. The first transmission unit forwards the second request message to the second transmission unit.

[0219] S506. The second transmission unit forwards the second request message to the second forwarding device.

[0220] S507: The second forwarding device unpacks the second request message.

[0221] S508: The second forwarding device initiates a second server request to the second source code repository according to the second request message.

[0222] S509: The second source code repository generates second response data corresponding to the second server request.

[0223] S510. The second source code repository sends second response data to the second forwarding device.

[0224] S511. The second forwarding device generates a second response message corresponding to the second response data.

[0225] S512. The second forwarding device sends a second response message to the second transmission unit.

[0226] S513. The second transmission unit sends a second response message to the first transmission unit.

[0227] S514. The first transmission unit sends a second response message to the first forwarding device.

[0228] S515. The first forwarding device parses the second response message.

[0229] S516. The first forwarding device generates a second HTTP response result corresponding to the second response message.

[0230] S517: The first forwarding device sends a second HTTP response result to the first source code repository.

[0231] S518: The first source code repository performs local operations.

[0232] Request 3: Remote repository update request

[0233] In actual applications, the Git upload package acquisition request may be a POST instruction of command=fetch, which is used to obtain the complete content of the target tag, that is, to obtain the target Git object data corresponding to the target tag.

[0234] The first forwarding device packages the address, method (GET) and header information of the request message in an agreed format and sends them to the second forwarding device. The agreed format may be JavaScript Object Notation (JSON) format or other encoding formats.

[0235] During the transmission process, the block transmission subsystem can be used, that is, the first forwarding device sends the data to the block transmission subsystem, and then the block transmission subsystem forwards it to the second forwarding device. However, because the message is small, the message is generally transmitted as a whole.

[0236] The second forwarding device forwards the received request information (first request message) to the second source code repository, and obtains a response from the second source code repository.

[0237] The second forwarding device packages the response content in an agreed format, and returns it to the first forwarding device based on the block transmission component, and then converts it into an HTTP response and returns it to the first source code repository.

[0238] Figure 6 This is a flow chart of the fifth embodiment of the remote synchronization method provided in the present application. Figure 6 As shown, the remote synchronization method can be implemented by the following steps:

[0239] S601. A first source code repository sends a remote repository update request to a first forwarding device.

[0240] S602: The first forwarding device performs address translation on the remote repository update request.

[0241] S603: The first forwarding device encapsulates the converted remote repository update request to generate a third request message.

[0242] S604: The first forwarding device forwards the third request message to the first transmission unit.

[0243] S605. The first transmission unit forwards the third request message to the second transmission unit.

[0244] S606. The second transmission unit forwards the third request message to the second forwarding device.

[0245] S607: The second forwarding device unpacks the third request message.

[0246] S608. The second forwarding device initiates a third server request to the second source code repository according to the third request message.

[0247] S609: The second source code repository generates third response data corresponding to the third server request.

[0248] S610. The second source code warehouse sends third response data to the second forwarding device.

[0249] S611. The second forwarding device generates a third response message corresponding to the third response data.

[0250] S612. The second forwarding device sends a third response message to the second transmission unit.

[0251] S613. The second transmission unit sends a third response message to the first transmission unit.

[0252] S614. The first transmission unit sends a third response message to the first forwarding device.

[0253] S615. The first forwarding device parses the third response message.

[0254] S616. The first forwarding device generates a third HTTP response result corresponding to the third response message.

[0255] S617: The first forwarding device sends a third HTTP response result to the first source code repository.

[0256] S618: The first source code repository performs local operations.

[0257] Generally speaking, the return value of the fetch command is a request of uncertain length, which reflects the Transfer-Encoding: chunked response of the database in the HTTP protocol, and its transmission message format: Content-Type: application / x-git-upload-pack-result is a binary file protocol customized by the Git protocol.

[0258] In the Git clone operation, the failure caused by network interruption usually comes from this process. Therefore, we use the block transmission mechanism to solve the transmission problem of this response message.

[0259] The basic protocol for block transmission can adopt common network request response protocols such as HTTP, or can adopt duplex communication protocols such as Transmission Control Protocol (TCP), WebSocket, etc.

[0260] Figure 7 This is a flow chart of the sixth embodiment of the remote synchronization method provided in the present application. Figure 7 As shown, S611-S614 can be implemented by the following steps:

[0261] S701. The second forwarding device generates third sub-response data corresponding to the third response data, and generates a third sub-response message corresponding to each sub-response data.

[0262] S702. The second forwarding device sends a third sub-response message to the second transmission unit.

[0263] S703: The second transmission unit sends a first control message to the first transmission unit.

[0264] S704: The first transmission unit starts a block receiving mode.

[0265] S705. The first transmission unit sends a response message corresponding to the first control message to the second transmission unit.

[0266] S706. The second transmission unit determines whether the response message corresponding to the first control message has timed out.

[0267] If it times out, execute S703 again; if it does not time out, execute S707.

[0268] S707. The second transmission unit sends a data message and a response message corresponding to each sub-response data to the first transmission unit in parallel.

[0269] S708: The first transmission unit converts the data message into a data stream.

[0270] S709: The first transmission unit sends a data stream to the first forwarding device.

[0271] S710. The first forwarding device converts the data stream into a third response message.

[0272] S711. The first transmission unit sends a response message corresponding to each sub-response data to the second transmission unit.

[0273] S712: The second transmission unit determines whether the response message corresponding to each sub-response data has timed out.

[0274] If the response message corresponding to a sub-response data times out, S707 is executed again for the sub-response data, that is, the sub-response data is re-processed. If there is no timeout, S713 is executed.

[0275] S713. The second transmission unit sends a second control message to the first transmission unit.

[0276] S714: The first transmission unit closes the response to Git.

[0277] S715: The first forwarding device determines that the transmission is completed.

[0278] S716. The first transmission unit sends a response message corresponding to the second control message to the second transmission unit.

[0279] S717. The second transmission unit determines whether the response message corresponding to the second control message has timed out.

[0280] If it times out, then S713 is executed again. If it does not time out, then the process ends.

[0281] Figure 8 This is a schematic diagram of the structure of the remote synchronization device provided in the embodiment of the present application. Figure 8 As shown, the remote synchronization device 80 is applied to the block transmission subsystem 103, including:

[0282] The acquisition module 801 is used to acquire multiple Git object data blocks corresponding to target Git object data, where the target Git object data is Git object data corresponding to a remote repository update request sent by a first source code repository to a second source code repository.

[0283] The transmission module 802 is used to transmit the multiple Git object data blocks one by one to the first forwarding device.

[0284] The transmission module 802 is further configured to retransmit the failed data block when the transmission of any Git object data block fails.

[0285] In one possible design, the transmission module 802 includes: a first transmission unit and a second transmission unit;

[0286] The first transmission unit is used to send a first control message to the second transmission unit, where the first control message is used to indicate that the target Git object data will be transmitted in a data block manner;

[0287] The first transmission unit and the second transmission unit are also used to continuously send data packets to the first forwarding device;

[0288] The first transmission unit is further configured to send a second control message to the second transmission unit after all Git object data blocks have been sent, wherein the second control message is used to indicate that the target Git object data has been transmitted;

[0289] The second transmission unit is used to send a response message to the first transmission unit after receiving the first control message, the second control message or the data message;

[0290] The second transmission unit is further configured to transmit the received Git object data block to the first forwarding device in a data stream manner.

[0291] In a possible design, the first transmission unit is specifically configured to:

[0292] For any data message, start timing after sending the data message to the second transmission unit to obtain the accumulated duration;

[0293] If the accumulated time reaches the preset time and the target response message is not obtained, the data message is resent to the second transmission unit, and the target response message is a response message corresponding to the data message.

[0294] In a possible design, the preset duration is pre-set according to a preset data length, and the preset data length is used to divide the target Git object data into multiple Git object data blocks.

[0295] In a possible design, the remote synchronization device 50 further includes:

[0296] The acquisition module 801 is further used to acquire a target data block identifier, where the target data block identifier is a data block identifier corresponding to a target Git object data block that has been read by the first source code repository.

[0297] The canceling module is used to cancel the transmission processing of the target Git object data block if there is a target Git object data block corresponding to the target data block identifier in the Git object data blocks that have not been transmitted.

[0298] It should be understood that the transmission module 802 may integrate the first transmission unit and the second transmission unit.

[0299] The remote synchronization device provided in the embodiment of the present application can be used to execute the technical solution of the remote synchronization method in the above embodiment. Its implementation principle and technical effect are similar and will not be repeated here.

[0300] It should be noted that it should be understood that the division of the various modules of the above device is only a division of logical functions. In actual implementation, they can be fully or partially integrated into one physical entity, or they can be physically separated. And these modules can all be implemented in the form of software called by processing elements; they can also be all implemented in the form of hardware; some modules can also be implemented in the form of software called by processing elements, and some modules can be implemented in the form of hardware. For example, the transmission module 802 can be a separately established processing element, or it can be integrated in a chip of the above device. In addition, it can also be stored in the memory of the above device in the form of program code, and called and executed by a processing element of the above device. The function of the above transmission module 802. The implementation of other modules is similar. In addition, all or part of these modules can be integrated together, or they can be implemented independently. The processing element here can be an integrated circuit with signal processing capabilities. In the implementation process, each step of the above method or each module above can be completed by an integrated logic circuit of hardware in the processor element or instructions in the form of software.

[0301] Fig. 9 This is a schematic diagram of the structure of the block transmission subsystem provided in the embodiment of the present application. Fig. 9 As shown, the block transmission subsystem 103 may include: a transceiver 901 , a processor 902 , and a memory 903 .

[0302] The processor 902 executes the computer execution instructions stored in the memory, so that the processor 902 executes the scheme in the above embodiment. The processor 902 can be a general-purpose processor, including a central processing unit CPU, a network processor (network processor, NP), etc.; it can also be a digital signal processor DSP, an application-specific integrated circuit ASIC, a field programmable gate array FPGA or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components.

[0303] The memory 903 is connected to the processor 902 via a system bus and completes communication between them. The memory 903 is used to store computer program instructions.

[0304] The transceiver 901 may be used to communicate with a first forwarding device and a second forwarding device in a remote synchronization system.

[0305] Optionally, in hardware implementation, the above Figure 8 The transmission module 802 in the illustrated embodiment corresponds to the transceiver 901 in this embodiment.

[0306] The system bus can be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus, etc. The system bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus. The transceiver is used to realize the communication between the database access device and other computers (such as clients, read-write libraries, and read-only libraries). The memory may include random access memory (RAM) and may also include non-volatile memory.

[0307] An embodiment of the present application also provides a chip for running instructions, which is used to execute the technical solution of the remote synchronization method in the above embodiment.

[0308] An embodiment of the present application further provides a computer-readable storage medium, in which computer instructions are stored. When the computer instructions are executed on a computer, the computer executes the technical solution of the remote synchronization method of the above embodiment.

[0309] An embodiment of the present application also provides a computer program product, which includes a computer program stored in a computer-readable storage medium. At least one processor can read the computer program from the computer-readable storage medium. When at least one processor executes the computer program, the technical solution of the remote synchronization method in the above embodiment can be implemented.

[0310] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of modules is only a logical function division, and there may be other division methods in actual implementation, such as multiple modules can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or modules, which can be electrical, mechanical or other forms.

[0311] The modules described as separate components may or may not be physically separated, and the components shown as modules may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to implement the solution of this embodiment.

[0312] In addition, each functional module in each embodiment of the present application can be integrated into one processing unit, or each module can exist physically separately, or two or more modules can be integrated into one unit. The above-mentioned module-composed unit can be implemented in the form of hardware or in the form of hardware plus software functional units.

[0313] The above-mentioned integrated module implemented in the form of a software function module can be stored in a computer-readable storage medium. The above-mentioned software function module is stored in a storage medium, including a number of instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor to perform some steps of the methods of various embodiments of the present application.

[0314] It should be understood that the processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), etc. A general-purpose processor may be a microprocessor or any conventional processor. The steps of the method disclosed in the invention may be directly implemented as being executed by a hardware processor, or may be implemented by a combination of hardware and software modules in the processor.

[0315] The memory may include a high-speed RAM memory, and may also include a non-volatile storage NVM, such as at least one disk memory, and may also be a USB flash drive, a mobile hard disk, a read-only memory, a magnetic disk or an optical disk, etc.

[0316] The bus may be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, the bus in the drawings of the present application is not limited to only one bus or one type of bus.

[0317] The above storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk. The storage medium can be any available medium that can be accessed by a general or special purpose computer.

[0318] An exemplary storage medium is coupled to a processor so that the processor can read information from the storage medium and write information to the storage medium. Of course, the storage medium can also be a component of the processor. The processor and the storage medium can be located in an application specific integrated circuit (ASIC). Of course, the processor and the storage medium can also exist as discrete components in an electronic control unit or a main control device.

[0319] Those skilled in the art can understand that all or part of the steps of implementing the above-mentioned method embodiments can be completed by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, the steps of the above-mentioned method embodiments are executed; and the aforementioned storage medium includes: ROM, RAM, disk or optical disk and other media that can store program codes.

[0320] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present application.

Claims

1. A remote synchronization system, characterized in that: include: a first forwarding device, a second forwarding device, and a block transmission subsystem, wherein the first forwarding device, the first source code warehouse, and a portion of the block transmission subsystem used for communicating with the first forwarding device are in a first network environment, and the second forwarding device, the second source code warehouse, and a portion of the block transmission subsystem used for communicating with the second forwarding device are in a second network environment, and the first network environment and the second network environment are different; The first forwarding device is used to forward data exchanged between the first source code repository and the block transmission subsystem, and the second forwarding device is used to forward data exchanged between the second source code repository and the block transmission subsystem; The block transmission subsystem is used to transmit multiple Git object data blocks to the first forwarding device one by one, and is also used to retransmit the Git object data blocks that failed to transmit. The multiple Git object data blocks belong to target Git object data, and the target Git object data is the Git object data corresponding to the remote repository update request sent by the first source code repository to the second source code repository.

2. The system according to claim 1, characterized in that The block transmission subsystem comprises: A first transmission unit and a second transmission unit, the first transmission unit is in the first network environment, and the second transmission unit is in the second network environment; The first transmission unit is used to send a first control message to the second transmission unit, where the first control message is used to indicate that the target Git object data will be transmitted in a data block manner; The first transmission unit is further used to continuously send data packets to the second transmission unit, each data packet including the Git object data block; The first transmission unit is further configured to send a second control message to the second transmission unit after all Git object data blocks have been sent, wherein the second control message is used to indicate that the transmission of the target Git object data has been completed; The second transmission unit is used to send a response message to the first transmission unit after receiving the first control message, the second control message or the data message; The second transmission unit is further configured to transmit the received Git object data block to the first forwarding device in a data stream manner.

3. The system according to claim 2, characterized in that The first transmission unit is specifically configured to: For any data message, start timing after sending the data message to the first forwarding device to obtain the accumulated duration; If the accumulated time reaches the preset time and the target response message is not obtained, the data message is resent to the first forwarding device, and the target response message is a response message corresponding to the data message.

4. The system according to claim 3, characterized in that The preset duration is pre-set according to a preset data length, and the preset data length is used to divide the target Git object data into a plurality of Git object data blocks.

5. The system according to any one of claims 1 to 4, characterized in that: The first forwarding device is further configured to: Obtaining a target data block identifier corresponding to the read target Git object data block sent by the first source code repository; The target data block identifier is sent to the block transmission subsystem.

6. The system according to claim 5, characterized in that The block transmission subsystem is further used for: Obtaining a target data block identifier sent by the first forwarding device; If the target Git object data block corresponding to the target data block identifier exists in the Git object data blocks that have not been transmitted, the transmission process of the target Git object data block is canceled.

7. A remote synchronization method, characterized in that: The method applied to the block transmission subsystem in the remote synchronization system according to any one of claims 1 to 6 comprises: Acquire multiple Git object data blocks corresponding to target Git object data, where the target Git object data is Git object data corresponding to a remote repository update request sent by the first source code repository to the second source code repository; Transmitting the plurality of Git object data blocks to the first forwarding device one by one; When the transmission of any Git object data block fails, the failed data block is retransmitted.

8. The method according to claim 7, characterized in that The block transmission subsystem includes a first transmission unit and a second transmission unit, and the transmitting of the plurality of Git object data blocks to the first forwarding device one by one includes: Sending a first control message to the second transmission unit through the first transmission unit, where the first control message is used to indicate that the target Git object data will be transmitted in a data block manner; Continuously sending data packets to the first forwarding device through the first transmission unit and the second transmission unit, each data packet including the Git object data block; After all Git object data blocks are sent, a second control message is sent to the second transmission unit through the first transmission unit, where the second control message is used to indicate that the transmission of the target Git object data is completed.

9. The method according to claim 8, characterized in that When any Git object data block fails to be transmitted, retransmitting the failed data block includes: For any data message, start timing after the data message is sent to the second transmission unit through the first transmission unit, and obtain the accumulated duration; If the accumulated time reaches the preset time and the first transmission unit does not obtain the target response message, the data message is resent to the second transmission unit, and the target response message is a response message corresponding to the data message.

10. The method according to claim 9, characterized in that The preset duration is pre-set according to a preset data length, and the preset data length is used to divide the target Git object data into a plurality of Git object data blocks.

11. The method according to claim 7 or 8, characterized in that: The method further comprises: Obtaining a target data block identifier, where the target data block identifier is a data block identifier corresponding to a target Git object data block that has been read by the first source code repository; If the target Git object data block corresponding to the target data block identifier exists in the Git object data blocks that have not been transmitted, the transmission process of the target Git object data block is canceled.

12. A remote synchronization device, characterized in that: The block transmission subsystem applied to the remote synchronization system according to any one of claims 1 to 6, the device comprising: An acquisition module, configured to acquire multiple Git object data blocks corresponding to target Git object data, wherein the target Git object data is Git object data corresponding to a remote repository update request sent by the first source code repository to the second source code repository; A transmission module, used for transmitting the plurality of Git object data blocks to the first forwarding device one by one; The transmission module is further configured to retransmit the failed data block when the transmission of any Git object data block fails.

13. A block transmission subsystem, characterized in that: include: A transceiver, a processor, and a memory communicatively connected to the processor; The memory stores computer-executable instructions; The transceiver is used to communicate with a first forwarding device and a second forwarding device in a remote synchronization system according to any one of claims 1 to 6; The processor executes the computer-executable instructions stored in the memory to implement the method according to any one of claims 7 to 11.

14. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, which are used to implement the method according to any one of claims 7 to 11 when executed by a processor.

15. A computer program product, characterized in that The invention comprises a computer program, which implements the method according to any one of claims 7 to 11 when being executed by a processor.

Citation Information

Patent Citations

  • Git central warehouse management system and control method

    CN106326372A

  • Multi-place cross-cluster server deployment method and system

    CN110572436A

  • Cross-network data processing method and device

    CN114124929A

  • Code processing method and device

    CN114416548A

  • Method and system for remote file synchronization of multiple servers

    CN115145868A