Remote synchronization system, method and chunked transfer subsystem

Abstract

The application provides a remote synchronization system, method and block transmission subsystem. It relates to the technical field of big data. The remote synchronization system comprises a first forwarding device, a second forwarding device and a block transmission subsystem. The first forwarding device, a first source code warehouse and a part of the block transmission subsystem for communicating with the first forwarding device are in a first network environment. The second forwarding device, a second source code warehouse and a part of the block transmission subsystem for communicating with the second forwarding device are in a second network environment. The first network environment and the second network environment are different. The block transmission subsystem is used for transmitting a plurality of Git object data blocks to the first forwarding device one by one and retransmitting the Git object data blocks that fail to be transmitted. In the remote synchronization system, stable transmission of data of the client (the first source code warehouse) and the server (the second source code warehouse) is realized.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of big data, and in particular to a remote synchronization system and method and a block transmission subsystem. BACKGROUND

[0002] Git is a distributed version control system that can effectively and quickly handle project version management. With Git, each developer can have a complete code repository locally, which not only speeds up operations, but also allows development and version control even without network connectivity.

[0003] In the use of Git, Git clone is a basic and commonly used command that plays a crucial role. Through this command, developers can obtain a complete copy of the code from a remote repository and copy it to a local computer. This process is not just a simple file copy, it also includes obtaining all historical commit records, branches and tag information of the remote repository. On the local computer, Git clone creates a new directory that is a mirror of the remote repository, containing the complete history and structure of the project. In addition, the command automatically sets up remote tracking branches, making it easy for developers to interact with the remote repository, such as pulling updates or pushing changes.

[0004] However, when the network environments of the two source code repositories are different, the stability of data transmission between them is poor. SUMMARY

[0005] The present application provides a remote synchronization system and method and a block transmission subsystem to solve the problem of poor stability of data transmission between two source code repositories when the network environments of the two source code repositories are different.

[0006] In a first aspect, the present application provides a remote synchronization system, comprising:

[0007] A first forwarding device, a second forwarding device, and a block transmission subsystem, wherein the first forwarding device, the first source code repository, and the part of the block transmission subsystem used for communication with the first forwarding device are in a first network environment, the second forwarding device, the second source code repository, and the part of the block transmission subsystem used for communication with the second forwarding device are in a second network environment, and the first network environment and the second network environment are different;

[0008] The first forwarding device is used to forward data exchanged between the first source code repository and the block transmission subsystem, and the second forwarding device is used to forward data exchanged between the second source code repository and the block transmission subsystem;

[0009] The block transmission subsystem is configured to transmit the plurality of Git object data blocks to the first forwarding device one by one, and retransmit the Git object data block that fails to be transmitted, the plurality of Git object data blocks belong to target Git object data, and the target Git object data is the Git object data corresponding to the remote repository update request sent by the first source code repository to the second source code repository.

[0010] In a possible design, the block transmission subsystem includes:

[0011] The first transmission unit is located in the first network environment, and the second transmission unit is located in the second network environment.

[0012] The first transmission unit is configured to send a first control packet to the second transmission unit, and the first control packet is configured to indicate that the target Git object data is to be transmitted in the form of data blocks.

[0013] The first transmission unit is further configured to continuously send data packets to the second transmission unit, and each data packet includes the Git object data block.

[0014] The first transmission unit is further configured to send a second control packet to the second transmission unit after all the Git object data blocks are sent, and the second control packet is configured to indicate that the target Git object data is transmitted completely.

[0015] The second transmission unit is configured to send an acknowledgement packet to the first transmission unit after receiving the first control packet, the second control packet or the data packet.

[0016] The second transmission unit is further configured to transmit the received Git object data block to the first forwarding device in the form of a data stream.

[0017] In a possible design, the first transmission unit is specifically configured to:

[0018] For any data packet, the first transmission unit starts timing after sending the data packet to the first forwarding device, and obtains an accumulated time length.

[0019] If the accumulated time length reaches a preset time length and a target response packet is not obtained, the first transmission unit re-sends the data packet to the first forwarding device, and the target response packet is a response packet corresponding to the data packet.

[0020] In a possible design, the preset time length is preset according to a preset data length, and the preset data length is used to divide the target Git object data into the plurality of Git object data blocks.

[0021] In a possible design of the first forwarding device, the first forwarding device is further configured to:

[0022] obtain a target data block identifier corresponding to the target Git object data block read by the first source code repository;

[0023] send the target data block identifier to the chunk transmission subsystem.

[0024] In a possible design of the chunk transmission subsystem, the chunk transmission subsystem is further configured to:

[0025] obtain the target data block identifier sent by the first forwarding device;

[0026] if the target Git object data block corresponding to the target data block identifier exists in the Git object data block that has not been transmitted, cancel the transmission processing of the target Git object data block.

[0027] In a second aspect, a remote synchronization method is provided, which is applied to the chunk transmission subsystem in the remote synchronization system as described in the first aspect and various possible designs of the first aspect, and the method comprises the following steps.

[0028] obtain a plurality of Git object data blocks corresponding to target Git object data, the target Git object data being Git object data corresponding to a remote repository update request sent by a first source code repository to a second source code repository;

[0029] transmit the plurality of Git object data blocks to the first forwarding device one by one;

[0030] when transmission of any Git object data block fails, retransmit the data block that fails in transmission.

[0031] In a possible design of the chunk transmission subsystem, the chunk transmission subsystem comprises a first transmission unit and a second transmission unit, and the step of transmitting the plurality of Git object data blocks to the first forwarding device one by one comprises the following steps.

[0032] sending, by the first transmission unit, a first control packet to the second transmission unit, the first control packet being used to indicate that the target Git object data is to be transmitted in the form of data blocks;

[0033] continuously sending, by the first transmission unit and the second transmission unit, data packets to the first forwarding device, each data packet comprising the Git object data block;

[0034] after all the Git object data blocks are sent, sending, by the first transmission unit, a second control packet to the second transmission unit, the second control packet being used to indicate that the transmission of the target Git object data is completed.

[0035] In a possible design, the retransmitting the data block for which the transmission fails when the transmission of any Git object data block fails includes the following steps.

[0036] For any data message, start timing after the first transmission unit sends the data message to the second transmission unit, and obtain an accumulated time length.

[0037] If the accumulated time length reaches a preset time length and the first transmission unit does not obtain a target response message, the data message is re-sent to the second transmission unit, where the target response message is a response message corresponding to the data message.

[0038] In a possible design, the preset time length is set according to a preset data length, and the preset data length is used to divide the target Git object data into a plurality of Git object data blocks.

[0039] In a possible design, the method further includes the following steps.

[0040] Obtain a target data block identifier, where the target data block identifier is a data block identifier corresponding to a target Git object data block that has been read by the first source code repository.

[0041] If the target Git object data block corresponding to the target data block identifier exists in the Git object data block that has not been transmitted, cancel the transmission processing of the target Git object data block.

[0042] In a third aspect, a remote synchronization apparatus is provided, which is applied to a block transmission sub-system in a remote synchronization system as described in the first aspect and various possible designs of the first aspect, and includes the following.

[0043] An obtaining module, configured to obtain a plurality of Git object data blocks corresponding to target Git object data, where the target Git object data is Git object data corresponding to a remote repository update request sent by a first source code repository to a second source code repository.

[0044] A transmission module, configured to transmit the plurality of Git object data blocks to a first forwarding device one by one.

[0045] The transmission module is further configured to retransmit any Git object data block for which the transmission fails when the transmission of the Git object data block fails.

[0046] In a possible design, the transmission module includes a first transmission unit and a second transmission unit.

[0047] The first transmission unit is configured to send a first control message to the second transmission unit, where the first control message is used to indicate that the target Git object data is to be transmitted in the form of data blocks.

[0048] The first transmission unit is further configured to continuously send data messages to the second transmission unit, where each data message comprises a Git object data block.

[0049] The first transmission unit is further configured to send a second control message to the second transmission unit after all the Git object data blocks are sent, where the second control message is used to indicate that the transmission of the target Git object data is completed.

[0050] The second transmission unit is configured to send an acknowledgement message to the first transmission unit after receiving the first control message, the second control message or the data message.

[0051] The second transmission unit is further configured to transmit the received Git object data blocks to the first forwarding device in the form of a data stream.

[0052] In a possible design, the first transmission unit is specifically configured to:

[0053] For any data message, the first transmission unit is configured to start timing after sending the data message to the second transmission unit, and obtain an accumulated time length.

[0054] If the accumulated time length reaches a preset time length and no target response message is obtained, the first transmission unit is configured to resend the data message to the second transmission unit, where the target response message is a response message corresponding to the data message.

[0055] In a possible design, the preset time length is preset according to a preset data length, and the preset data length is used to divide the target Git object data into a plurality of Git object data blocks.

[0056] In a possible design, the remote synchronization device further includes:

[0057] The obtaining module is further configured to obtain a target data block identifier, where the target data block identifier is a data block identifier corresponding to a target Git object data block that has been read by the first source code repository.

[0058] The canceling module is configured to cancel the transmission processing of the target Git object data block if the target Git object data block corresponding to the target data block identifier exists in the Git object data blocks that have not been transmitted.

[0059] In a fourth aspect, an embodiment of the present application provides a block transmission subsystem, including a transceiver, a processor, and a memory connected with the processor in communication.

[0060] The memory stores computer-executable instructions;

[0061] The transceiver is configured to communicate with the first forwarding device and the second forwarding device in the remote synchronization system of the first aspect and various possible designs of the first aspect;

[0062] The processor executes the computer-executable instructions stored in the memory to implement the remote synchronization method of the first aspect and various possible designs of the second aspect.

[0063] In a fifth aspect, an embodiment of the present application provides a computer-readable storage medium, the computer-readable storage medium stores computer-executable instructions, and the computer-executable instructions are executed by a processor to implement the remote synchronization method of the second aspect and various possible designs of the second aspect.

[0064] In a sixth aspect, an embodiment of the present application provides a computer program product, including a computer program, and the computer program is executed by a processor to implement the remote synchronization method of the second aspect and various possible designs of the second aspect.

[0065] The remote synchronization system, method and block transmission subsystem provided by the present application, the remote synchronization system includes a first forwarding device, a second forwarding device and a block transmission subsystem, the first forwarding device, a first source code repository and a part of the block transmission subsystem used for communication with the first forwarding device are in a first network environment, the second forwarding device, a second source code repository and a part of the block transmission subsystem used for communication with the second forwarding device are in a second network environment, and the first network environment and the second network environment are different. The first forwarding device is configured to forward data exchanged between the first source code repository and the block transmission subsystem, and the second forwarding device is configured to forward data exchanged between the second source code repository and the block transmission subsystem; the block transmission subsystem is configured to transmit a plurality of Git object data blocks to the first forwarding device one by one, and retransmit the Git object data blocks that fail to be transmitted, and the plurality of Git object data blocks belong to target Git object data, and the target Git object data is Git object data corresponding to a remote repository update request sent by the first source code repository to the second source code repository. In the remote synchronization system, the entire transmission process is divided into two reliable networks (the first network environment and the second network environment) and an unreliable network (a network in which the block transmission subsystem is located), and through the block transmission subsystem that can retransmit, data transmission between the two first forwarding devices and the second forwarding device is realized, so as to solve the transmission interruption problem of the unreliable network. The scheme can realize stable completion of the Git clone operation of the unreliable network without changing the client (the first source code repository) and the server (the second source code repository). BRIEF DESCRIPTION OF DRAWINGS

[0066] The accompanying drawings, which are incorporated herein and constitute part of this specification, illustrate embodiments consistent with the application and, together with the description, further serve to explain the principles of the application.

[0067] Figure 1 Structure diagram of remote synchronization system provided for embodiments of the application;

[0068] Figure 2 Structure diagram of remote synchronization method embodiment one provided for embodiments of the application;

[0069] Figure 3 Structure diagram of remote synchronization method embodiment two provided for embodiments of the application;

[0070] Figure 4 Structure diagram of remote synchronization method embodiment three provided for embodiments of the application;

[0071] Figure 5 Structure diagram of remote synchronization method embodiment four provided for embodiments of the application;

[0072] Figure 6 Structure diagram of remote synchronization method embodiment five provided for embodiments of the application;

[0073] Figure 7 Structure diagram of remote synchronization method embodiment six provided for embodiments of the application;

[0074] Figure 8 Structure diagram of remote synchronization device provided for embodiments of the application;

[0075] Figure 9 Structure diagram of block transmission subsystem provided for embodiments of the application.

[0076] The specific embodiments have been shown by the above-described drawings, and will be described in more detail hereinafter. These drawings and written descriptions are not intended to limit the scope of the concept of the present application in any way, but to illustrate the concept of the present application to those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION

[0077] The exemplary embodiments will be described in detail herein with reference to the attached drawings. The description of the exemplary embodiments is intended to apply to various alternative embodiments of the application. It is to be understood that features illustrated or described as part of one embodiment can be provided with one or more of the other embodiments as well. Variations to these embodiments can become apparent to those of ordinary skill in the art once the concepts of the application are understood. The following detailed description is, therefore, not to be taken in a limiting sense, as the scope of the application is defined by the claims.

[0078] The collection, storage, use, processing, transmission, provision and disclosure of the financial data or user data and other information involved in the technical solutions of the present application comply with relevant laws and regulations and do not violate public order and good customs.

[0079] It should be noted that in the embodiments of the present application, some existing industry solutions such as software, components, models, etc. may be mentioned, which should be considered as exemplary, and the purpose is only to illustrate the feasibility of the implementation of the technical solutions of the present application, but it does not mean that the applicant has or will necessarily use the solution.

[0080] Before introducing the present application, the application background involved in the present application is explained and described.

[0081] In the field of software development, source code management is a crucial link, which involves code version control, collaborative development, history tracking and other aspects. Git, as a distributed version control system, has become a dominant tool in the field of source code management due to its excellent design, extensive community support, and efficient, flexible and powerful branch management capabilities. The design philosophy of Git is that each developer can have a complete code repository locally, which not only speeds up the operation, but also allows development and version control even without network connection.

[0082] Git-based source code repository technologies such as GitHub, Gitee and GitLab provide different levels of services for users and have been widely applied in various industries. GitHub, as the largest code hosting platform, has a large user base and open source projects, and is a model of open source culture and collaborative development. Gitee is also a code hosting service, but it provides a more friendly interface and faster access speed for Chinese users. GitLab allows users to privately deploy, which means that enterprises can build GitLab services on their own servers to achieve private code management and higher security control.

[0083] In the use of Git, Git clone is a fundamental and commonly used command that allows developers to copy a complete code copy from a remote repository to the local. This process can be done through multiple protocols, the most common of which are Secure Shell (SSH), Hypertext Transfer Protocol (HTTP), and Hypertext Transfer Protocol Secure (HTTPS). The SSH protocol provides secure encrypted transmission, suitable for use in private networks, while the HTTP and HTTPS protocols are more general, facilitating access in public networks. Git clone operation is not just a simple file copy, it also copies all the history and branch structure of the remote repository, so that the local repository is synchronized with the remote repository, laying the foundation for subsequent development and collaboration.

[0084] Specifically, when Git clone is used, if the HTTP protocol is used, the fetch method of the HTTPS protocol can be used to obtain data from the remote repository. Under this communication protocol framework, the data transmission uses the Transfer-Encoding:chunke encoding mechanism, which allows data to be transmitted in chunks without knowing the exact content length. Specifically, the Git internal data protocol specifies Content-Type:application / x-Git-upload-pack-result, which identifies the transmitted data as a Git-specific upload-pack result, and its format is binary. Therefore, the Git clone operation through the HTTP protocol essentially uses the indefinite length transmission mode of Transfer-Encoding:chunked to transmit binary data based on the Git internal data protocol.

[0085] However, in some scenarios, the geographical locations of two source code repositories are far apart, and the network environments they are in are different. When data transmission occurs between different network environments, stability is a serious problem. Under unstable network transmission conditions, the response result of Git fetch often experiences transmission interruption. And because the storage space of the source code repository is very large, for a specific large source code project, it is almost impossible to perform a one-time transmission of the entire project.

[0086] In summary, the prior art has the problem of poor transmission stability.

[0087] Based on the above technical problems, the technical solutions of the present application are as follows: the operation failure caused by network interruption in the Git clone operation is generally derived from a remote repository update request (the request is a POST instruction with a command = fetch), and the fetch operation part in the instruction cannot perform breakpoint retransmission on too large transmission data, resulting in poor data transmission stability. The present application provides a remote synchronization system, which divides the entire transmission process into two (a first network environment and a second network environment) and an unreliable network based on the basic protocol of Git and the http transmission protocol version, establishes a first forwarding device and a second forwarding device in the reliable network at the first source code repository end and the second source code repository end respectively, and realizes data transmission between the two first forwarding devices and the second forwarding devices through a retransmissible block transmission subsystem, so as to solve the transmission interruption problem of the unreliable network.

[0088] The technical solutions of the present application and how the technical solutions of the present application solve the above technical problems will be described in detail below with specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes can not be described again in some embodiments. The embodiments of the present application will be described below with reference to the drawings.

[0089] Figure 1 The structure schematic diagram of the remote synchronization system provided by the embodiments of the present application is shown in FIG. 1. Figure 1 As shown in FIG. 1, the remote synchronization system 10 comprises a first forwarding device 101, a second forwarding device 102 and a block transmission subsystem 103.

[0090] Among them, the first forwarding device 101, the first source code repository and the part of the block transmission subsystem used for communication with the first forwarding device are in the first network environment, the second forwarding device 102, the second source code repository and the part of the block transmission subsystem used for communication with the second forwarding device are in the second network environment, and the first network environment and the second network environment are different.

[0091] It should be understood that the related operations of the first source code repository can be realized through the client component of the first source code repository operation. The related operations of the second source code repository can be realized through any standard Git server component.

[0092] In actual application, the locations where the first source code repository and the second source code repository are deployed are generally far apart, resulting in different network environments. For example, the first source code repository is deployed in A city, and the second source code repository is deployed in B city, and A city and B city are far apart. For another example, the first source code repository is deployed in C country, and the second source code repository is deployed in D country.

[0093] Further, the first forwarding device 101 is configured to forward data exchanged between the first source code warehouse and the chunk transmission subsystem 103, and the second forwarding device 102 is configured to forward data exchanged between the second source code warehouse and the chunk transmission subsystem 103.

[0094] Specifically, the first forwarding device 101 is configured to forward the remote warehouse update request to the second forwarding device 102.

[0095] In actual application, the remote warehouse update request can be a POST instruction of command = fetch, and the remote warehouse update request is configured to synchronize target Git object data corresponding to a target tag in the second source code warehouse to the first source code warehouse.

[0096] It should be understood that the first forwarding device 101 is configured with a basic address of the second source code warehouse and an address switching rule. When the first forwarding device 101 receives the remote warehouse update request, the first forwarding device 101 converts the request address of the remote warehouse update request, and converts the address of the first forwarding device 101 to the address of the second source code warehouse.

[0097] For example, after receiving the remote warehouse update request, the first forwarding device 101 converts address 1 (the address of the first forwarding device 101) in the remote warehouse update request to address 2 (the address of the second source code warehouse), and forwards the converted remote warehouse update request to the second source code warehouse.

[0098] For example, the target Git object data includes a commit object (English: Commit Object), a tree object (English: Tree Object), or a file object (English: Blob Object), and the commit object, the tree object, and the file object are explained as follows:

[0099] The commit object is a snapshot of each commit in the second source code warehouse, which contains the author of the commit, the committer, the commit date, the commit message, and the pointer to the parent commit of the commit (if any). The commit object also contains a pointer to a tree object, which represents the directory structure at the time of the commit.

[0100] The tree object represents the directory structure at a certain time point in the second source code warehouse, which records the name, permission, and pointer to the file object or sub-tree object of the file and subdirectory. Each tree object corresponds to a directory in the second source code warehouse, including the top-level directory and the subdirectory.

[0101] The file object represents the content of the file in the second source code warehouse, and each file object contains the actual data of a file, but does not contain the file name or directory structure information. The file object is identified by calculating the hash value of the file content.

[0102] Further, the first forwarding device 101 is further configured to obtain the target Git object data sent by the chunk transmission subsystem 103, and forward the target Git object data to the first source code repository.

[0103] In a possible implementation, the first forwarding device 101 receives a target Git object data stream sent by the chunk transmission subsystem 103, and the target Git object data contains a plurality of Git object data blocks corresponding to the target Git object data. Further, the first forwarding device 101 forwards the target Git object data stream to the first source code repository. Correspondingly, the first source code repository receives and reads the target Git object data stream.

[0104] For example, the first forwarding device 101 uses the form of Transfer-Encoding: chunked to respond to the first source code repository with the target Git object data stream returned by the chunk transmission subsystem 103 in the form of blocks.

[0105] Optionally, the first source code repository parses the block data according to the Git protocol, and completes the final local persistence and parsing, and subsequent checkout operations. It should be understood that checkout is a process of checking out the target Git object data to the local working directory.

[0106] Further, the first source code repository can also determine the read target Git object data block, and send the target data block identifier corresponding to the read target Git object data block to the first forwarding device 101. The first forwarding device 101 receives the target data block identifier sent by the first source code repository, and forwards it to the chunk transmission subsystem 103. Correspondingly, the chunk transmission subsystem 103 obtains the target data block identifier.

[0107] In this implementation, the chunk transmission subsystem 103 obtains the target data block identifier corresponding to the read target Git object data block of the first source code repository, so as to subsequently determine whether the target Git object data block exists in the untransmitted Git object data block, thereby avoiding repeated transmission of the same Git object data block, and ensuring transmission efficiency.

[0108] In the implementation, the second forwarding device 102 is configured to receive the remote repository update request sent by the first forwarding device 101, and forward it to the second source code repository. Further, the second forwarding device 102 is further configured to obtain the target Git object data sent by the second source code repository, and forward the target Git object data to the chunk transmission subsystem 103.

[0109] In a possible implementation, the second source code warehouse splits the target Git object data according to preset splitting rules (for example, preset data length) to obtain a plurality of Git object data blocks, and then transmits the plurality of Git object data blocks to the block transmission subsystem 103.

[0110] The block transmission subsystem 103 is configured to transmit the plurality of Git object data blocks to the first forwarding device 101 one by one.

[0111] In a specific implementation, the block transmission subsystem 103 includes a first transmission unit and a second transmission unit, the first transmission unit is located in a first network environment, and the second transmission unit is located in a second network environment.

[0112] The first transmission unit is configured to send a first control packet to the second transmission unit, and the first control packet is configured to indicate that the target Git object data is to be transmitted in the form of data blocks.

[0113] The first transmission unit is further configured to continuously send data packets to the second transmission unit, and each data packet includes a Git object data block.

[0114] The first transmission unit is further configured to send a second control packet to the second transmission unit after all the Git object data blocks are sent, and the second control packet is configured to indicate that the transmission of the target Git object data is completed.

[0115] The second transmission unit is configured to send a response packet to the first transmission unit after receiving the first control packet, the second control packet, or the data packet.

[0116] The second transmission unit is further configured to transmit the received Git object data blocks to the first forwarding device 101 in the form of a data stream.

[0117] In the above implementation, the first transmission unit first sends the first control packet to the second transmission unit to inform the second transmission unit that the target Git object data is to be transmitted to the second transmission unit in the form of data blocks. Then, the first transmission unit continuously sends the Git object data blocks to the second transmission unit until all the Git object data blocks are sent. Finally, the first transmission unit sends the second control packet to the second transmission unit to inform the second transmission unit that the target Git object data has been transmitted and there is no need to continue waiting. The second transmission unit returns a corresponding response packet to the first transmission unit after receiving the packet (the first control packet, the second control packet, or the data packet), so that the first transmission unit can know the transmission status of the packet.

[0118] In practical applications, the first control message includes a chunked data transmission (Transfer-Encoding) flag, a chunked transmission start flag, and a transmission task unique flag. Subsequently, the first transmission unit and the second transmission unit need to verify whether the transmission task unique flag is consistent when sending data messages, control messages, and response messages. The subsequent description is not repeated.

[0119] Further, the second transmission unit reads a data segment of a certain length (i.e., reads a Git object data block), which is generally of a fixed length. When the end of the data is encountered, if the fixed length is not sufficient, the actual transmittable length is used. Subsequently, the second transmission unit sends a data message to the second transmission unit according to the read Git object data block.

[0120] Meanwhile, when sending a data message to the second transmission unit, a third control message also needs to be sent synchronously. The third control message is used to indicate the length and starting offset of the next data segment.

[0121] The third control message and the data message can be transmitted simultaneously in one request or can be transmitted as two independent messages.

[0122] For example, under the premise of an HTTP-based transmission protocol, the third control message can be encapsulated as HTTP request header information, and the data message can be encapsulated as request message body information.

[0123] When using the WebSocket protocol, the third control message and the data message can be split into two independent packages.

[0124] Further, the second control message includes the overall transmission data length (the data length of the target Git object data) and the hash check value.

[0125] After receiving each message, the second transmission unit sends a response message (e.g., an acknowledgment (ack) message) to the second transmission unit to confirm the successful reception of the message.

[0126] Similarly, when the first transmission unit sends multiple data messages to the second transmission unit, a parallel mechanism can be used. Multiple data messages are sent simultaneously at a certain parallel degree, and the sending of the next data message does not necessarily wait for the completion of the previous data message.

[0127] However, it should be ensured that the third control message and the data message of the same Git object data have strict sequentiality or use the HTTP protocol header and message body merging simultaneous arrival mechanism.

[0128] For the second transmission unit, a cache buffer mechanism is needed to rearrange the out-of-order Git object data blocks and form a target Git object data stream in order to return to the first forwarding device 101.

[0129] In a possible implementation, to ensure the effectiveness of the target Git object data transmission, the first transmission unit and the second transmission unit adopt necessary multi-link transmission and retransmission mechanisms. That is, the block transmission subsystem 103 is further configured to retransmit the Git object data blocks that fail in transmission.

[0130] Specifically, further, the first transmission unit is specifically configured to:

[0131] For any data message, the first forwarding device 101 starts timing after sending the data message, and obtains the cumulative duration. If the cumulative duration reaches the preset duration and no target response message is obtained, the data message is retransmitted to the first forwarding device 101. The target response message is the response message corresponding to the data message.

[0132] In the prior art, when the network environments of the two source code repositories are different, the stability of data transmission between the two is poor, which leads to data transmission failure. In the above implementation, by setting a preset duration for the data message, if no corresponding response message is received within the preset duration, it means that the data message transmission fails, and the data message is retransmitted, thereby solving the data transmission failure caused by unstable network.

[0133] Since the interaction between the first forwarding device 101 and the first source code repository is a single HTTP request response, a too long data reading interval will cause a reading timeout interruption exception. Therefore, it is very important for the block transmission subsystem 103 to maintain the timeliness of sending the Git object data blocks.

[0134] Therefore, the preset duration needs to be set in advance according to the actual situation to avoid too frequent retransmission operation caused by too small preset duration or too long preset duration affecting the timeliness of sending the Git object data blocks.

[0135] Specifically, the preset duration is set in advance according to a preset data length, and the preset data length is used to divide the target Git object data into multiple Git object data blocks.

[0136] In actual application, according to the size of the Git object data block, the maximum time T required for transmitting the Git object data block is determined, and T should be much smaller than the preset duration to ensure the pseudo-continuity of normal transmission in interruption.

[0137] Further, when a certain message is not responded within M T (preset time length) after being sent, or the transmission time exceeds M T (preset time length), the Git object data block should be immediately retransmitted to avoid the occurrence of read timeout event of the first source code warehouse.

[0138] In the above implementation mode, the preset time length of the timeout retransmission mechanism is set according to the size of the Git object data block, which ensures the accuracy of the timeout retransmission mechanism.

[0139] In a possible implementation mode, the block transmission subsystem 103 is further configured to acquire the target data block identifier sent by the first forwarding device 101. If the target Git object data block corresponding to the target data block identifier exists in the untransmitted Git object data block, the transmission processing of the target Git object data block is cancelled.

[0140] In the above implementation mode, by determining the target Git object data block read by the first source code warehouse, repeated transmission of the same Git object data block is avoided, the transmission efficiency is improved, and the transmission time is saved.

[0141] The remote synchronization system provided by the embodiment of the present application comprises a first forwarding device, a second forwarding device and a block transmission subsystem. The first forwarding device and the first source code warehouse are in a first network environment, the second forwarding device and the second source code warehouse are in a second network environment, and the first network environment and the second network environment are different. The first forwarding device is configured to forward the data exchanged between the first source code warehouse and the block transmission subsystem, and the second forwarding device is configured to forward the data exchanged between the second source code warehouse and the block transmission subsystem. The block transmission subsystem is configured to transmit a plurality of Git object data blocks to the first forwarding device one by one, and retransmit the Git object data block that fails in transmission. The plurality of Git object data blocks belong to target Git object data, and the target Git object data is the Git object data corresponding to the remote warehouse update request sent by the first source code warehouse to the second source code warehouse. In the technical solution, the first forwarding device and the first source code warehouse are arranged in the same first network environment, which ensures the stability of the first transmission device and the first source code warehouse during data transmission. Similarly, the second forwarding device and the second source code warehouse are arranged in the same second network environment, which ensures the stability of the second transmission device and the second source code warehouse during data transmission. Further, the block transmission subsystem has the retransmission feature, which realizes the data transmission between the block transmission subsystem and the second forwarding device, solves the problem of data transmission interruption caused by unreliable network, and ensures the smooth transmission of data.

[0142] Next, the remote synchronization method performed by the block transmission subsystem shown in FIG. 1 will be specifically explained. Figure 1 In the above implementation mode, the preset time length of the timeout retransmission mechanism is set according to the size of the Git object data block, which ensures the accuracy of the timeout retransmission mechanism.

[0143] Figure 2 The flowchart of the remote synchronization method embodiment provided by the embodiment of the present application is shown in Figure 2 The remote synchronization method can be implemented by the following steps:

[0144] S21, obtaining a plurality of Git object data blocks corresponding to target Git object data.

[0145] The target Git object data is the Git object data corresponding to the remote repository update request sent by the first source code repository to the second source code repository.

[0146] In a specific implementation, the first source code repository sends a remote repository update request to the first forwarding device, and the first forwarding device forwards the remote repository update request to the second forwarding device after receiving the remote repository update request. The second forwarding device forwards the remote repository update request to the second source code repository after receiving the remote repository update request. The second source code repository determines the target Git object data corresponding to the target label based on the target label carried in the remote repository update request. Further, the target Git object data is divided into a plurality of Git object data blocks according to a preset division rule, and is sent to the block transmission subsystem. Correspondingly, the block transmission subsystem receives the plurality of Git object data blocks corresponding to the target Git object data sent by the second forwarding device.

[0147] For example, the target Git object data includes commit objects, tree objects, or file objects, and it should be understood that the target Git object data can also include other types of content, which will not be described here.

[0148] S22, transmitting the plurality of Git object data blocks to the first forwarding device one by one.

[0149] It should be understood that the implementation and principles of this step can refer to the contents in the embodiments shown in Figure 1 and Figure 3 The embodiments will not be described here.

[0150] S23, when the transmission of any Git object data block fails, retransmitting the data block that fails to transmit.

[0151] In order to ensure the effectiveness of the transmission of the target Git object data, the retransmission of the data block that fails to transmit is required.

[0152] In a possible implementation, for any data message, the first transmission unit starts timing after sending the data message to the second transmission unit, and obtains the cumulative duration; if the cumulative duration reaches the preset duration and the first transmission unit does not obtain the target response message, the data message is re-sent to the second transmission unit, and the target response message is the response message corresponding to the data message.

[0153] In the above implementation, the preset time length is set for the data message, and if no corresponding response message is received within the preset time length, it indicates that the data message transmission fails, so that the data message is retransmitted, thereby solving the data transmission failure caused by unstable network.

[0154] Optionally, the preset time length is set in advance according to a preset data length, and the preset data length is used to split the target Git object data into a plurality of Git object data blocks.

[0155] By setting the preset time length in advance according to the preset data length, it is avoided that the preset time length is too small to cause the retransmission operation too frequent, or the preset time length is too long to affect the timeliness of sending the Git object data block, thereby improving the effect of data retransmission.

[0156] The remote synchronization method provided by the embodiment of the application includes that a block transmission subsystem acquires a plurality of Git object data blocks corresponding to target Git object data, and transmits the plurality of Git object data blocks to a first forwarding device one by one, and when transmission of any Git object data block fails, retransmits the data block that fails in transmission. The target Git object data is Git object data corresponding to a remote repository update request sent by a first source code repository to a second source code repository. In the technical solution, the block transmission subsystem transmits the Git object data block in a block manner, and when transmission of any Git object data block fails, retransmits the Git object data block, thereby ensuring the stability of data transmission of the first source code repository and the second source code repository.

[0157] Based on the remote synchronization method shown in FIG. 8, the implementation process of S23 is specifically explained and described. Figure 2 As shown in FIG. 9, the block transmission subsystem includes a first transmission unit and a second transmission unit, and S23 can be implemented by the following steps.

[0158] Figure 3 FIG. 10 is a flowchart of a remote synchronization method according to Embodiment Two of the remote synchronization method provided by the embodiment of the application. As shown in FIG. 10, the block transmission subsystem includes a first transmission unit and a second transmission unit, and S23 can be implemented by the following steps. Figure 3 As shown in FIG. 10, the block transmission subsystem includes a first transmission unit and a second transmission unit, and S23 can be implemented by the following steps.

[0159] S31, the first transmission unit transmits a first control message to the second transmission unit.

[0160] The first control message is used to indicate that the target Git object data will be transmitted in a data block manner.

[0161] The first control message includes a chunked data transmission flag, a block transmission start flag, and a transmission task unique flag. Then, when the first transmission unit and the second transmission unit transmit data messages, control messages, and response messages, they need to verify whether the transmission task unique flag is consistent with each other, and the subsequent will not be repeated.

[0162] S32, continuously sending data packets to the first forwarding device through the first transmission unit and the second transmission unit.

[0163] Each data packet includes a Git object data block.

[0164] When sending data packets through the first transmission unit to the second transmission unit, a third control packet needs to be sent synchronously, which is used to indicate the length and starting offset of the next data sending paragraph.

[0165] The third control packet and the data packet can be transmitted simultaneously in one request, or can be transmitted as two independent packets.

[0166] For example, under the premise of HTTP-based transmission protocol, the third control packet can be encapsulated as HTTP request header information, and the data packet can be encapsulated as request packet body information.

[0167] When using the WebSocket protocol, the third control packet and the data packet can be split into two independent packages.

[0168] Similarly, when the first transmission unit sends multiple data packets to the second transmission unit, a parallel mechanism can be used. Multiple data packets are sent simultaneously with a certain degree of parallelism, and the sending of the next data packet does not need to wait for the completion of the previous data packet.

[0169] However, the third control packet and the data packet of the same Git object data should have strict sequentiality or use the merging and simultaneous arrival mechanism of the HTTP protocol header and packet body.

[0170] S33, after sending all Git object data blocks, sending a second control packet to the second transmission unit through the first transmission unit.

[0171] The second control packet is used to indicate that the target Git object data transmission is complete.

[0172] For example, the second control packet includes the overall sending data length (the data length of the target Git object data) and the hash check value.

[0173] Optionally, for the second transmission unit, a cache buffer mechanism is needed to rearrange the out-of-order Git object data blocks and form a target Git object data stream in order to return to the first forwarding device.

[0174] In the above embodiment, the first transmission unit sends a first control message to the second transmission unit, to inform the second transmission unit that the target Git object data is to be transmitted in the form of data blocks. Then, the first transmission unit continuously sends the Git object data blocks to the second transmission unit, until all the Git object data blocks are transmitted. Finally, the first transmission unit sends a second control message to the second transmission unit, to inform the second transmission unit that the target Git object data has been transmitted and there is no need to continue waiting. After receiving the message (the first control message, the second control message or the data message), the second transmission unit returns a corresponding response message to the first transmission unit, so that the first transmission unit can determine the transmission status of the message.

[0175] Optionally, the target data block identifier can be acquired during the transmission process. The target data block identifier is the data block identifier corresponding to the target Git object data block that has been read by the first source code repository. If the target Git object data block corresponding to the target data block identifier exists in the Git object data block that has not been transmitted, the transmission process of the target Git object data block is cancelled.

[0176] In the above implementation manner, by determining the target Git object data block that has been read by the first source code repository, repeated transmission of the same Git object data block is avoided, the transmission efficiency is improved, and the transmission time is saved.

[0177] Based on the remote synchronization method proposed in the embodiments shown in Figure 2 and Figure 3 , the remote synchronization method will be explained and described through several specific embodiments.

[0178] In actual applications, a user initiates a Git clone instruction from a Git. In actual execution, there are three specific requests, which are a Git upload package acquisition request, a reference information acquisition request and a remote repository update request.

[0179] Request 1, Git upload package acquisition request

[0180] In actual applications, the Git upload package acquisition request can be service=git-upload-pack, which is used to query the basic information of the second source code repository.

[0181] The first forwarding device packs and sends the address, method (GET) and header information of the request message to the second forwarding device in an agreed format. The agreed format can be JavaScript Object Notation (JSON) format or other encoding formats.

[0182] In the transmission process, the block transmission subsystem can be used, that is, the first forwarding device sends data to the block transmission subsystem first, and then the block transmission subsystem forwards the data to the second forwarding device. However, because the message is small, the message is generally transmitted as a whole.

[0183] The second forwarding device forwards the received request information (the first request message) to the second source code warehouse and obtains a response of the second source code warehouse.

[0184] The second forwarding device packs the response content in a predetermined format, returns the response to the first forwarding device through the block transmission component, and returns the HTTP response to the first source code warehouse.

[0185] Figure 4 A flowchart of a third embodiment of the remote synchronization method provided by the embodiments of the present application is shown in FIG. 6. Figure 4 As shown in FIG. 6, the remote synchronization method can be implemented through the following steps:

[0186] S401, the first source code warehouse receives a Git clone command.

[0187] The Git clone command is a Git clone.

[0188] S402, the first source code warehouse performs a local file directory operation according to the Git clone command.

[0189] S403, the first source code warehouse sends a Git upload package acquisition request to the first forwarding device.

[0190] S404, the first forwarding device performs address conversion on the Git upload package acquisition request.

[0191] S405, the first forwarding device encapsulates the converted Git upload package acquisition request to generate a first request message.

[0192] S406, the first forwarding device forwards the first request message to the first transmission unit.

[0193] S407, the first transmission unit forwards the first request message to the second transmission unit.

[0194] S408, the second transmission unit forwards the first request message to the second forwarding device.

[0195] S409, the second forwarding device unpacks the first request message.

[0196] S410, the second forwarding device initiates a first server request to the second source code warehouse according to the first request message.

[0197] S411, the second source code warehouse generates a first server request corresponding first response data.

[0198] S412, the second source code warehouse sends the first response data to the second forwarding device.

[0199] S413, the second forwarding device generates a first response message corresponding to the first response data.

[0200] S414, the second forwarding device sends the first response message to the second transmission unit.

[0201] S415, the second transmission unit sends the first response message to the first transmission unit.

[0202] S416, the first transmission unit sends the first response message to the first forwarding device.

[0203] S417, the first forwarding device parses the first response message.

[0204] S418, the first forwarding device generates a first HTTP response result corresponding to the first response message.

[0205] S419, the first forwarding device sends the first HTTP response result to the first source code warehouse.

[0206] S420, the first source code warehouse executes a local operation.

[0207] Request 2, reference information acquisition request

[0208] In actual application, the reference information acquisition request can be a POST instruction of command = ls-refs, which is used to query the branch, tag and other information in the second source code warehouse.

[0209] The first forwarding device packs the address, method (GET) and header information of the request message into an agreed format and sends it to the second forwarding device. The agreed format can be JavaScript Object Notation (JSON) format or other encoding format.

[0210] In the transmission process, a chunked transfer subsystem can be used, that is, the first forwarding device sends data to the chunked transfer subsystem first, and then the chunked transfer subsystem forwards it to the second forwarding device. However, because the message is small, the message is generally transmitted as a whole.

[0211] The second forwarding device forwards the received request information (first request message) to the second source code warehouse and obtains the response of the second source code warehouse.

[0212] The second forwarding device packs the response content in an agreed format, and returns to the first forwarding device relying on the block transmission component, and then exchanges the HTTP response to return to the first source code repository.

[0213] Figure 5 A flowchart of a remote synchronization method embodiment four provided by the embodiments of the application is shown in FIG. 5. Figure 5 As shown in the figure, the remote synchronization method can be implemented by the following steps:

[0214] S501, the first source code repository sends a reference information acquisition request to the first forwarding device.

[0215] S502, the first forwarding device performs address conversion on the reference information acquisition request.

[0216] S503, the first forwarding device encapsulates the converted reference information acquisition request to generate a second request message.

[0217] S504, the first forwarding device forwards the second request message to the first transmission unit.

[0218] S505, the first transmission unit forwards the second request message to the second transmission unit.

[0219] S506, the second transmission unit forwards the second request message to the second forwarding device.

[0220] S507, the second forwarding device unpacks the second request message.

[0221] S508, the second forwarding device initiates a second server request to the second source code repository according to the second request message.

[0222] S509, the second source code repository generates second response data corresponding to the second server request.

[0223] S510, the second source code repository sends the second response data to the second forwarding device.

[0224] S511, the second forwarding device generates a second response message corresponding to the second response data.

[0225] S512, the second forwarding device sends the second response message to the second transmission unit.

[0226] S513, the second transmission unit sends the second response message to the first transmission unit.

[0227] S514, the first transmission unit sends the second response message to the first forwarding device.

[0228] S515, the first forwarding device parses the second response message.

[0229] S516, the first forwarding device generates a second HTTP response result corresponding to the second response message.

[0230] S517, the first forwarding device sends the second HTTP response result to the first source code warehouse.

[0231] S518, the first source code warehouse performs a local operation.

[0232] Request 3, remote warehouse update request

[0233] In actual application, the Git upload package acquisition request can be a POST instruction of command = fetch, to acquire the complete content of the target label. That is, the target Git object data corresponding to the target label is acquired.

[0234] The first forwarding device packs and sends the address, method (GET) and header information of the request message to the second forwarding device in an agreed format, which can be JavaScript Object Notation (JSON) format or other encoding format.

[0235] In the transmission process, a chunk transmission subsystem can be used, that is, the first forwarding device sends data to the chunk transmission subsystem first, and then the chunk transmission subsystem forwards it to the second forwarding device. However, because the message is small, the message is generally transmitted as a whole.

[0236] The second forwarding device forwards the received request information (the first request message) to the second source code warehouse, and acquires the response of the second source code warehouse.

[0237] The second forwarding device packs the response content in an agreed format, returns it to the first forwarding device relying on the chunk transmission component, and then exchanges it into an HTTP response and returns it to the first source code warehouse.

[0238] Figure 6 The flowchart of the fifth embodiment of the remote synchronization method provided by the embodiments of the present application is shown in Figure 5. Figure 6 As shown in the figure, the remote synchronization method can be implemented by the following steps:

[0239] S601, the first source code warehouse sends a remote warehouse update request to the first forwarding device.

[0240] S602, the first forwarding device performs address conversion on the remote warehouse update request.

[0241] S603, the first forwarding device encapsulates the converted remote warehouse update request to generate a third request message.

[0242] S604, the first forwarding device forwards the third request message to the first transmission unit.

[0243] S605, the first transmission unit forwards the third request message to the second transmission unit.

[0244] S606, the second transmission unit forwards the third request message to the second forwarding device.

[0245] S607, the second forwarding device unpacks the third request message.

[0246] S608, the second forwarding device initiates a third server request to the second source code repository according to the third request message.

[0247] S609, the second source code repository generates third response data corresponding to the third server request.

[0248] S610, the second source code repository sends the third response data to the second forwarding device.

[0249] S611, the second forwarding device generates a third response message corresponding to the third response data.

[0250] S612, the second forwarding device sends the third response message to the second transmission unit.

[0251] S613, the second transmission unit sends the third response message to the first transmission unit.

[0252] S614, the first transmission unit sends the third response message to the first forwarding device.

[0253] S615, the first forwarding device parses the third response message.

[0254] S616, the first forwarding device generates a third HTTP response result corresponding to the third response message.

[0255] S617, the first forwarding device sends the third HTTP response result to the first source code repository.

[0256] S618, the first source code repository performs a local operation.

[0257] Generally, the return value of the fetch command is a request of uncertain length, which is reflected in the HTTP protocol as Transfer-Encoding: chunked of the response database, and the message format of its transmission: Content-Type: application / x-git-upload-pack-result is a binary file protocol defined by the Git protocol.

[0258] And the Git clone operation, due to network problems caused by the failure of the operation, generally from this process. Therefore, we use block transfer mechanism to solve the transmission of this response message.

[0259] The basic protocol of block transmission can adopt common network request response protocols such as HTTP, etc., or can adopt duplex communication protocols such as Transmission Control Protocol (TCP), WebSocket, etc.

[0260] Figure 7 The flowchart of the remote synchronization method embodiment six provided by the embodiment of the application is shown in the figure. Figure 7 As shown, S611-S614 can be implemented by the following steps:

[0261] S701, the second forwarding device generates third sub-response data corresponding to the third response data, and generates a third sub-response message corresponding to each sub-response data.

[0262] S702, the second forwarding device sends the third sub-response message to the second transmission unit.

[0263] S703, the second transmission unit sends the first control message to the first transmission unit.

[0264] S704, the first transmission unit starts the block receiving mode.

[0265] S705, the first transmission unit sends the response message corresponding to the first control message to the second transmission unit.

[0266] S706, the second transmission unit judges whether the response message corresponding to the first control message is timed out.

[0267] If it is timed out, re-execute S703, if it is not timed out, execute S707.

[0268] S707, the second transmission unit sends the data message and the response message corresponding to each sub-response data to the first transmission unit in parallel.

[0269] S708, the first transmission unit converts the data message into a data stream.

[0270] S709, the first transmission unit sends the data stream to the first forwarding device.

[0271] S710, the first forwarding device converts the data stream into the third response message.

[0272] S711, the first transmission unit sends the response message corresponding to each sub-response data to the second transmission unit.

[0273] S712, the second transmission unit determines whether the response message corresponding to each sub-response data is timed out.

[0274] If the response message corresponding to a certain sub-response data is timed out, S707 is re-executed on the sub-response data, that is, the sub-response data is re-transmitted. If there is no timeout, S713 is executed.

[0275] S713, the second transmission unit sends a second control message to the first transmission unit.

[0276] S714, the first transmission unit closes the response to Git.

[0277] S715, the first forwarding device determines that the transmission is completed.

[0278] S716, the first transmission unit sends the response message corresponding to the second control message to the second transmission unit.

[0279] S717, the second transmission unit determines whether the response message corresponding to the second control message is timed out.

[0280] If it is timed out, S713 is re-executed. If there is no timeout, it is ended.

[0281] Figure 8 The structure diagram of the remote synchronization device provided by the embodiment of the application is shown in FIG. 8. As shown in the figure, the remote synchronization device 80 is applied to the block transmission subsystem 103, and includes: Figure 8

[0282] The acquisition module 801 is configured to acquire a plurality of Git object data blocks corresponding to target Git object data, the target Git object data being the Git object data corresponding to a remote repository update request sent by a first source code repository to a second source code repository.

[0283] The transmission module 802 is configured to transmit the plurality of Git object data blocks to the first forwarding device one by one.

[0284] The transmission module 802 is further configured to retransmit any Git object data block when the transmission of the Git object data block fails.

[0285] In a possible design, the transmission module 802 includes a first transmission unit and a second transmission unit.

[0286] The first transmission unit is configured to send a first control message to the second transmission unit, the first control message being used to indicate that the target Git object data will be transmitted in the form of data blocks.

[0287] The first transmission unit and the second transmission unit are further configured to continuously send data messages to the first forwarding device.

[0288] ​The first transmission unit is further configured to send a second control message to the second transmission unit after sending all the Git object data blocks, the second control message being used to indicate that the target Git object data transmission is completed.

[0289] The second transmission unit is configured to send a response message to the first transmission unit after receiving the first control message, the second control message or the data message.

[0290] The second transmission unit is further configured to transmit the received Git object data blocks to the first forwarding device in a data stream manner.

[0291] In a possible design, the first transmission unit is specifically configured to:

[0292] For any data message, the first transmission unit starts timing after sending the data message to the second transmission unit, and obtains an accumulated time length.

[0293] If the accumulated time length reaches a preset time length and no target response message is obtained, the first transmission unit re-sends the data message to the second transmission unit, the target response message being a response message corresponding to the data message.

[0294] In a possible design, the preset time length is set according to a preset data length, and the preset data length is used to divide the target Git object data into a plurality of Git object data blocks.

[0295] In a possible design, the remote synchronization device 50 further includes:

[0296] The obtaining module 801 is further configured to obtain a target data block identifier, the target data block identifier being a data block identifier corresponding to a target Git object data block that has been read by the first source code warehouse.

[0297] The canceling module is configured to, if the target Git object data block corresponding to the target data block identifier exists in the untransmitted Git object data blocks, cancel the transmission processing of the target Git object data block.

[0298] It should be understood that the transmission module 802 can integrate the first transmission unit and the second transmission unit.

[0299] The remote synchronization device provided by the embodiments of the present application can be used to execute the technical solutions of the remote synchronization method in the above embodiments, and has similar implementation principles and technical effects, which will not be described herein.

[0300] It should be noted that the division of each module of the above apparatus is only a logical function division, and all or part of them can be integrated into a physical entity or physically separated when actually implemented. And these modules can all be realized in the form of software called by a processing element; or all be realized in the form of hardware; or part of the modules are realized in the form of software called by a processing element, and part of the modules are realized in the form of hardware. For example, the transmission module 802 can be a separately set processing element, or can be integrated in a chip of the above apparatus, in addition, the transmission module 802 can also be stored in the form of program code in the memory of the above apparatus, and the functions of the transmission module 802 are called and executed by a processing element of the above apparatus. The implementation of other modules is similar. In addition, all or part of these modules can be integrated together or independently implemented. The processing element herein can be an integrated circuit with signal processing capability. In the implementation process, each step of the above method or each module can be completed by the integrated logic circuit of hardware or the instruction of software in the processing element.

[0301] Figure 9 The structure diagram of the block transmission subsystem provided by the embodiment of the application is shown in the figure. Figure 9 The block transmission subsystem 103 can include a transceiver 901, a processor 902 and a memory 903.

[0302] The processor 902 executes the computer execution instructions stored in the memory, so that the processor 902 executes the scheme in the above embodiment. The processor 902 can be a general-purpose processor, including a central processing unit CPU, a network processor NP, etc.; and can also be a digital signal processor DSP, an application-specific integrated circuit ASIC, a field programmable gate array FPGA or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0303] The memory 903 is connected with the processor 902 through a system bus and completes mutual communication, and the memory 903 is used for storing computer program instructions.

[0304] The transceiver 901 can be used for communicating with the first forwarding device and the second forwarding device in the remote synchronization system.

[0305] Optionally, in the hardware implementation, the transmission module 802 in the above embodiment corresponds to the transceiver 901 in the embodiment. Figure 8

[0306] ​The system bus can be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus or the like. The system bus can be divided into an address bus, a data bus, a control bus, and the like. For ease of representation, only one thick line is shown in the figure, but it does not mean that there is only one bus or only one type of bus. The transceiver is used to realize the communication between the database access device and other computers (such as clients, read-write libraries and read-only libraries). The memory can include random access memory (RAM) and can also include non-volatile memory.

[0307] The embodiment of the present application also provides a chip for executing the operation instruction, which is used to execute the technical scheme of the remote synchronization method in the above embodiment.

[0308] The embodiment of the present application also provides a computer readable storage medium, which stores computer instructions, and when the computer instructions are executed on a computer, the computer executes the technical scheme of the remote synchronization method in the above embodiment.

[0309] The embodiment of the present application also provides a computer program product, which includes a computer program stored in a computer readable storage medium, at least one processor can read the computer program from the computer readable storage medium, and when the at least one processor executes the computer program, the technical scheme of the remote synchronization method in the above embodiment can be realized.

[0310] In several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of the modules is only a logical function division, and actual implementation can have another division manner, for example, a plurality of modules can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the displayed or discussed modules can be indirect coupling or communication connection through some interfaces, devices or modules, and can be electrical, mechanical or other forms.

[0311] The modules described as separate components can or can not be physically separated, and the components displayed as modules can or can not be physical units, that is, they can be located in one place, or can be distributed on a plurality of network units. According to actual needs, some or all of the modules can be selected to realize the embodiment scheme.

[0312] In addition, each functional module in each embodiment of the present application can be integrated in one processing unit, or each module can be physically present alone, or two or more modules can be integrated in one unit. The above-mentioned modules can be realized in the form of hardware or in the form of hardware plus software functional modules.

[0313] The integrated modules realized in the form of software functional modules can be stored in a computer readable storage medium. The software functional modules stored in a storage medium include a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor to execute part of the steps of the method of each embodiment of the present application.

[0314] It should be understood that the above-mentioned processor can be a central processing unit (CPU), and can also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), etc. The general-purpose processor can be a microprocessor or any conventional processor, etc. The steps of the method disclosed in the present application can be directly embodied as hardware processor execution or executed by a combination of hardware and software modules in the processor.

[0315] The memory can include a high-speed RAM memory, and can also include a non-volatile storage NVM, for example at least one disk memory, and can also be a U disk, a mobile hard disk, a read-only memory, a magnetic disk or an optical disk, etc.

[0316] The bus can be an industry standard architecture (ISA) bus, a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For the convenience of representation, the bus in the drawings of the present application does not limit to only one bus or one type of bus.

[0317] The aforementioned storage medium can be realized by any type of volatile or nonvolatile storage devices or a combination thereof, such as a static random access memory (SRAM), an electrically erasable programmable read-only memory (EEPROM), an erasable programmable read-only memory (EPROM), a programmable read-only memory (PROM), a read-only memory (ROM), a magnetic storage, a flash memory, a magnetic disk or an optical disk. The storage medium can be any available medium that can be accessed by a general or special purpose computer.

[0318] An exemplary storage medium is coupled to the processor so that the processor can read information from, and write information to, the storage medium. Of course, the storage medium can be a part of the processor. The processor and the storage medium can be located in an application specific integrated circuit (ASIC). Of course, the processor and the storage medium can exist as discrete components in the electrical control unit or the host device.

[0319] Those of ordinary skill in the art can understand that all or part of the steps of the above-mentioned method embodiments can be completed by relevant hardware instructed by programs. The aforementioned programs can be stored in a computer readable storage medium. When the programs are executed, the steps of the above-mentioned method embodiments are executed; and the aforementioned storage medium includes various storage media that can store program codes, such as a ROM, a RAM, a magnetic disk or an optical disk.

[0320] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than limit the same; although the present application has been described in detail with reference to the above embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions recorded in the above embodiments, or make equivalent replacements to some or all of the technical features; and these modifications or replacements do not cause the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present application.

Claims

1. A remote synchronization system, characterized in that, include: The first forwarding device, the second forwarding device, and the block transmission subsystem; The first forwarding device is used to forward data exchanged between the first source code repository and the chunked transmission subsystem, and the second forwarding device is used to forward data exchanged between the second source code repository and the chunked transmission subsystem. The chunked transmission subsystem is used to transmit multiple Git object data blocks one by one to the first forwarding device, and is also used to retransmit Git object data blocks that fail to be transmitted. The multiple Git object data blocks belong to target Git object data, which is the Git object data corresponding to the remote repository update request sent by the first source code repository to the second source code repository. The segmented transmission subsystem includes: a first transmission unit and a second transmission unit; The first transmission unit, the first forwarding device, and the first source code repository are all located in a first network environment, while the second transmission unit, the second forwarding device, and the second source code repository are all located in a second network environment. The first network environment and the second network environment are different. The first transmission unit is used to send a first control message to the second transmission unit, the first control message being used to indicate that the target Git object data will be transmitted in the form of data blocks; The first transmission unit is further configured to continuously send data packets to the second transmission unit, each data packet including the Git object data block; The first transmission unit is further configured to send a second control message to the second transmission unit after all Git object data blocks have been sent, the second control message being used to indicate that the target Git object data transmission is complete; The second transmission unit is used to send a response message to the first transmission unit after receiving the first control message, the second control message, or the data message; The second transmission unit is also used to transmit the received Git object data blocks to the first forwarding device in the form of a data stream.

2. The system according to claim 1, characterized in that, The first transmission unit is specifically used for: For any data packet, the timer starts after the data packet is sent to the first forwarding device, and the cumulative duration is obtained; If the cumulative time reaches the preset time and no target response message is obtained, the data packet is resent to the first forwarding device, and the target response message is the response message corresponding to the data packet.

3. The system according to claim 2, characterized in that, The preset duration is set in advance based on the preset data length, which is used to divide the target Git object data into multiple Git object data blocks.

4. The system according to any one of claims 1-3, characterized in that, The first forwarding device is further configured to: Obtain the target data block identifier corresponding to the target Git object data block that has been read and sent by the first source code repository; The target data block identifier is sent to the block transmission subsystem.

5. The system according to claim 4, characterized in that, The segmented transmission subsystem is also used for: Obtain the target data block identifier sent by the first forwarding device; If the target Git object data block corresponding to the target data block identifier exists in the untransmitted Git object data block, then the transmission process for the target Git object data block is cancelled.

6. A remote synchronization method, characterized in that, The method, applied to a chunked transmission subsystem in a remote synchronization system as described in any one of claims 1 to 5, comprises: Obtain multiple Git object data blocks corresponding to the target Git object data, wherein the target Git object data is the Git object data corresponding to the remote repository update request sent from the first source code repository to the second source code repository; Transmit multiple Git object data blocks one by one to the first forwarding device; If the transfer of any Git object data block fails, the failed data block will be retransmitted.

7. The method according to claim 6, characterized in that, The chunked transmission subsystem includes a first transmission unit and a second transmission unit. The step of transmitting multiple Git object data blocks one by one to the first forwarding device includes: The first control message is sent from the first transmission unit to the second transmission unit. The first control message is used to indicate that the target Git object data will be transmitted in the form of data blocks. Data packets are continuously sent to the first forwarding device through the first transmission unit and the second transmission unit, and each data packet includes the Git object data block; After all Git object data blocks have been sent, a second control message is sent from the first transmission unit to the second transmission unit. The second control message is used to indicate that the target Git object data transmission is complete.

8. The method according to claim 7, characterized in that, The retransmission of the failed data block when the transmission of any Git object data block fails includes: For any data packet, timing begins after the data packet is sent from the first transmission unit to the second transmission unit, and the cumulative duration is obtained; If the cumulative duration reaches the preset duration and the first transmission unit does not obtain the target response message, the data packet will be resent to the second transmission unit, and the target response message is the response message corresponding to the data packet.

9. The method according to claim 8, characterized in that, The preset duration is set in advance based on the preset data length, which is used to divide the target Git object data into multiple Git object data blocks.

10. The method according to claim 6 or 7, characterized in that, The method further includes: Obtain the target data block identifier, which is the data block identifier corresponding to the target Git object data block that has been read by the first source code repository; If the target Git object data block corresponding to the target data block identifier exists in the untransmitted Git object data block, then the transmission process for the target Git object data block is cancelled.

11. A remote synchronization device, characterized in that, The device is applied to a chunked transmission subsystem in a remote synchronization system as described in any one of claims 1 to 5, the device comprising: The acquisition module is used to acquire multiple Git object data blocks corresponding to the target Git object data, wherein the target Git object data is the Git object data corresponding to the remote repository update request sent from the first source code repository to the second source code repository; The transmission module is used to transmit multiple Git object data blocks one by one to the first forwarding device; The transmission module is also used to retransmit the failed data block when the transmission of any Git object data block fails.

12. A chunked transmission subsystem, characterized in that, include: A transceiver, a processor, and a memory communicatively connected to the processor; The memory stores computer-executed instructions; The transceiver is used to communicate with the first and second forwarding devices in the remote synchronization system as described in any one of claims 1 to 5; The processor executes computer execution instructions stored in the memory to implement the method as described in any one of claims 6-10.

13. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the method as described in any one of claims 6-10.

14. A computer program product, characterized in that, Includes a computer program that, when executed by a processor, implements the method of any one of claims 6-10.

Images