A distributed image breakpoint resume transmission method and device based on shared storage

Abstract

The present invention provides a distributed image breakpoint-resume transmission method and device based on shared storage, which relates to a distributed system and is applied to a server. The method comprises: receiving a connection request from a client to establish a first connection channel between the client and a first server; receiving a file upload command packet sent by the client through the first connection channel, and determining a storage path of the image file in the shared storage server; receiving the image file, storing it in the shared storage server according to the storage path, and transmitting the image file; if a transmission interruption occurs during the transmission of the image file, receiving a reconnection request from the client, determining a second server of the server according to a preset load balancing strategy, and establishing a second connection channel between the client and the second server; determining, by the second server, the residual files stored in the shared storage server when the transmission is interrupted according to the reconnection request from the client, and performing breakpoint-resume transmission on the image file that has not been completely transmitted through the second connection channel.

Description

Technical Field

[0001] The present invention relates to the technical field of computer data processing, and in particular to a distributed image breakpoint resume transmission method and device based on shared storage. Background Art

[0002] This section is intended to provide a background or context to embodiments of the invention that are recited in the claims. No description herein is admitted to be prior art by inclusion in this section.

[0003] In the existing distributed image file transmission system, the client and the server are bridged and shielded through the load balancer. In order to relieve the service pressure on the server, the server generally sets up multiple servers according to the distributed architecture. When the client initiates a file transfer, the load balancer selects one of the multiple servers on the server to receive the file. At this time, if the transmission is interrupted for some reason, when the client uploads the file again, the load balancer reselects one of the multiple servers on the server to receive the file. According to the server selection strategy of the load balancer, it cannot be guaranteed that the server for the two transmissions is the same server. Therefore, the existing distributed image file transmission based on the load balancing scenario cannot realize the breakpoint resume function.

[0004] Therefore, how to provide a new solution that can solve the above technical problems is a technical problem that needs to be solved urgently in this field. Summary of the invention

[0005] The embodiment of the present invention provides a distributed image breakpoint resume method based on shared storage, which is applied to a server. By adding and designing a shared storage mechanism, different connection channels and different servers are realized. The unique target storage path can be locked according to the file information, so as to find the original residual files and realize the breakpoint resume function. The method includes:

[0006] Receive a connection request from a client, determine a first server on the server side according to a preset load balancing strategy, and establish a first connection channel between the client and the first server;

[0007] Receiving a file upload command packet sent by the client through the first connection channel, and determining a storage path of the image file in the shared storage server;

[0008] Receiving the image file sent by the client through the first connection channel, storing the image file in the shared storage server according to the storage path, and transmitting the image file;

[0009] If the transmission between the client and the server is interrupted during the transmission of the image file, a reconnection request from the client is received, a second server of the server is determined according to a preset load balancing strategy, and a second connection channel between the client and the second server is established;

[0010] The second server determines the residual files stored when the transmission is interrupted in the shared storage server according to the reconnection request of the client, and performs breakpoint-resume transmission of the image files that have not been completely transmitted through the second connection channel;

[0011] If the transmission between the client and the server is interrupted during the transmission of the image file, a reconnection request from the client is received, a second server of the server is determined according to a preset load balancing strategy, and a second connection channel between the client and the second server is established, including:

[0012] If the transmission between the client and the server is interrupted during the transmission of the image file, the first connection channel between the client and the first server is disconnected;

[0013] Receive a reconnection request from the client to the load balancer, and the load balancer determines the second server in the server according to the preset load balancing strategy and the reconnection request, and sends the reconnection request to the second server;

[0014] After the second server confirms that the reconnection request is valid, it shakes hands with the client, establishes a second connection channel between the client and the second server, generates a connection success response, and forwards it to the client via the load balancer;

[0015] Also includes:

[0016] Set a read-write lock or inter-process communication during a write operation.

[0017] The embodiment of the present invention further provides a distributed image breakpoint resume transmission device based on shared storage, which is applied to a server, including:

[0018] A first connection channel establishing module, used to receive a connection request from a client, determine a first server on the server side according to a preset load balancing strategy, and establish a first connection channel between the client and the first server;

[0019] A command transmission module, used to receive a file upload command packet sent by the client through the first connection channel, and determine a storage path of the image file in the shared storage server;

[0020] An image file transmission module is used to receive an image file sent by the client through the first connection channel, store the image file in a shared storage server according to a storage path, and transmit the image file;

[0021] A transmission interruption reconnection module is used to receive a reconnection request from the client if a transmission interruption occurs between the client and the server during the transmission of the image file, determine a second server on the server side according to a preset load balancing strategy, and establish a second connection channel between the client and the second server;

[0022] A breakpoint resume module, used to determine the residual files stored when the transmission is interrupted in the shared storage server according to the reconnection request of the client through the second server, and resume the image files that have not been completely transmitted through the second connection channel;

[0023] Among them, the transmission interruption reconnection module is specifically used for:

[0024] If the transmission between the client and the server is interrupted during the transmission of the image file, the first connection channel between the client and the first server is disconnected;

[0025] Receive a reconnection request from the client to the load balancer, and the load balancer determines the second server in the server according to the preset load balancing strategy and the reconnection request, and sends the reconnection request to the second server;

[0026] After the second server confirms that the reconnection request is valid, it shakes hands with the client, establishes a second connection channel between the client and the second server, generates a connection success response, and forwards it to the client via the load balancer;

[0027] The breakpoint resume module is specifically used for:

[0028] Set a read-write lock or inter-process communication during a write operation.

[0029] An embodiment of the present invention also provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the above-mentioned distributed image breakpoint resume transmission method based on shared storage when executing the computer program.

[0030] An embodiment of the present invention further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the above-mentioned distributed image breakpoint resume transmission method based on shared storage is implemented.

[0031] An embodiment of the present invention further provides a computer program product, which includes a computer program. When the computer program is executed by a processor, the distributed image breakpoint resume transmission method based on shared storage is implemented.

[0032] A distributed image breakpoint resume method and device based on shared storage provided by an embodiment of the present invention is applied to a server, including: receiving a connection request from a client, determining a first server of the server according to a preset load balancing strategy, and establishing a first connection channel between the client and the first server; receiving a file upload command packet sent by the client through the first connection channel, and determining the storage path of the image file in the shared storage server; receiving an image file sent by the client through the first connection channel, and storing the image file in the shared storage server according to the storage path to transmit the image file; if a transmission interruption occurs between the client and the server during the transmission of the image file, receiving a reconnection request from the client, determining a second server of the server according to a preset load balancing strategy, and establishing a second connection channel between the client and the second server; determining the residual files stored in the shared storage server when the transmission is interrupted according to the reconnection request of the client through the second server, and performing breakpoint resume on the image file that has not been transmitted through the second connection channel. The present invention is mainly to solve the problem that in the existing distributed image management system, the image file breakpoint resume function cannot be realized because the IP and port of the server are in a protected state and invisible to the client in the scenario of using a load balancer. The present invention solves the above problem by introducing a shared storage mechanism in the distributed image management system. The present invention adds a shared storage server on the server side by designing a reasonable shared storage mechanism. All servers of the server can connect to the shared storage server. All servers on the server side will land the received files on the shared storage server at a unified fixed point, ensuring that the same image file transmission lands on the server side as the same cache file. When a transmission exception occurs and the file transfer is initiated again, the server side accurately finds the file that was not completed last time from the shared storage server and implements breakpoint resume based on the file. The core of the present invention is that the shared storage server is used for storage during the storage process of the file transfer to the server. Although each transmission selects any different server through the load balancer and establishes different connection channels, the same storage location can still be locked according to the file information of the connection request, and the breakpoint position of the residual file that has not been completed is found, so as to realize the breakpoint resume function in the load balancing scenario. The present invention realizes different connection channels and different servers by adding and designing a shared storage mechanism, and can complete the unique target storage path locking according to the file information, so as to find the original residual file and realize the breakpoint resume function. Introducing the shared storage mechanism provided by the present invention into the existing image management system can realize the breakpoint resume function in the load balancing scenario under the distributed architecture, thereby improving the stability of the system, the file transfer efficiency and greatly improving the customer experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the prior art descriptions. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work. In the drawings:

[0034] Figure 1 The present invention is a schematic diagram of a distributed image breakpoint resume transmission method based on shared storage according to an embodiment of the present invention.

[0035] Figure 2 The present invention is a framework diagram of an image management system that applies a distributed image breakpoint resume method based on shared storage according to an embodiment of the present invention.

[0036] Figure 3 This is a diagram of the image file transmission process of a distributed image breakpoint resume transmission method based on shared storage according to an embodiment of the present invention.

[0037] Figure 4 A schematic diagram of a computer device for running a distributed image breakpoint resume transmission method based on shared storage implemented by the present invention.

[0038] Figure 5 The present invention is a schematic diagram of a distributed image breakpoint resume transmission device based on shared storage according to an embodiment of the present invention. DETAILED DESCRIPTION

[0039] To make the purpose, technical solution and advantages of the embodiments of the present invention more clear, the embodiments of the present invention are further described in detail below in conjunction with the accompanying drawings. Here, the exemplary embodiments of the present invention and their descriptions are used to explain the present invention, but are not intended to limit the present invention.

[0040] The present invention relates to distributed technology. Figure 1 FIG. 1 is a schematic diagram of a distributed image breakpoint resume transmission method based on shared storage according to an embodiment of the present invention. Figure 1 As shown, an embodiment of the present invention provides a distributed image breakpoint resume method based on shared storage, which is applied to a server. By adding and designing a shared storage mechanism, different connection channels and different servers are realized, and a unique target storage path can be locked according to file information, so as to find the original residual files and realize the breakpoint resume function. The method includes:

[0041] Step 101: receiving a connection request from a client, determining a first server on the server side according to a preset load balancing strategy, and establishing a first connection channel between the client and the first server;

[0042] Step 102: receiving a file upload command packet sent by the client through the first connection channel, and determining a storage path of the image file in the shared storage server;

[0043] Step 103: receiving the image file sent by the client through the first connection channel, storing the image file in the shared storage server according to the storage path, and transmitting the image file;

[0044] Step 104: if the transmission between the client and the server is interrupted during the transmission of the image file, a reconnection request from the client is received, a second server of the server is determined according to a preset load balancing strategy, and a second connection channel between the client and the second server is established;

[0045] Step 105: The second server determines the residual files stored when the transmission is interrupted in the shared storage server according to the reconnection request of the client, and performs breakpoint-resume transmission of the image files that have not been completely transmitted through the second connection channel.

[0046] A distributed image breakpoint resume method based on shared storage provided by an embodiment of the present invention is applied to a server, including: receiving a connection request from a client, determining a first server of the server according to a preset load balancing strategy, and establishing a first connection channel between the client and the first server; receiving a file upload command packet sent by the client through the first connection channel, and determining the storage path of the image file in the shared storage server; receiving an image file sent by the client through the first connection channel, and storing the image file in the shared storage server according to the storage path to transmit the image file; if a transmission interruption occurs between the client and the server during the transmission of the image file, receiving a reconnection request from the client, determining a second server of the server according to a preset load balancing strategy, and establishing a second connection channel between the client and the second server; determining the residual files stored in the shared storage server when the transmission is interrupted according to the reconnection request of the client through the second server, and performing breakpoint resume on the image file that has not been transmitted through the second connection channel. The present invention is mainly to solve the problem that in the existing distributed image management system, the image file breakpoint resume function cannot be realized because the IP and port of the server are in a protected state and invisible to the client in the scenario of using a load balancer. The present invention solves the above problem by introducing a shared storage mechanism in the distributed image management system. The present invention adds a shared storage server on the server side by designing a reasonable shared storage mechanism. All servers of the server can connect to the shared storage server. All servers on the server side will land the received files on the shared storage server at a unified fixed point, ensuring that the same image file transmission lands on the server side as the same cache file. When a transmission exception occurs and the file transfer is initiated again, the server side accurately finds the file that was not completed last time from the shared storage server and implements breakpoint resume based on the file. The core of the present invention is that the shared storage server is used for storage during the storage process of the file transfer to the server. Although each transmission selects any different server through the load balancer and establishes different connection channels, the same storage location can still be locked according to the file information of the connection request, and the breakpoint position of the residual file that has not been completed is found, so as to realize the breakpoint resume function in the load balancing scenario. The present invention realizes different connection channels and different servers by adding and designing a shared storage mechanism, and can complete the unique target storage path locking according to the file information, so as to find the original residual file and realize the breakpoint resume function. Introducing the shared storage mechanism provided by the present invention into the existing image management system can realize the breakpoint resume function in the load balancing scenario under the distributed architecture, thereby improving the stability of the system, the file transfer efficiency and greatly improving the customer experience.

[0047] Figure 2 The framework diagram of an image management system for applying a distributed image breakpoint resume transmission method based on shared storage according to an embodiment of the present invention is shown as follows: Figure 2As shown, the image management system includes a client, a load balancer and a server.

[0048] The above-mentioned image management system is built based on a distributed framework, for example, it can be a distributed image management system (Distributed Image Management System), which is used to upload image files (such as photos, videos, contract scans, etc.) to the service storage through the network for query, call, modification, deletion and other operations. In the image management system, the client can include multiple clients; the server includes: a shared storage server, multiple business-related servers, each server is connected to the shared storage server. Therefore, each server can access the shared storage server.

[0049] The client may include multiple Socket clients; the server may include multiple Socket servers. The embodiment of the present invention implements breakpoint resume based on Socket technology.

[0050] The aforementioned SOCKET, which stands for socket, is used to describe IP address and port, and is the handle of the communication chain. Applications can send requests to the network or respond to network requests through Socket. Socket is the basic operation unit of network communication that supports TCP / IP protocol. It is an abstract representation of endpoints in the process of network communication, and contains five kinds of information necessary for network communication: the protocol used for the connection, the IP address of the local host, the local remote protocol port, the IP address of the remote host, and the protocol port of the remote process.

[0051] The aforementioned load balancer is mainly used for load balancing in the image file transmission process. Its purpose is to balance and distribute the load (work tasks) to multiple operating units for operation, such as FTP servers, Web servers, enterprise core application servers and other major task servers, so as to complete the work tasks in a coordinated manner. Load balancing is a key component of high-availability network infrastructure, and is usually used to distribute workloads to multiple servers to improve the performance and reliability of websites, applications, databases or other services.

[0052] The aforementioned broken-point continuing-transferring means that when downloading files through the network, if the download fails due to some reason, you do not have to start the download from the beginning, but continue downloading from the specified location. This function is called broken-point continuing-transferring. The understanding of broken-point continuing-transferring can be divided into two parts: one is the breakpoint, and the other is continuing. The origin of the breakpoint is that the task is paused during the download process. The location where the download is paused is the breakpoint. Resuming transfer means that when an unfinished download task is started again, it will continue to be transmitted from the last breakpoint.

[0053] In the existing point-to-point resumable transmission scheme, the client initiates the image file transmission request as the sender, and the load balancer selects a server A from the server side according to the established strategy as the receiving server for the transmitted file. When the transmission is interrupted due to network anomalies and other problems, the client initiates the connection request again, and the load balancer selects a server B from the server side according to the established strategy (note: there is no guarantee that B and A are the same machine) as the new file transmission receiving end. Therefore, the original residual files on server A cannot be found on the receiving end of server B, and the breakpoint resumable transmission function cannot be realized.

[0054] Most of the existing breakpoint resume technologies are based on Socket technology, and the existing image management system also implements file transfer based on Socket. Figure 2 The image management system shown in the figure can be roughly divided into three types of network nodes: client, load balancer, and server. Only the node modules strongly related to the present invention are listed here. The object storage server, encryption and decryption server, database-related server, etc. not involved in the present invention are not listed. Considering security and other reasons, the IP and port of the server are only exposed to the load balancer. The load balancer is connected to the client as a bridge, and the client and server networks are not interoperable. The image management system pre-runs the Socket server on several server-side servers (such as 8) to continuously monitor the connection request information of the client. When a customer wants to upload an image file involved in a transaction information to the background server for unified management, it is necessary to initiate a Socket connection request from the client, and after establishing a connection channel with a server-side server (such as server A among the 8 servers) through the load balancer, the file is transferred to server A. If the transmission is interrupted due to network abnormalities and other reasons, after the client initiates the request again, it is expected that the breakpoint can be resumed based on the partial file received by server A before the interruption. However, in the existing design implementation, when the client initiates a request again, the load balancer will select one B from the eight background servers according to the established strategy (there is no guarantee that B and A are the same machine), which makes it impossible to implement the breakpoint resume function. The present invention designs a file transfer shared storage mechanism, that is, a file transfer shared storage server is added to store the files uploaded by the client, so that the original files can be found stably and accurately in the breakpoint resume scenario.

[0055] An embodiment of the present invention provides a distributed image breakpoint resume method based on shared storage, in which a shared storage server N is set on the server side to store files received by all server-side servers. When the client initiates a transmission request as the sender, the load balancer randomly selects a server A from the server side according to the established strategy, and only provides receiving services. The received files are stored in the shared storage server N. When the transmission is interrupted due to network anomalies and other problems, the client initiates a connection request again, and the load balancer re-selects a server B from the server side according to the established strategy (Note: there is no guarantee that B and A are the same machine), which also only provides receiving services, and the received files are still stored in the shared storage server N. At this time, the image file receiving service on server B can find the original residual files that have not been fully transmitted on the shared storage server N, thereby realizing the breakpoint resume function.

[0056] Specific implementation of a distributed image breakpoint resume transmission method based on shared storage provided by an embodiment of the present invention is applied to a server. In one embodiment, the method may include:

[0057] Receive a connection request from a client, determine a first server on the server side according to a preset load balancing strategy, and establish a first connection channel between the client and the first server;

[0058] Receiving a file upload command packet sent by the client through the first connection channel, and determining a storage path of the image file in the shared storage server;

[0059] Receiving the image file sent by the client through the first connection channel, storing the image file in the shared storage server according to the storage path, and transmitting the image file;

[0060] If the transmission between the client and the server is interrupted during the transmission of the image file, a reconnection request from the client is received, a second server of the server is determined according to a preset load balancing strategy, and a second connection channel between the client and the second server is established;

[0061] The second server determines the residual files stored when the transmission is interrupted in the shared storage server according to the reconnection request of the client, and performs breakpoint-resume transmission of the image files that have not been completely transmitted through the second connection channel.

[0062] The present invention is mainly to solve the problem that in the existing distributed image management system, in the scenario of using a load balancer, the IP and port of the server are in a protected state and invisible to the client, and the image file breakpoint resume function cannot be realized. The present invention solves the above problem by introducing a shared storage mechanism in the distributed image management system. The present invention adds a shared storage server to the server by designing a reasonable shared storage mechanism. The servers of the server can all connect to the shared storage server. All servers on the server side will land the received files on the shared storage server at a unified fixed point, ensuring that the same image file transmission lands on the server as the same cache file. When a transmission exception occurs and the file transfer is initiated again, the server accurately finds the file that was not completed last time from the shared storage server and realizes breakpoint resume based on the file. The core of the present invention is that the shared storage server is used for storage during the storage process of the file transfer to the server. Although each transmission selects any different server through the load balancer and establishes different connection channels, the same storage location can still be locked according to the file information of the connection request, and the breakpoint position of the residual file that has not been completed is found, so as to realize the breakpoint resume function in the load balancing scenario. The present invention realizes different connection channels and different servers by adding and designing a shared storage mechanism, and can complete the unique target storage path locking according to the file information, so as to find the original residual files and realize the breakpoint resume function. Introducing the shared storage mechanism provided by the present invention into the existing image management system can realize the breakpoint resume function in the load balancing scenario under the distributed architecture, thereby improving the system stability, file transmission efficiency and greatly improving the customer experience.

[0063] An image management system using a distributed image breakpoint resume method based on shared storage according to an embodiment of the present invention mainly includes: a client, a load balancer, and a server; the aforementioned client includes: a Socket client; the aforementioned server includes: a Socket server and a shared storage server.

[0064] Socket client: A software module belonging to the client. When a user initiates a request to upload an image file, this module initiates information transmission with the server, including command messages, image file data, etc.

[0065] Load balancer: Use a hardware load balancer to select a server from the server side for message routing based on the preset load balancing strategy.

[0066] Socket server: A software module belonging to the server. It starts the Socket service and continuously listens to messages routed by the load balancer, including executing commands, receiving files, storing files, returning messages, etc.

[0067] Shared storage server: a server used to store uploaded files. All servers on the server side can access it so that they can find the original breakpoint files during the breakpoint resume process and implement the breakpoint resume function.

[0068] Figure 3 FIG. 4 is a diagram of an image file transmission process of a distributed image breakpoint resume transmission method based on shared storage according to an embodiment of the present invention. Figure 3 As shown, when a distributed image breakpoint resume transmission method based on shared storage provided by an embodiment of the present invention is specifically implemented, in one embodiment, a connection request from a client is received, a first server of the server is determined according to a preset load balancing strategy, and a first connection channel between the client and the first server is established, including:

[0069] Receive a connection request from the client to the load balancer, and the load balancer determines the first server in the server according to the preset load balancing strategy and the connection request, and sends the connection request to the first server;

[0070] After the first server confirms that the connection request is valid, it shakes hands with the client, establishes a first connection channel between the client and the first server, generates a connection success response, and forwards it to the client via the load balancer.

[0071] In the embodiment, during the connection phase, the Socket server software module is run on all Socket servers on the server side, and the connection request is continuously listened. At this time, after the user initiates a file upload request through the client, the Socket client module initiates a connection request to the server side. After the load balancer receives this request, it selects a server A on the server side according to the preset load balancing strategy and sends the connection request to server A. After server A confirms that the connection request is valid and completes multiple handshakes, it establishes the first Socket connection channel 1 and feeds back the result information to the client.

[0072] When implementing a distributed image breakpoint resume transmission method based on shared storage provided by an embodiment of the present invention, in one embodiment, receiving a file upload command packet sent by a client through a first connection channel, and determining a storage path of the image file in a shared storage server, includes:

[0073] receiving a file upload command packet sent by the client through the first connection channel, and transmitting the command packet to the first server;

[0074] The command packet is parsed by the first server to determine the storage path of the image file in the shared storage server.

[0075] In the embodiment, after receiving a successful connection response, the client confirms the establishment of the first connection channel and begins to enter the command transmission phase. The client sends a file upload command packet through the Socket first connection channel. The server receives the file upload command packet sent by the client through the first connection channel and transmits it to the first server. The command packet is parsed by the first server to determine the storage path of the image file in the shared storage server.

[0076] The present invention designs a reasonable shared storage mechanism, adds a shared storage server on the service side, and all servers of the server can connect to the shared storage server. All servers on the service side will land the received files on the shared storage server at a unified fixed point, ensuring that the same image file transmission lands on the server side as the same cache file.

[0077] When implementing a distributed image breakpoint resume method based on shared storage provided by an embodiment of the present invention, in one embodiment, after determining the storage path of the image file in the shared storage server, the method further includes:

[0078] Initialize the storage path of the shared storage server and assign the file offset to zero; the file offset represents the file size of the currently transmitted image file;

[0079] The file offset is assembled into a successful execution command packet and returned to the client through the first connection channel.

[0080] In an embodiment, after determining the storage path of the image file in the shared storage server, the storage path of the shared storage server is initialized. At this time, there is no image file stored in the storage path of the shared storage server, that is, the file size of the currently transmitted image file is zero, so the file offset is assigned to zero, and the file offset is assembled into a successful execution command package, which is returned to the client through the first connection channel.

[0081] In the command transmission phase, the Socket client sends a file upload command message through the established Socket connection channel 1, the content of which includes the file to be transmitted and its size, including: data packet type, request type, file name, file length, file offset, file path and other information. After receiving and parsing the file upload command packet, server A calculates the storage path on the shared storage server, completes the relevant initialization actions of the storage path, reads whether the file in the storage path exists, and reads the file size if it exists. If not, the file size of the currently transmitted image file is zero. After server A completes information processing and file information acquisition, it assembles the relevant information into a successful execution command packet and returns it to the client. After receiving the command packet, the client can parse whether the file to be transmitted already exists. If it does, the size of the transmitted file can be obtained through the file offset. Since there is no stored image file in the storage path at this time, the file offset is zero.

[0082] When implementing a distributed image breakpoint resume transmission method based on shared storage provided by an embodiment of the present invention, in one embodiment, receiving an image file sent by a client through a first connection channel, storing the image file in a shared storage server according to a storage path, and transmitting the image file includes:

[0083] Receiving, through the first connection channel, an image file data packet determined by the client in response to the successful execution command packet; the image file data packet is an image file data packet obtained by the client dividing the image file into a plurality of image file data packets of set file sizes;

[0084] The image file data packet is stored in the shared storage server according to the storage path, the file offset of the image file already existing in the storage path is updated, the file offset of the next image file data packet to be transmitted is determined, and the data packet transmission response is assembled;

[0085] Returning the data packet transmission response to the client via the first connection channel;

[0086] The next image file data packet determined by the client in response to the data packet transmission response is received through the first connection channel, and is appended to the existing image file in the storage path, and the file offset of the image file already existing in the storage path is updated, and the file offset of the next image file data packet to be transmitted is determined, and the data packet transmission response is assembled, and the execution is repeated to return the data packet transmission response to the client through the first connection channel to transmit the image file.

[0087] In the embodiment, during the file transfer phase:

[0088] The client, in response to the successful execution command packet, divides the image file into a plurality of image file data packets of set file sizes;

[0089] The server receives the image file data packet determined by the client in response to the successful execution command packet through the first connection channel; stores the image file data packet to the shared storage server according to the storage path, updates the file offset of the image file already existing in the storage path, determines the file offset of the next image file data packet to be transmitted, and assembles the data packet transmission response; and returns the data packet transmission response to the client through the first connection channel;

[0090] The client determines the next image file data packet in response to the data packet transmission response;

[0091] The server receives the next image file data packet determined by the client in response to the data packet transmission response through the first connection channel, appends it to the existing image file in the storage path, updates the file offset of the image file already existing in the storage path at this time, determines the file offset of the next image file data packet to be transmitted, assembles it into the data packet transmission response, and repeatedly returns the data packet transmission response to the client through the first connection channel to transmit the image file.

[0092] Furthermore, in the file transfer stage, the client can parse the information obtained in the command packet to form a data packet (including: data packet type, operation return code, compression flag, file offset, transfer file length, current actual packet data length, etc.) with the remaining part of the file (i.e., the file content from the offset to the file size), and send it out through the established Socket first connection channel 1. After receiving the data packet, the server appends the file content transmitted by the data packet to the file on the shared storage server, and returns the next step of the file offset and size to be transmitted and other related result information.

[0093] When specifically implementing a distributed image breakpoint resumption method based on shared storage provided by an embodiment of the present invention, in one embodiment, the method also includes: if no transmission interruption occurs between the client and the server during the transmission of the image file, when the client determines that the next image file data packet is empty in response to the data packet transmission response, the current image file has been transmitted, and the first connection channel between the client and the first server is disconnected.

[0094] In an embodiment, if there is no transmission interruption between the client and the server during the transmission of the image file, the transmission of the image file data packet will continue. When the client determines that the next image file data packet is empty in response to the data packet transmission response, the current image file has been transmitted and the first connection channel between the client and the first server is disconnected.

[0095] Specifically, when the client receives the data packet transmission response returned by the server, it verifies the file offset and file size to be transmitted next. After finding that the file has been completely transmitted, it sends a message to disconnect the Socket connection. After receiving the command, the server disconnects the socket connection channel 1 after multiple handshakes and returns the result information to the client, thereby completing the disconnection of the entire Socket first connection channel 1.

[0096] When implementing a distributed image breakpoint resume transmission method based on shared storage provided by an embodiment of the present invention, in one embodiment, if a transmission interruption occurs between a client and a server during the transmission of an image file, a reconnection request from the client is received, a second server of the server is determined according to a preset load balancing strategy, and a second connection channel between the client and the second server is established, including:

[0097] If the transmission between the client and the server is interrupted during the transmission of the image file, the first connection channel between the client and the first server is disconnected;

[0098] Receive a reconnection request from the client to the load balancer, and the load balancer determines the second server in the server according to the preset load balancing strategy and the reconnection request, and sends the reconnection request to the second server;

[0099] After the second server confirms that the reconnection request is valid, it shakes hands with the client, establishes a second connection channel between the client and the second server, generates a connection success response, and forwards it to the client via the load balancer.

[0100] In an embodiment, if the transmission between the client and the server is interrupted due to network anomalies or other reasons during the transmission of the image file, the first connection channel between the client and the first server is disconnected, which will cause part of the image file to be transmitted successfully and the other part to be transmitted unsuccessfully. The successfully transmitted part will form a residual file. In order to resume the transmission based on the residual file, the client needs to continue to send connection requests, receive the client's reconnection request to the load balancer, and the load balancer determines the second server in the server according to the preset load balancing strategy and the reconnection request, and sends the reconnection request to the second server; after the second server confirms that the reconnection request is valid, it shakes hands with the client, establishes a second connection channel between the client and the second server, generates a connection success response, and forwards it to the client via the load balancer.

[0101] When implementing a distributed image breakpoint resume transmission method based on shared storage provided by an embodiment of the present invention, in one embodiment, the second server determines the residual files stored when the transmission is interrupted in the shared storage server according to the reconnection request of the client, and resumes the image files that have not been transmitted through the second connection channel, including:

[0102] Determining, by the second server according to the reconnection request of the client, a storage path in the shared storage server when the client and the first server are performing file transfer;

[0103] Determine, according to the storage path of the shared storage server when the client and the first server are performing file transmission, the residual files stored in the storage path of the shared storage server when the transmission is interrupted;

[0104] Obtaining the file size of the stored residual file, determining the file offset of the stored residual file, assembling the data packet breakpoint transmission response, and returning it to the client through the second connection channel;

[0105] An image file resume transmission data packet determined by the client in response to a data packet breakpoint transmission response is received through a second connection channel, the data packet is appended to the stored residual file, the file offset is updated, the file offset of the next image file resume transmission data packet to be transmitted is determined, and the execution is repeated to perform breakpoint transmission. The image file resume transmission data packet is obtained by the client parsing the file offset of the stored residual file in the data packet breakpoint transmission response, determining the image file that has not been transmitted, and dividing the image file that has not been transmitted into multiple image file resume transmission data packets of set file size.

[0106] In an embodiment, in order to realize breakpoint resume based on residual files, the second server determines the residual files stored when the transmission is interrupted in the shared storage server according to the reconnection request of the client, and performs breakpoint resume for the image files that have not been transferred through the second connection channel, including: first, the second server determines the storage path of the shared storage server when the client and the first server perform file transfer according to the reconnection request of the client; then, according to the storage path of the shared storage server when the client and the first server perform file transfer, determines the residual files stored in the storage path of the shared storage server when the transmission is interrupted; then, obtains the file size of the stored residual files , determine the file offset of the stored residual file, assemble it into a data packet breakpoint transmission response, and return it to the client through the second connection channel; finally, receive the image file resume transmission data packet determined by the client in response to the data packet breakpoint transmission response through the second connection channel, append it to the stored residual file, update the file offset, determine the file offset of the next image file resume transmission data packet to be transmitted, repeat the execution, and perform breakpoint resume transmission; the image file resume transmission data packet is the file offset of the stored residual file parsed by the client according to the data packet breakpoint transmission response, determine the image file that has not been transmitted, and divide the image file that has not been transmitted into multiple image file resume transmission data packets with set file sizes.

[0107] In the breakpoint-resume stage of the embodiment of the present invention, it is assumed that during the first file transfer, the client and server A establish Socket connection channel 1. During the process of Step 3 file transfer, due to network anomalies and other reasons, the file transfer is partially successful, forming a residual file. At this time, Socket connection channel 1 is also disconnected according to the original mechanism. According to the working mechanism of breakpoint-resume, the client automatically initiates a second file transfer request. This time, the client and server B establish Socket connection channel 2. Server B calculates the same path as the first transmission on the shared storage server (the running software module and the file name to be transmitted are the same, and the calculated path is also the same). During the transmission of Step 2 command, the server will find the previous residual file and obtain the file size, store it in the form of an offset and return it to the client. The client can assemble a data packet based on the original residual file offset to achieve breakpoint-resume.

[0108] In an embodiment of the present invention, when a transmission anomaly occurs and a file transfer is initiated again, the server side accurately finds the file that was not completed last time from the shared storage server and implements breakpoint-resume transmission based on the file. The core of the present invention is that a shared storage server is used for storage during the storage process of file transfer to the server side. Although each transmission selects any different server-side server through the load balancer and establishes different connection channels, the same storage location can still be locked according to the file information of the connection request, and the breakpoint location of the residual file that has not been completed is found, so as to realize the breakpoint-resume transmission function in the load balancing scenario. The present invention realizes different connection channels and different servers by adding and designing a shared storage mechanism, and can complete the locking of a unique target storage path according to the file information, so as to find the original residual file and realize the breakpoint-resume transmission function. Introducing the shared storage mechanism provided by the present invention into the existing image management system can realize the breakpoint-resume transmission function in the load balancing scenario under a distributed architecture, thereby improving the stability of the system, the efficiency of file transmission and greatly improving the customer experience.

[0109] The core of the embodiment of the present invention is to use a shared storage server for storage during the file storage process. Although each transmission selects any different server through the load balancer and establishes different Socket connection channels, the same storage location can still be locked according to the file information, and the breakpoint position of the residual file that has not been completed is found, thereby realizing the breakpoint resumption function in the load balancing scenario.

[0110] The following is a brief description of a distributed image breakpoint resume transmission method based on shared storage provided by an embodiment of the present invention in conjunction with a specific scenario:

[0111] A shared storage server N is set up on the server side to store files received by all server-side servers. When the client initiates a transmission request as the sender, the load balancer randomly selects a server A from the server side according to the established strategy, which only provides receiving services, and the received files are stored in the shared storage server N. When network anomalies and other problems cause the transmission to be interrupted, the client initiates a connection request again, and the load balancer re-selects a server B from the server side according to the established strategy (Note: there is no guarantee that B and A are the same machine), which also only provides receiving services, and the received files are still stored in the shared storage server N. At this time, the image file receiving service on server B can find the original residual files that have not been fully transmitted on the shared storage server N, thereby realizing the breakpoint resume function.

[0112] The present invention designs a file transfer shared storage mechanism, that is, a file transfer shared storage server is added to store the files uploaded by the client, so that the original files can be stably and accurately found in the breakpoint resume scenario. By designing a reasonable shared storage mechanism, all servers on the server side will land the received files in a unified fixed-point shared storage, ensuring that the same image file transmission lands on the server side as the same cache file. When a transmission exception occurs and the file transfer is initiated again, the server side accurately finds the file that was not completed last time and implements breakpoint resume based on the file.

[0113] The present invention mainly relates to a client, a load balancer, a service end and a shared storage server.

[0114] Socket client: A software module belonging to the client. When a user initiates a request to upload an image file, this module initiates information transmission with the server, including command messages, image file data, etc.

[0115] Load balancer: Use a hardware load balancer to select a server from the server side for message routing based on the preset load balancing strategy.

[0116] Socket server: A software module belonging to the server. It starts the Socket service and continuously listens to messages routed by the load balancer, including executing commands, receiving files, storing files, returning messages, etc.

[0117] Shared storage server: a server used to store uploaded files. All servers on the server side can access it so that they can find the original breakpoint files during the breakpoint resume process and implement the breakpoint resume function.

[0118] like Figure 3As shown in the figure, the file transfer process in the image management platform, in order to briefly describe the shared storage mechanism, ignores other irrelevant modules and some irrelevant processes in the system, simplifies the detailed message definition of data and command transmission, and is described as follows:

[0119] 1. Step 1 Establish a connection: On all servers on the server side, the Socket server software module is running and continuously listening for connection requests. At this time, after the user initiates a file upload request, the Socket client module initiates a connection request to the server. After the load balancer receives this request, it selects a server A according to the established routing strategy and sends the connection request to server A. After server A confirms that the connection request is valid and completes multiple handshakes (not detailed), it establishes Socket connection channel 1 and feeds back the result information to the client.

[0120] 2. Step 2 Command transmission: The Socket client sends a file upload command message through the established Socket connection channel 1. The content includes the file to be uploaded and its size (specifically including: data packet type, request type, file name, file length, file offset, file path, etc.). After receiving and parsing the command packet, Server A calculates the storage path on the shared storage server and completes the relevant initialization actions for the storage path (such as reading whether the file exists, and if so, reading the file size). After completing information processing and obtaining file information, Server A assembles the relevant information into a command packet and returns it to the client. After receiving the command packet, the client can parse whether the file to be uploaded already exists. If so, the size (offset) of the transferred file can be obtained.

[0121] 3. Step 3 File Transfer: The client can parse the information obtained in the command packet to form a data packet (including: data packet type, operation return code, compression flag, file offset, transfer file length, current actual packet data length, etc.) with the remaining part of the file (i.e., the file content from the offset to the file size), and send it out through the established Socket connection channel 1. After receiving the data packet, the server appends the file content transferred by the data packet to the file on the shared storage server, and returns the next step of the file to be transferred, such as the offset and size of the file.

[0122] 4. Step 4 Disconnection: When the client receives the data packet transmission response returned by the server, it verifies the file offset and file size to be transmitted in the next step. After finding that the file has been completely transmitted, it sends a disconnection command for the Socket connection. After receiving the command, the server disconnects the socket connection channel 1 after multiple handshakes (not expanded in detail) and returns the result information to the client, thus completing the disconnection of the entire Socket connection channel 1.

[0123] Among them, Step 2 command transmission is the core step for realizing breakpoint resume under load balancing scenario, and is also the core mechanism of the present invention. Now the process of breakpoint resume is briefly described as follows: Assuming that during the first file transfer process, the client and server A establish Socket connection channel 1, and in the process of Step 3 file transfer, due to network abnormalities and other reasons, the file transfer is partially successful, forming a residual file. At this time, Socket link channel 1 is also disconnected according to the original mechanism. According to the working mechanism of breakpoint resume, the client automatically initiates the second file transfer request. This time, the client and server B establish Socket link channel 2, and server B calculates the same path as the first transmission on the shared storage server (the running software module and the file name to be transmitted are the same, and the calculated path is also the same). During the transmission of Step 2 command, the service end will find the previous residual file and obtain the file size, store it in the form of offset and return it to the client. The client can assemble a data packet based on the original residual file offset to realize breakpoint resume.

[0124] In the process of implementing the above solution, the technical staff needs to design the solution of the breakpoint-resume transmission process, and complete the overall and detailed design of the solution for the Bank of China's image file transmission system, including the process of command transmission and file transmission, the format definition of command packets and data packets, the creation of shared storage, and a complete lock protection mechanism. The technical staff needs to complete the detailed design and implementation of the program according to the solution, based on the two programming languages ​​​​C and JAVA, using Socket sockets to implement the transmission of the FTP protocol, including the function code of breakpoint-resume transmission, and complete the function development of the FTP breakpoint-resume client and server. The technical staff needs to do sufficient testing and verification to ensure that the newly added breakpoint-resume transmission function can be stably executed, and that other original upload functions, image file query, review, modification, and deletion functions of the system that are affected are normal and stable. The technical staff needs to design a reasonable file storage path algorithm, that is, the software service module running on the server can calculate the same storage path based on the same file information (name, length, format, etc.).

[0125] The core process of the present invention is to use a shared storage server for storage during the file storage process. Although each transmission selects any different server through the load balancer and establishes different Socket connection channels, the same storage location can still be locked according to the file information, and the breakpoint position of the residual file that has not been completed is found, thereby realizing the breakpoint resumption function in the load balancing scenario.

[0126] The core process of the present invention is to use a shared storage mechanism to store files during the file storage process, thereby realizing the breakpoint resume function in the load balancing scenario, which specifically includes the following contents:

[0127] According to the image file information (including name, size, format, etc.), the calculation strategy of the storage path is pre-designed in the server to ensure that the Socket link channel and different server-side servers can accurately lock the unique storage path location. Design a shared storage server to protect disk file data through read-write locks or inter-process communication during write operations. Design a reasonable command package and data packet message format to ensure that the information exchange between the client and the server is complete and reliable.

[0128] The core part of the present invention is: by adding and designing a shared storage mechanism, different Socket link channels and different servers are realized, and a unique target path can be locked according to file information, so as to find the original residual file (if the file has failed to be transmitted before), and realize the breakpoint resume function.

[0129] Introducing the shared storage mechanism provided by the present invention into the existing image management system can realize the breakpoint resume function in the load balancing scenario, thereby improving the stability of the system, the file transmission efficiency and greatly improving the customer experience.

[0130] Figure 4 A schematic diagram of a computer device for running a distributed image breakpoint resume transmission method based on shared storage implemented by the present invention, such as Figure 4 As shown, an embodiment of the present invention further provides a computer device 400, including a memory 410, a processor 420, and a computer program 430 stored in the memory and executable on the processor, wherein the processor implements the above-mentioned distributed image breakpoint resumption method based on shared storage when executing the computer program.

[0131] An embodiment of the present invention further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the above-mentioned distributed image breakpoint resume transmission method based on shared storage is implemented.

[0132] An embodiment of the present invention further provides a computer program product, which includes a computer program. When the computer program is executed by a processor, the distributed image breakpoint resume transmission method based on shared storage is implemented.

[0133] The embodiment of the present invention also provides a distributed image breakpoint resume transmission device based on shared storage, as described in the following embodiment. Since the principle of solving the problem by the device is similar to that of a distributed image breakpoint resume transmission method based on shared storage, the implementation of the device can refer to the implementation of a distributed image breakpoint resume transmission method based on shared storage, and the repeated parts will not be repeated.

[0134] Figure 5FIG. 1 is a schematic diagram of a distributed image breakpoint resume transmission device based on shared storage according to an embodiment of the present invention. Figure 5 As shown, the embodiment of the present invention further provides a distributed image breakpoint resume transmission device based on shared storage, which is applied to the server and may include:

[0135] The first connection channel establishing module 501 is used to receive a connection request from a client, determine a first server on the server side according to a preset load balancing strategy, and establish a first connection channel between the client and the first server;

[0136] The command transmission module 502 is used to receive a file upload command packet sent by the client through the first connection channel, and determine the storage path of the image file in the shared storage server;

[0137] The image file transmission module 503 is used to receive the image file sent by the client through the first connection channel, store the image file in the shared storage server according to the storage path, and transmit the image file;

[0138] The transmission interruption reconnection module 504 is used to receive a reconnection request from the client if a transmission interruption occurs between the client and the server during the transmission of the image file, determine a second server on the server side according to a preset load balancing strategy, and establish a second connection channel between the client and the second server;

[0139] The breakpoint resume module 505 is used to determine the residual files stored when the transmission is interrupted in the shared storage server according to the reconnection request of the client through the second server, and resume the image files that have not been completely transmitted through the second connection channel.

[0140] When implementing a distributed image breakpoint resume transmission device based on shared storage provided by an embodiment of the present invention, in one embodiment, the first connection channel establishment module is specifically used to:

[0141] Receive a connection request from the client to the load balancer, and the load balancer determines the first server in the server according to the preset load balancing strategy and the connection request, and sends the connection request to the first server;

[0142] After the first server confirms that the connection request is valid, it shakes hands with the client, establishes a first connection channel between the client and the first server, generates a connection success response, and forwards it to the client via the load balancer.

[0143] When implementing a distributed image breakpoint resume transmission device based on shared storage provided by an embodiment of the present invention, in one embodiment, a command transmission module is specifically used to:

[0144] receiving a file upload command packet sent by the client through the first connection channel, and transmitting the command packet to the first server;

[0145] The command packet is parsed by the first server to determine the storage path of the image file in the shared storage server.

[0146] When implementing a distributed image breakpoint resume transmission device based on shared storage provided by an embodiment of the present invention, in one embodiment, the command transmission module is further used to:

[0147] Initialize the storage path of the shared storage server and assign the file offset to zero; the file offset represents the file size of the currently transmitted image file;

[0148] The file offset is assembled into a successful execution command packet and returned to the client through the first connection channel.

[0149] When implementing a distributed image breakpoint resume transmission device based on shared storage provided by an embodiment of the present invention, in one embodiment, an image file transmission module is specifically used to:

[0150] Receiving, through the first connection channel, an image file data packet determined by the client in response to the successful execution command packet; the image file data packet is an image file data packet obtained by the client dividing the image file into a plurality of image file data packets of set file sizes;

[0151] The image file data packet is stored in the shared storage server according to the storage path, the file offset of the image file already existing in the storage path is updated, the file offset of the next image file data packet to be transmitted is determined, and the data packet transmission response is assembled;

[0152] Returning the data packet transmission response to the client via the first connection channel;

[0153] The next image file data packet determined by the client in response to the data packet transmission response is received through the first connection channel, and is appended to the existing image file in the storage path, and the file offset of the image file already existing in the storage path is updated, and the file offset of the next image file data packet to be transmitted is determined, and the data packet transmission response is assembled, and the execution is repeated to return the data packet transmission response to the client through the first connection channel to transmit the image file.

[0154] When specifically implementing a distributed image breakpoint resume device based on shared storage provided by an embodiment of the present invention, in one embodiment, the image file transmission module is also used for: if no transmission interruption occurs between the client and the server during the transmission of the image file, when the client determines that the next image file data packet is empty in response to the data packet transmission response, the current image file has been transmitted, and the first connection channel between the client and the first server is disconnected.

[0155] When implementing a distributed image breakpoint resume transmission device based on shared storage provided by an embodiment of the present invention, in one embodiment, a transmission interruption reconnection module is specifically used to:

[0156] If the transmission between the client and the server is interrupted during the transmission of the image file, the first connection channel between the client and the first server is disconnected;

[0157] Receive a reconnection request from the client to the load balancer, and the load balancer determines the second server in the server according to the preset load balancing strategy and the reconnection request, and sends the reconnection request to the second server;

[0158] After the second server confirms that the reconnection request is valid, it shakes hands with the client, establishes a second connection channel between the client and the second server, generates a connection success response, and forwards it to the client via the load balancer.

[0159] When implementing a distributed image breakpoint resume transmission device based on shared storage provided by an embodiment of the present invention, in one embodiment, a breakpoint resume transmission module is specifically used to:

[0160] Determining, by the second server according to the reconnection request of the client, a storage path in the shared storage server when the client and the first server are performing file transfer;

[0161] Determine, according to the storage path of the shared storage server when the client and the first server are performing file transmission, the residual files stored in the storage path of the shared storage server when the transmission is interrupted;

[0162] Obtaining the file size of the stored residual file, determining the file offset of the stored residual file, assembling the data packet breakpoint transmission response, and returning it to the client through the second connection channel;

[0163] An image file resume transmission data packet determined by the client in response to a data packet breakpoint transmission response is received through a second connection channel, the data packet is appended to the stored residual file, the file offset is updated, the file offset of the next image file resume transmission data packet to be transmitted is determined, and the execution is repeated to perform breakpoint transmission. The image file resume transmission data packet is obtained by the client parsing the file offset of the stored residual file in the data packet breakpoint transmission response, determining the image file that has not been transmitted, and dividing the image file that has not been transmitted into multiple image file resume transmission data packets of set file size.

[0164] In summary, an embodiment of the present invention provides a distributed image breakpoint resumption method and device based on shared storage, which is applied to a server, and includes: receiving a connection request from a client, determining a first server of the server according to a preset load balancing strategy, and establishing a first connection channel between the client and the first server; receiving a file upload command packet sent by the client through the first connection channel, and determining a storage path of the image file in the shared storage server; receiving an image file sent by the client through the first connection channel, storing the image file according to the storage path to the shared storage server, and transmitting the image file; if a transmission interruption occurs between the client and the server during the transmission of the image file, receiving a reconnection request from the client, determining a second server of the server according to a preset load balancing strategy, and establishing a second connection channel between the client and the second server; determining, through the second server, the residual files stored when the transmission was interrupted in the shared storage server according to the client's reconnection request, and resuming the image file that has not been completely transmitted through the second connection channel. The present invention is mainly to solve the problem that in the existing distributed image management system, in the scenario of using a load balancer, the IP and port of the server are in a protected state and invisible to the client, and the image file breakpoint resume function cannot be realized. The present invention solves the above problem by introducing a shared storage mechanism in the distributed image management system. The present invention adds a shared storage server to the server by designing a reasonable shared storage mechanism. The servers of the server can all connect to the shared storage server. All servers on the server side will land the received files on the shared storage server at a unified fixed point, ensuring that the same image file transmission lands on the server as the same cache file. When a transmission exception occurs and the file transfer is initiated again, the server accurately finds the file that was not completed last time from the shared storage server and realizes breakpoint resume based on the file. The core of the present invention is that the shared storage server is used for storage during the storage process of the file transfer to the server. Although each transmission selects any different server through the load balancer and establishes different connection channels, the same storage location can still be locked according to the file information of the connection request, and the breakpoint position of the residual file that has not been completed is found, so as to realize the breakpoint resume function in the load balancing scenario. The present invention realizes different connection channels and different servers by adding and designing a shared storage mechanism, and can complete the unique target storage path locking according to the file information, so as to find the original residual files and realize the breakpoint resume function. Introducing the shared storage mechanism provided by the present invention into the existing image management system can realize the breakpoint resume function in the load balancing scenario under the distributed architecture, thereby improving the system stability, file transmission efficiency and greatly improving the customer experience.

[0165] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0166] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0167] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0168] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0169] The specific embodiments described above further illustrate the objectives, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above description is only a specific embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A distributed image breakpoint resume transmission method based on shared storage, characterized in that: Applied to the server, including: Receive a connection request from a client, determine a first server on the server side according to a preset load balancing strategy, and establish a first connection channel between the client and the first server; Receiving a file upload command packet sent by the client through the first connection channel, and determining a storage path of the image file in the shared storage server; Receiving the image file sent by the client through the first connection channel, storing the image file in the shared storage server according to the storage path, and transmitting the image file; If the transmission between the client and the server is interrupted during the transmission of the image file, a reconnection request from the client is received, a second server of the server is determined according to a preset load balancing strategy, and a second connection channel between the client and the second server is established; The second server determines the residual files stored when the transmission is interrupted in the shared storage server according to the reconnection request of the client, and performs breakpoint-resume transmission of the image files that have not been completely transmitted through the second connection channel; If the transmission between the client and the server is interrupted during the transmission of the image file, a reconnection request from the client is received, a second server of the server is determined according to a preset load balancing strategy, and a second connection channel between the client and the second server is established, including: If the transmission between the client and the server is interrupted during the transmission of the image file, the first connection channel between the client and the first server is disconnected; Receive a reconnection request from the client to the load balancer, and the load balancer determines the second server in the server according to the preset load balancing strategy and the reconnection request, and sends the reconnection request to the second server; After the second server confirms that the reconnection request is valid, it shakes hands with the client, establishes a second connection channel between the client and the second server, generates a connection success response, and forwards it to the client via the load balancer; Also includes: Set a read-write lock or inter-process communication during a write operation.

2. The method according to claim 1, characterized in that Receiving a connection request from a client, determining a first server on the server side according to a preset load balancing strategy, and establishing a first connection channel between the client and the first server, including: Receive a connection request from the client to the load balancer, and the load balancer determines the first server in the server according to the preset load balancing strategy and the connection request, and sends the connection request to the first server; After the first server confirms that the connection request is valid, it shakes hands with the client, establishes a first connection channel between the client and the first server, generates a connection success response, and forwards it to the client via the load balancer.

3. The method according to claim 1, characterized in that Receiving a file upload command packet sent by the client through the first connection channel and determining a storage path of the image file in the shared storage server includes: receiving a file upload command packet sent by the client through the first connection channel, and transmitting the command packet to the first server; The command packet is parsed by the first server to determine the storage path of the image file in the shared storage server.

4. The method according to claim 3, characterized in that After determining the storage path of the image file on the shared storage server, it also includes: Initialize the storage path of the shared storage server and assign the file offset to zero; the file offset represents the file size of the currently transmitted image file; The file offset is assembled into a successful execution command packet and returned to the client through the first connection channel.

5. The method according to claim 4, characterized in that Receiving an image file sent by the client through the first connection channel, storing the image file in a shared storage server according to a storage path, and transmitting the image file, including: Receiving, through the first connection channel, an image file data packet determined by the client in response to the successful execution command packet; the image file data packet is an image file data packet obtained by the client dividing the image file into a plurality of image file data packets of set file sizes; The image file data packet is stored in the shared storage server according to the storage path, the file offset of the image file already existing in the storage path is updated, the file offset of the next image file data packet to be transmitted is determined, and the data packet transmission response is assembled; Returning the data packet transmission response to the client via the first connection channel; The next image file data packet determined by the client in response to the data packet transmission response is received through the first connection channel, and is appended to the existing image file in the storage path, the file offset of the image file already existing in the storage path is updated, the file offset of the next image file data packet to be transmitted is determined, and the data packet transmission response is assembled and repeatedly returned to the client through the first connection channel to transmit the image file.

6. The method according to claim 5, characterized in that Also includes: If there is no transmission interruption between the client and the server during the transmission of the image file, when the client determines that the next image file data packet is empty in response to the data packet transmission response, the current image file has been transmitted and the first connection channel between the client and the first server is disconnected.

7. The method according to claim 5, characterized in that The second server determines the residual files stored when the transmission is interrupted in the shared storage server according to the reconnection request of the client, and performs breakpoint-resume transmission of the image files that have not been completely transmitted through the second connection channel, including: Determining, by the second server according to the reconnection request of the client, a storage path in the shared storage server when the client and the first server are performing file transfer; Determine, according to the storage path of the shared storage server when the client and the first server are performing file transmission, the residual files stored in the storage path of the shared storage server when the transmission is interrupted; Obtaining the file size of the stored residual file, determining the file offset of the stored residual file, assembling the data packet breakpoint transmission response, and returning it to the client through the second connection channel; An image file resume transmission data packet determined by the client in response to a data packet breakpoint transmission response is received through a second connection channel, the data packet is appended to the stored residual file, the file offset is updated, the file offset of the next image file resume transmission data packet to be transmitted is determined, and the execution is repeated to perform breakpoint transmission. The image file resume transmission data packet is obtained by the client parsing the file offset of the stored residual file in the data packet breakpoint transmission response, determining the image file that has not been transmitted, and dividing the image file that has not been transmitted into multiple image file resume transmission data packets of set file size.

8. A distributed image breakpoint resume transmission device based on shared storage, characterized in that: Applied to the server, including: A first connection channel establishing module, used to receive a connection request from a client, determine a first server on the server side according to a preset load balancing strategy, and establish a first connection channel between the client and the first server; A command transmission module, used to receive a file upload command packet sent by the client through the first connection channel, and determine a storage path of the image file in the shared storage server; An image file transmission module is used to receive an image file sent by the client through the first connection channel, store the image file in a shared storage server according to a storage path, and transmit the image file; A transmission interruption reconnection module is used to receive a reconnection request from the client if a transmission interruption occurs between the client and the server during the transmission of the image file, determine a second server on the server side according to a preset load balancing strategy, and establish a second connection channel between the client and the second server; A breakpoint resume module, used to determine the residual files stored when the transmission is interrupted in the shared storage server according to the reconnection request of the client through the second server, and resume the image files that have not been completely transmitted through the second connection channel; Among them, the transmission interruption reconnection module is specifically used for: If the transmission between the client and the server is interrupted during the transmission of the image file, the first connection channel between the client and the first server is disconnected; Receive a reconnection request from the client to the load balancer, and the load balancer determines the second server in the server according to the preset load balancing strategy and the reconnection request, and sends the reconnection request to the second server; After the second server confirms that the reconnection request is valid, it shakes hands with the client, establishes a second connection channel between the client and the second server, generates a connection success response, and forwards it to the client via the load balancer; The breakpoint resume module is specifically used for: Set a read-write lock or inter-process communication during a write operation.

9. The device according to claim 8, characterized in that The first connection channel establishing module is specifically used for: Receive a connection request from the client to the load balancer, and the load balancer determines the first server in the server according to the preset load balancing strategy and the connection request, and sends the connection request to the first server; After the first server confirms that the connection request is valid, it shakes hands with the client, establishes a first connection channel between the client and the first server, generates a connection success response, and forwards it to the client via the load balancer.

10. The device according to claim 8, characterized in that Command transmission module, specifically used for: receiving a file upload command packet sent by the client through the first connection channel, and transmitting the command packet to the first server; The command packet is parsed by the first server to determine the storage path of the image file in the shared storage server.

11. The device according to claim 10, characterized in that The command transmission module is also used to: Initialize the storage path of the shared storage server and assign the file offset to zero; the file offset represents the file size of the currently transmitted image file; The file offset is assembled into a successful execution command packet and returned to the client through the first connection channel.

12. The device according to claim 11, characterized in that Image file transmission module, specifically used for: Receiving, through the first connection channel, an image file data packet determined by the client in response to the successful execution command packet; the image file data packet is an image file data packet obtained by the client dividing the image file into a plurality of image file data packets of set file sizes; The image file data packet is stored in the shared storage server according to the storage path, the file offset of the image file already existing in the storage path is updated, the file offset of the next image file data packet to be transmitted is determined, and the data packet transmission response is assembled; Returning the data packet transmission response to the client via the first connection channel; The next image file data packet determined by the client in response to the data packet transmission response is received through the first connection channel, and is appended to the existing image file in the storage path, the file offset of the image file already existing in the storage path is updated, the file offset of the next image file data packet to be transmitted is determined, and the data packet transmission response is assembled and repeatedly returned to the client through the first connection channel to transmit the image file.

13. The device according to claim 12, characterized in that The image file transmission module is also used for: if no transmission interruption occurs between the client and the server during the transmission of the image file, when the next image file data packet determined by the client in response to the data packet transmission response is empty, the current image file has been transmitted, and the first connection channel between the client and the first server is disconnected.

14. The device according to claim 12, characterized in that The breakpoint resume module is specifically used for: Determining, by the second server according to the reconnection request of the client, a storage path in the shared storage server when the client and the first server are performing file transfer; Determine, according to the storage path of the shared storage server when the client and the first server are performing file transmission, the residual files stored in the storage path of the shared storage server when the transmission is interrupted; Obtaining the file size of the stored residual file, determining the file offset of the stored residual file, assembling the data packet breakpoint transmission response, and returning it to the client through the second connection channel; An image file resume transmission data packet determined by the client in response to a data packet breakpoint transmission response is received through a second connection channel, the data packet is appended to the stored residual file, the file offset is updated, the file offset of the next image file resume transmission data packet to be transmitted is determined, and the execution is repeated to perform breakpoint transmission. The image file resume transmission data packet is obtained by the client parsing the file offset of the stored residual file in the data packet breakpoint transmission response, determining the image file that has not been transmitted, and dividing the image file that has not been transmitted into multiple image file resume transmission data packets of set file size.

15. A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the method according to any one of claims 1 to 7 is implemented.

16. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

17. A computer program product, characterized in that The computer program product comprises a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images