File transmission method and device, electronic equipment and storage medium

Abstract

The invention discloses a file transmission method and device, electronic equipment and a storage medium, and the method comprises the steps: responding to a file data transmission instruction of a data center, and starting a file scanning device to obtain a file ID of file data under a current directory; transmitting the file data to a file sending device according to the file ID of the file data; in response to a sending result of the file sending device, storing the file state and the file ID of the file data to a cache; and under the condition that the file state is updated, determining a file ID of file data needing to be retransmitted, so as to transmit incremental file data corresponding to the file ID to a server. By means of the method, incremental file data can be found in time in the file transmission process. Meanwhile, the invention further provides a file transmission scheme for the data center operation and maintenance scene, and the operation and maintenance workload is reduced while the security of file data transmission is guaranteed.

Description

Technical Field

[0001] The present application relates to the technical field of file transmission, and in particular to a file transmission method, device, electronic device, and storage medium. Background Art

[0002] As the scale of basic resources in data centers grows, the file transfer requirements in operation and maintenance scenarios have become more diverse and complex than ever before. Currently, the industry generally uses FTP, SFTP, SCP and other commands to implement file transfer, which can synchronize all files in a directory to the server at one time.

[0003] During the file transfer process, it is impossible to monitor the directory and timely discover and automatically synchronize new files or changes in the file content in the monitored directory to the server, which increases the file transfer workload in the data center operation and maintenance scenario. Summary of the invention

[0004] The embodiments of the present application provide a file transmission method, device, electronic device, and storage medium to timely discover incremental file data during the file transmission process, while improving the data transmission security in the operation and maintenance scenarios of the data center.

[0005] The present application embodiment adopts the following technical solutions:

[0006] In a first aspect, an embodiment of the present application provides a file transmission method, wherein the method includes:

[0007] In response to the file data transmission instruction of the data center, the file scanning device is started to obtain the file ID of the file data in the current directory;

[0008] transmitting the file data to a file sending device according to the file ID of the file data;

[0009] In response to the sending result of the file sending device, saving the file status and file ID of the file data to a cache;

[0010] In the case where the file status is updated, the file ID of the file data that needs to be retransmitted is determined, so that the incremental file data corresponding to the file ID is transmitted to the server.

[0011] In some embodiments, in response to the file data transmission instruction of the data center, starting the file scanning device to obtain the file ID of the file data in the current directory includes:

[0012] Acquire a specified directory according to a file data transmission instruction of the data center;

[0013] Start a file scanning device to monitor the file ID of each file in the specified directory based on the file full path, file size and the last modification time of the file to form file data;

[0014] The file ID is used to uniquely identify file information, and when any item in the file ID changes, it is regarded as a new file or a file update and transmitted to the file sending device.

[0015] In some embodiments, in response to the sending result of the file sending device, saving the file status and the file ID of the file data to a cache includes:

[0016] When the sending result of the file sending device is a transmission success or failure, updating the file status of the successful or failed transmission of the file data, and saving the updated file status and file ID to a cache;

[0017] The file scanning device is started again to obtain the file status of the successful or failed transmission of the file data.

[0018] In some embodiments, when the file status is updated, determining the file ID of the file data that needs to be retransmitted, so as to transmit the incremental file data corresponding to the file ID to the server, includes:

[0019] In the case of the file status update, if the file status transmission is successful, the current file data is filtered;

[0020] In the case where the file status is updated, if the file status transmission fails, the incremental file data corresponding to the file ID is transmitted to the server.

[0021] In some embodiments, the method further comprises

[0022] The file sending device transmits the file data to the server based on a custom transmission protocol, and the custom transmission protocol is used to be implemented in the operation and maintenance scenario of the data center.

[0023] In some embodiments, the method further comprises

[0024] In response to the asynchronous notification request submitted by the file receiving device, the asynchronous message is synchronously pushed to the third-party association system.

[0025] In some embodiments, the method comprises:

[0026] Receiving, by means of a file receiving device, file data of a data center sent by a file sending device of a client;

[0027] Returning the reception result of the file data to the file sending device of the client to update the file status of the file data in the cache;

[0028] In response to the file status of the file data, incremental file data corresponding to the file ID of the file data that needs to be retransmitted is received.

[0029] In a second aspect, an embodiment of the present application further provides a file transmission device, wherein the device comprises:

[0030] A first response module, configured to respond to a file data transmission instruction from a data center and start a file scanning device to obtain a file ID of file data in a current directory;

[0031] A transmission module, used for transmitting the file data to a file sending device according to the file ID of the file data;

[0032] A second response module, configured to save the file status and file ID of the file data to a cache in response to a sending result of the file sending device;

[0033] The determination module is used to determine the file ID of the file data that needs to be retransmitted when the file status is updated, so as to transmit the incremental file data corresponding to the file ID to the server.

[0034] In a third aspect, an embodiment of the present application further provides an electronic device, comprising: a processor; and a memory arranged to store computer executable instructions, wherein the executable instructions, when executed, cause the processor to perform the above method.

[0035] In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium, which stores one or more programs. When the one or more programs are executed by an electronic device including multiple application programs, the electronic device executes the above method.

[0036] At least one of the above technical solutions adopted in the embodiments of the present application can achieve the following beneficial effects: by responding to the file data transmission instruction of the data center, the file scanning device is started to obtain the file ID of the file data in the current directory. Then, according to the file ID of the file data, the file data is transmitted to the file sending device. Further, in response to the sending result of the file sending device, the file status and file ID of the file data are saved to the cache, so that when the file status is updated, the file ID of the file data that needs to be retransmitted can be determined, so as to transmit the incremental file data corresponding to the file ID to the server. Through the above method, new files can be discovered in time and automatically transmitted to the server, thereby reducing operation and maintenance costs and improving the transmission efficiency of file data. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0038] Figure 1 A schematic diagram of a client and a server in a file transmission method in an embodiment of the present application;

[0039] Figure 2 This is a flowchart of the file transmission method in the embodiment of the present application (client);

[0040] Figure 3 This is a flowchart of the file transmission method in the embodiment of the present application (server side);

[0041] Figure 4 This is a schematic diagram of the structure of a file transmission device in an embodiment of the present application;

[0042] Figure 5 This is a timing diagram of the implementation process of the file transmission method in the embodiment of the present application;

[0043] Figure 6 This is a schematic diagram of the structure of an electronic device in an embodiment of the present application. DETAILED DESCRIPTION

[0044] In order to make the purpose, technical solution and advantages of the present application clearer, the technical solution of the present application will be clearly and completely described below in combination with the specific embodiments of the present application and the corresponding drawings. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present application.

[0045] The technical terms involved in this application are as follows:

[0046] FTP: File Transfer Protocol is a widely used standard protocol for file transfer in the network. It allows users to connect to the FTP server through client software to upload, download, delete, rename files, etc.

[0047] SFTP: (SSH File Transfer Protocol) is a secure file transfer protocol.

[0048] SCP: (Secure Copy Protocol), also known as secure copy protocol. It is mainly used to securely transfer files between hosts on a computer network.

[0049] Monitoring transmission: It means that in the file upload task, the specified directory can be monitored in real time. When there are new files or files with changed content in the monitored directory, they can be discovered in time and automatically transmitted to the server.

[0050] In the data center operation and maintenance scenario, operation and maintenance personnel generally write shell and bat scripts and use FTP, SFTP, and SCP commands to transfer local files to the target server. However, during the file transfer process, there is a lack of the ability to monitor the local directory and automatically transfer new files in the local directory to the server. Although files in the client directory can be transferred to the server at one time or at a scheduled time through FTP, SFTP, SCP and other commands, it is not possible to detect and automatically synchronize incremental files to the server in a timely manner when new files are generated or when the file content changes. In addition, the current operation and maintenance transmission method is not compatible with different types of operating systems, and the ability to provide asynchronous notification after each file transfer is insufficient, and it is not possible to notify the third-party system in a universal way for the next step of processing.

[0051] In view of the above shortcomings, a file transfer method is provided in the embodiment of the present application. Through custom scanning technology and transmission protocol, it is possible to monitor the client directory in real time, timely discover new files or files with changed content, and automatically transfer them to the server in increments. This improves the efficiency of file transfer in operation and maintenance scenarios, reduces operation and maintenance costs, and has cross-platform compatibility and asynchronous notification capabilities.

[0052] The technical solutions provided by various embodiments of the present application are described in detail below in conjunction with the accompanying drawings.

[0053] like Figure 1 As shown, the process of file data transmission mainly involves the client and the server. The client transmits the file data to the server through the transmission protocol. The client and the server include the file transfer client and the file transfer server respectively, and both have the file transfer basic components. When the client sends a file to the server, the specific process is as follows:

[0054] S1, the client scans the local monitored directory. When the file ID (mainly composed of the three elements of the full file path, file size, and the last modified time of the file) does not exist in the cache (including but not limited to the local Ehcache cache or Redis cache), the file needs to be transferred and submitted to the client's waiting queue.

[0055] S2: The client file sending component takes the task to be sent from the waiting queue and creates a new file transfer thread.

[0056] S3, the client file transfer thread, establishes a connection with the server through a custom socket protocol.

[0057] S4, after receiving the connection request, the server authenticates the client to ensure the security of the connection, and completes the connection request with the client after passing.

[0058] S5, after establishing a connection with the server, the client encapsulates the information such as the client path of the file to be sent, the server receiving path, the file size, the file MD5 into a message and sends it to the server.

[0059] S6: The server performs permission verification on the upload directory requested by the client to ensure the legitimacy of data access and returns the verification result to the client.

[0060] S7, if the server returns a verification failure, the transmission ends; if the verification is successful, the file content transmission begins. During the file transfer process, the system will calculate and update the transfer progress in real time, and display it to the user through an intuitive progress bar or percentage, so that the user can clearly understand the current transfer status.

[0061] S8, after the server completes receiving the file, it performs integrity check based on the file message information transmitted by the client and the received file. If the check succeeds, the transmission process ends normally, otherwise it notifies the client to retransmit.

[0062] S9, after the server-side file transfer is completed, the asynchronous notification function is executed, and the third-party system can perform subsequent processing based on the notification.

[0063] S10, after the client completes sending the file, the file status in the cache is updated (ie, the corresponding relationship between the file ID and the file status).

[0064] The present application embodiment provides a file transmission method, such as Figure 2 As shown, a schematic diagram of a file transmission method in an embodiment of the present application is provided, and the method at least includes the following steps S210 to S240:

[0065] Step S210, in response to the file data transmission instruction of the data center, start the file scanning device to obtain the file ID of the file data in the current directory.

[0066] The process of file transfer refers to transferring a file from one computer system to another, usually the process of transferring files from the client to the server. In data center operation and maintenance scenarios, there are a large number of file backup and file recovery requirements. For example, off-site backup and recovery of databases. Data backup is required between databases deployed in different regions, and data recovery is required when data anomalies occur. If new file data or old file data with changed file content cannot be discovered and transferred in time during the file transfer process, the cost of operation and maintenance will increase.

[0067] The client starts the file scanning device in the client according to the file data transmission instruction of the data center to obtain the file ID of the file data in the current directory. "File data transmission instruction" includes but is not limited to the process of transmitting multiple or a certain file data from the client to the server.

[0068] The file scanning device is deployed in the client and is mainly used to periodically scan (cache) and obtain the file ID of the file data in the specified directory. The file ID is the unique identifier of the file data and can be used to distinguish different file data.

[0069] Step S220: transmitting the file data to a file sending device according to the file ID of the file data.

[0070] The client selects the file data to be transmitted according to the file ID of the file data, and transmits the file data to the file sending device in the client. The file sending device is used to receive the file data to be transmitted determined according to the file ID sent by the file scanning device.

[0071] It is understandable that the successfully transmitted files can be filtered by the file ID, thereby determining the files that are not successfully transmitted and need to be transmitted. Preferably, an operation and maintenance table of file ID and file data is established, so that the file data corresponding to the file ID can be quickly queried and located.

[0072] Step S230: In response to the sending result of the file sending device, the file status and file ID of the file data are saved in a cache.

[0073] The client saves the file status and file ID of the file data to the cache according to the sending result of the file sending device, such as success or failure. The file status in the cache is determined according to the sending result of the file sending device of the client, and whether the file ID changes is monitored by the scanning device of the client.

[0074] It can be understood that the cache includes, but is not limited to, a local Ehcache cache or a Redis cache.

[0075] Step S240, when the file status is updated, determine the file ID of the file data that needs to be retransmitted, so as to transmit the incremental file data corresponding to the file ID to the server.

[0076] When the file status is updated, the client determines the file ID of the file data that needs to be retransmitted, thereby finding the corresponding incremental file data through the file ID, and transmitting the incremental file data to the server.

[0077] Through the above method, in response to the file data transmission instruction of the data center, the file scanning device is started to obtain the file ID of the file data in the current directory, and in response to the sending result of the file sending device, the file status and file ID of the file data are saved to the cache. In this way, when the file status is updated, the file ID of the file data that needs to be retransmitted is determined, so that the incremental file data corresponding to the file ID is transmitted to the server. Based on file transmission in the data center operation and maintenance scenario, file synchronization requirements can be promptly resolved, and the operation and maintenance efficiency of the data center can be improved.

[0078] Through the above method, in response to the file data transmission instruction of the data center, the file scanning device is started to obtain the file ID of the file data in the current directory; according to the file ID of the file data, the file data is transmitted to the file sending device, so that the directory specified by the client can be monitored through the file scanning device, and when new files are under the monitored directory, or old files with changed file contents, they can be discovered in time and automatically transmitted to the server.

[0079] Through the above method, in the data center operation and maintenance scenario, new files can be discovered in time and automatically transferred to the server. That is, all files, subdirectories, and all files under subdirectories under directory a on host 1 are synchronized to the corresponding directory b on host 2 in time, and the service is notified to perform parsing after each file synchronization is completed.

[0080] Different from the FTP, SFTP, SCP and other tools used in related technologies, these file data transmission protocols can usually only achieve full file synchronization, and it is difficult to achieve timely and incremental synchronization. Through the above method, new files or old files with changed file contents can be scanned and found in time, and automatically transferred to the server in increments.

[0081] In one embodiment of the present application, in response to a file data transmission instruction from a data center, a file scanning device is started to obtain a file ID of file data in a current directory, including: obtaining a specified directory according to the file data transmission instruction from the data center; starting a file scanning device to monitor the file ID of file data composed of the full path, file size, and last modification time of each file in the specified directory; the file ID is used as a unique identifier of file information, and when any item in the file ID changes, it is regarded as a new file or an updated file and transmitted to the file sending device.

[0082] In order to better, all files, subdirectories, and all file data in the specified directory are regularly scanned by the file scanning device in the client. According to the file data transmission instruction of the data center, the specified directory is obtained; the file scanning device is started to monitor the file ID of each file in the specified directory based on the full path of the file, the file size, and the last modification time of the file.

[0083] Based on the fact that the full file path, file size, and the last modified time of the file can be used as the file ID, when the file scanning device in the client performs a file scan, each file in the directory specified by the client is combined with the full file path, file size, and the last modified time of the file into a file ID to uniquely identify a file. In other words, when any of these three types of information changes, it is considered to be a new file or the file has been updated, and the file is submitted to the sending device for automatic transmission. In addition, when the file transfer is completed, the combination of the file ID and the file transfer status is cached, so that the file scanning device in the client will filter the successfully transmitted files based on the cached data during subsequent file scans.

[0084] It should be noted that the above three types of information are only examples and are not used to limit the scope of protection in the embodiments of the present application.

[0085] In one embodiment of the present application, in response to the sending result of the file sending device, the file status and file ID of the file data are saved to the cache, including: when the sending result of the file sending device is a successful or failed transmission, the file status of the successful or failed transmission of the file data is updated, and the updated file status and file ID are saved to the cache and the file ID is saved to the cache; the file scanning device is started again to obtain the file status of the successful or failed transmission of the file data.

[0086] The file status mainly includes transmission success or failure, which is determined by the sending result of the file sending device of the client. After updating the file status of the successful or failed transmission of the file data, the updated file status and file ID are saved to the cache and the file ID is saved to the cache. Furthermore, the file scanning device of the client will be started again regularly to obtain the file status of the successful or failed transmission of the file data. If the file status changes, there may be file data that needs to be resent or the file data has been successfully sent and does not need to be sent again.

[0087] In one embodiment of the present application, in the case where the file status is updated, the file ID of the file data that needs to be retransmitted is determined to transmit the incremental file data corresponding to the file ID to the server, including: in the case where the file status is updated, if the file status transmission is successful, the current file data is filtered; in the case where the file status is updated, if the file status transmission fails, the incremental file data corresponding to the file ID is transmitted to the server.

[0088] When the client finishes sending the file, the file status in the cache is updated, that is, the correspondence between the file ID and the file status. If the result is that the file status transmission is successful, the current file data is filtered. If the result is that the file status transmission fails, the incremental file data corresponding to the file ID is transmitted to the server, and it is considered necessary to resend the incremental file data corresponding to the file ID. "Incremental file data" refers to new files, or old files with changed file content. In addition, in addition to incremental file data, data transmission failure may also be caused by the transmission protocol or network problems between the client and the server, so the file data corresponding to the file ID needs to be resent.

[0089] Through the above method, the status of the transmitted files (including successful transmission and failed transmission) is cached in the file transmission device of the client. The file scanning device of the client refers to the file status information here when scanning the file. In addition, the file sending device of the client also updates the file status here after sending the file.

[0090] In one embodiment of the present application, the method further includes the file sending device transmitting the file data to the server based on a custom transmission protocol, and the custom transmission protocol is used to be implemented in an operation and maintenance scenario of a data center.

[0091] Considering the particularity of data center operation and maintenance scenarios, especially in the financial field, data is usually transmitted between systems through the intranet. In order to ensure the security of data transmission and the compatibility between different systems, when sending and receiving files, the client and the server use a custom socket protocol to implement file transfer. The client first connects to the server, and the server performs actual file transfer after completing protocol authentication and permission verification. After the client completes the transmission, it updates the file status in the cache. If the transmission fails, it performs a timed retransmission until the transmission is successful. After the server completes the file reception, it executes asynchronous notification, and the third-party system can perform subsequent logical processing based on this notification.

[0092] It is understood that the socket protocol is only an example of a file data transmission protocol and is not used in the protection scope of the embodiments of this application. Customizing the socket protocol is related to specific services, and those skilled in the art can configure the transmission interface and protocol according to the actual usage scenario.

[0093] In one embodiment of the present application, the method further includes synchronously pushing the asynchronous message to the third-party association system in response to the asynchronous notification request submitted by the file receiving device.

[0094] Asynchronous notification refers to treating file transfer as an independent method and system. After each file transfer is successful, a timely and decoupled notification function is provided to inform the associated system for subsequent processing. Usually, asynchronous notification includes but is not limited to providing Restful API and pushing messages to Kafka message queues. It can be understood that "third-party associated system" refers to a collaborative associated system that needs to synchronously obtain the file data status, such as an off-net transaction system.

[0095] Through the above method, when each file transfer is completed, it supports sending asynchronous notifications to the third-party system so that the three-party related systems can perform subsequent logical processing. Different from the file data transmission protocol used in related technologies, which cannot meet the requirements of cross-platform compatibility and asynchronous notification, the above method can fully meet such scenarios without any modification or customization.

[0096] Different from the related technologies that have security risks if direct transmission or password-free transmission is used, the above method is based on a custom transmission protocol and developed in Java language, and does not use specific operating system features, so it has cross-platform compatibility with multiple operating systems.

[0097] like Figure 3 As shown, in an embodiment of the present application, a file transmission method is also provided, wherein the method includes:

[0098] Step S310: receiving the file data of the data center sent by the file sending device of the client through the file receiving device.

[0099] The server receives the file data of the data center sent by the client's file sending device through the file receiving device. After receiving the request sent by the client, the file receiving device on the server first performs permission verification and other operations according to the configuration, and then starts to receive the file. After receiving the file, it performs file integrity verification and executes asynchronous notification.

[0100] Step S320, returning the reception result of the file data to the file sending device of the client to update the file status of the file data in the cache.

[0101] The server returns the result of receiving the file data to the file sending device of the client to update the file status of the file data in the cache, so that the client saves the file status and file ID of the file data to the cache according to the sending result of the file sending device, such as success or failure. The file status in the cache is determined according to the sending result of the file sending device of the client, and whether the file ID changes is monitored by the scanning device of the client.

[0102] Step S330: In response to the file status of the file data, receiving incremental file data corresponding to the file ID of the file data that needs to be retransmitted.

[0103] The server can continue to receive incremental file data according to the file status of the file data. The incremental file data is the corresponding incremental file data found according to the file ID of the file data to be retransmitted.

[0104] The present application embodiment also provides a file transmission device 400, such as Figure 4 As shown, a schematic diagram of the structure of a file transmission device in an embodiment of the present application is provided, wherein the file transmission device 400 at least includes: a first response module 410, a transmission module 420, a second response module 430 and a determination module 440, wherein:

[0105] In one embodiment of the present application, the first response module 410 is specifically used to: respond to a file data transmission instruction from a data center, and start a file scanning device to obtain a file ID of file data in a current directory.

[0106] The process of file transfer refers to transferring a file from one computer system to another, usually the process of transferring files from the client to the server. In data center operation and maintenance scenarios, there are a large number of file backup and file recovery requirements. For example, off-site backup and recovery of databases. Data backup is required between databases deployed in different regions, and data recovery is required when data anomalies occur. If new file data or old file data with changed file content cannot be discovered and transferred in time during the file transfer process, the cost of operation and maintenance will increase.

[0107] The client starts the file scanning device in the client according to the file data transmission instruction of the data center to obtain the file ID of the file data in the current directory. "File data transmission instruction" includes but is not limited to the process of transmitting multiple or a certain file data from the client to the server.

[0108] The file scanning device is deployed in the client and is mainly used to periodically scan (cache) and obtain the file ID of the file data in the specified directory. The file ID is the unique identifier of the file data and can be used to distinguish different file data.

[0109] In one embodiment of the present application, the transmission module 420 is specifically used to: transmit the file data to a file sending device according to a file ID of the file data.

[0110] The client selects the file data to be transmitted according to the file ID of the file data, and transmits the file data to the file sending device in the client. The file sending device is used to receive the file data to be transmitted determined according to the file ID sent by the file scanning device.

[0111] It is understandable that the successfully transmitted files can be filtered by the file ID, thereby determining the files that are not successfully transmitted and need to be transmitted. Preferably, an operation and maintenance table of file ID and file data is established, so that the file data corresponding to the file ID can be quickly queried and located.

[0112] In one embodiment of the present application, the second response module 430 is specifically used to save the file status and file ID of the file data to a cache in response to the sending result of the file sending device.

[0113] The client saves the file status and file ID of the file data to the cache according to the sending result of the file sending device, such as success or failure. The file status in the cache is determined according to the sending result of the file sending device of the client, and whether the file ID changes is monitored by the scanning device of the client.

[0114] It can be understood that the cache includes, but is not limited to, a local Ehcache cache or a Redis cache.

[0115] In one embodiment of the present application, the determination module 440 is specifically used to: when the file status is updated, determine the file ID of the file data that needs to be retransmitted, so as to transmit the incremental file data corresponding to the file ID to the server.

[0116] When the file status is updated, the client determines the file ID of the file data that needs to be retransmitted, thereby finding the corresponding incremental file data through the file ID, and transmitting the incremental file data to the server.

[0117] In one embodiment of the present application, the second response module 430 is also used to

[0118] A designated directory is obtained according to the file data transmission instruction of the data center; a file scanning device is started to monitor the file ID of each file under the designated directory which is composed of file data based on the full path of the file, the file size and the last modification time of the file; the file ID is used as a unique identifier of the file information, and when any item in the file ID changes, it is regarded as a new file or a file update and transmitted to the file sending device.

[0119] In one embodiment of the present application, the second response module 430 is also used to

[0120] When the sending result of the file sending device is a successful or failed transmission, the file status of the successful or failed transmission of the file data is updated, and the updated file status and file ID are saved to the cache; the file scanning device is started again to obtain the file status of the successful or failed transmission of the file data.

[0121] In one embodiment of the present application, the determining module 440 is further configured to:

[0122] In the case of the file status update, if the file status transmission is successful, the current file data is filtered; in the case of the file status update, if the file status transmission fails, the incremental file data corresponding to the file ID is transmitted to the server.

[0123] In one embodiment of the present application, a transmission module is also included for

[0124] The file sending device transmits the file data to the server based on a custom transmission protocol, and the custom transmission protocol is used to be implemented in the operation and maintenance scenario of the data center.

[0125] In one embodiment of the present application, an asynchronous module is also included for

[0126] In response to the asynchronous notification request submitted by the file receiving device, the asynchronous message is synchronously pushed to the third-party association system.

[0127] It can be understood that the above-mentioned file transmission device can implement each step of the file transmission method provided in the above-mentioned embodiment, and the relevant explanations about the file transmission method are applicable to the file transmission device, which will not be repeated here.

[0128] like Figure 5 As shown, it is the business process of the file transmission method in the embodiment of the present application, including a file scanning device, a file status management, a file sending device, a file receiving device, and an asynchronous notification status.

[0129] The file scanning device deployed on the client: uniquely encodes all files in the directory, scans the monitored directory of the client, compares the status of each file in the file status management, promptly screens out new files or files with changed content, and submits the discovered incremental files to the file sending device for subsequent logical processing.

[0130] The file sending device deployed on the client side includes a queue to be sent, a file sending component, and a file status update component. The queue to be sent stores the tasks to be transmitted submitted by the file scanning device in sequence; the file sending component extracts tasks from the queue to be sent, connects to the server, sends files, and calculates the transmission progress in real time according to the server target address, target port, receiving directory, whether the transmission is limited, file integrity check mark, and whether the permission is checked in the basic configuration; after the file transmission is completed, the file status update component updates the file status to the file status management.

[0131] File receiving device deployed on the server: After receiving the request sent by the client, the server-side file receiving device first performs permission verification and other operations according to the configuration, and then starts to receive the file. After receiving the file, it performs file integrity verification and executes asynchronous notification.

[0132] File status management is performed on the client: status of the transferred files (including successful and failed transfers) is cached. The file scanning device refers to the file status information here when scanning files. In addition, the file sending device on the client also updates the file status here after sending files.

[0133] Implement asynchronous notification on the client: receive the asynchronous notification request submitted by the file receiving device, push the asynchronous notification, and let the third-party related system know in time to facilitate the next step of processing.

[0134] Figure 6 This is a schematic diagram of the structure of an electronic device according to an embodiment of the present application. Figure 6At the hardware level, the electronic device includes a processor, and optionally also includes an internal bus, a network interface, and a memory. The memory may include a memory, such as a high-speed random access memory (RAM), and may also include a non-volatile memory (non-volatile memory), such as at least one disk storage. Of course, the electronic device may also include hardware required for other services.

[0135] The processor, network interface and memory can be interconnected through an internal bus, which can be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 6 Only one bidirectional arrow is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0136] The memory is used to store the program. Specifically, the program may include a program code, and the program code includes a computer operation instruction. The memory may include a memory and a non-volatile memory, and provides instructions and data to the processor.

[0137] The processor reads the corresponding computer program from the non-volatile memory into the memory and then runs it, forming a file transfer device at the logical level. The processor executes the program stored in the memory and is specifically used to perform the following operations:

[0138] In response to the file data transmission instruction of the data center, the file scanning device is started to obtain the file ID of the file data in the current directory;

[0139] transmitting the file data to a file sending device according to the file ID of the file data;

[0140] In response to the sending result of the file sending device, saving the file status and file ID of the file data to a cache;

[0141] In the case where the file status is updated, the file ID of the file data that needs to be retransmitted is determined, so that the incremental file data corresponding to the file ID is transmitted to the server.

[0142] or,

[0143] Receiving, by means of a file receiving device, file data of a data center sent by a file sending device of a client;

[0144] Returning the reception result of the file data to the file sending device of the client to update the file status of the file data in the cache;

[0145] In response to the file status of the file data, the incremental file data corresponding to the file ID of the file data that needs to be retransmitted is received. Figure 2 or Figure 3 The method performed by the file transfer device disclosed in the illustrated embodiment can be applied to a processor or implemented by a processor. The processor may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by an integrated logic circuit of hardware in the processor or an instruction in the form of software. The above processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The methods, steps and logic block diagrams disclosed in the embodiments of the present application can be implemented or executed. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. The steps of the method disclosed in the embodiments of the present application can be directly embodied as a hardware decoding processor for execution, or a combination of hardware and software modules in the decoding processor for execution. The software module can be located in a storage medium mature in the art such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory, and the processor reads the information in the memory and completes the steps of the above method in combination with its hardware.

[0146] The electronic device may also perform Figure 2 or Figure 3 The method executed by the file transmission device in Figure 2 or Figure 3 The functions of the illustrated embodiment will not be described in detail in the embodiments of the present application.

[0147] The present application also provides a computer-readable storage medium, which stores one or more programs, wherein the one or more programs include instructions, which, when executed by an electronic device including multiple application programs, enable the electronic device to execute Figure 2 or Figure 3 The method performed by the file transmission device in the illustrated embodiment is specifically used to perform:

[0148] In response to the file data transmission instruction of the data center, the file scanning device is started to obtain the file ID of the file data in the current directory;

[0149] transmitting the file data to a file sending device according to the file ID of the file data;

[0150] In response to the sending result of the file sending device, saving the file status and file ID of the file data to a cache;

[0151] In the case where the file status is updated, the file ID of the file data that needs to be retransmitted is determined, so that the incremental file data corresponding to the file ID is transmitted to the server.

[0152] or,

[0153] Receiving, by means of a file receiving device, file data of a data center sent by a file sending device of a client;

[0154] Returning the reception result of the file data to the file sending device of the client to update the file status of the file data in the cache;

[0155] In response to the file status of the file data, incremental file data corresponding to the file ID of the file data that needs to be retransmitted is received.

[0156] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0157] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0158] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0159] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0160] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0161] The memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0162] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0163] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0164] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0165] The above is only an embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.

Claims

1. A file transmission method, wherein: The method comprises: In response to the file data transmission instruction of the data center, the file scanning device is started to obtain the file ID of the file data in the current directory; transmitting the file data to a file sending device according to the file ID of the file data; In response to the sending result of the file sending device, saving the file status and file ID of the file data to a cache; In the case where the file status is updated, the file ID of the file data that needs to be retransmitted is determined, so that the incremental file data corresponding to the file ID is transmitted to the server.

2. The method of claim 1, wherein: The step of starting a file scanning device to obtain a file ID of file data in a current directory in response to a file data transmission instruction from a data center includes: Acquire a specified directory according to a file data transmission instruction of the data center; Start a file scanning device to monitor the file ID of each file in the specified directory based on the file full path, file size and the last modification time of the file to form file data; The file ID is used to uniquely identify file information, and when any item in the file ID changes, it is regarded as a new file or a file update and transmitted to the file sending device.

3. The method of claim 2, wherein: The step of saving the file status and the file ID of the file data to a cache in response to the sending result of the file sending device comprises: When the sending result of the file sending device is a transmission success or failure, updating the file status of the successful or failed transmission of the file data, and saving the updated file status and file ID to a cache; The file scanning device is started again to obtain the file status of the successful or failed transmission of the file data.

4. The method of claim 2, wherein: In the case where the file status is updated, determining the file ID of the file data that needs to be retransmitted, so as to transmit the incremental file data corresponding to the file ID to the server, includes: In the case of the file status update, if the file status transmission is successful, the current file data is filtered; In the case where the file status is updated, if the file status transmission fails, the incremental file data corresponding to the file ID is transmitted to the server.

5. The method of claim 1, wherein: The method further comprises The file sending device transmits the file data to the server based on a custom transmission protocol, and the custom transmission protocol is used to be implemented in the operation and maintenance scenario of the data center.

6. The method of claim 5, wherein: The method further comprises In response to the asynchronous notification request submitted by the file receiving device, the asynchronous message is synchronously pushed to the third-party association system.

7. A file transmission method, wherein: The method comprises: Receiving, by means of a file receiving device, file data of a data center sent by a file sending device of a client; Returning the reception result of the file data to the file sending device of the client to update the file status of the file data in the cache; In response to the file status of the file data, incremental file data corresponding to the file ID of the file data that needs to be retransmitted is received.

8. A file transmission device, wherein: The device comprises: A first response module, configured to respond to a file data transmission instruction from a data center and start a file scanning device to obtain a file ID of file data in a current directory; A transmission module, used for transmitting the file data to a file sending device according to the file ID of the file data; A second response module, configured to save the file status and file ID of the file data to a cache in response to a sending result of the file sending device; The determination module is used to determine the file ID of the file data that needs to be retransmitted when the file status is updated, so as to transmit the incremental file data corresponding to the file ID to the server.

9. An electronic device, comprising: processor; as well as A memory arranged to store computer executable instructions, which when executed cause the processor to perform the method of any one of claims 1 to 7.

10. A computer-readable storage medium storing one or more programs, which, when executed by an electronic device including a plurality of application programs, causes the electronic device to execute any one of the methods of claims 1 to 7.

Images