File transmission method, device, equipment and storage medium

Abstract

The present disclosure provides a file transmission method, apparatus, device, and storage medium, which can be applied to nodes in the fields of data processing technology and financial technology. The method includes: obtaining attribute information and transmission link information of a target file, wherein the transmission link information is determined based on the transmission path of the target file in the business operation and maintenance architecture; the attribute information of the target file includes content attribute information and data volume attribute information; inputting the data volume attribute information into a trained transmission prediction model, and outputting information on a predicted transmission time interval for the target file to be transmitted on the transmission path; determining priority information of the target file based on the transmission link information and content attribute information; generating a transmission strategy for the target file based on the priority information when the actual transmission time interval of the target file is greater than the predicted transmission time interval; and executing a transmission operation for the target file according to the transmission strategy.

Description

Technical Field

[0001] The present disclosure relates to the fields of data processing technology and financial technology, and in particular to a file transmission method, apparatus, device, medium, and program product. Background Art

[0002] With the development of Internet financial services, transaction chains are becoming increasingly complex. During the transaction process, batch files are transmitted between various application node programs in the transaction chain. Once an abnormality occurs in any transmission link, it will directly affect the data processing of the entire transaction chain.

[0003] In related technologies, the cause of abnormalities is generally determined by manually identifying and analyzing the transmission logs of batch files. This method may cause the transmission of batch files to be interrupted, affecting the normal operation of the entire transaction chain. Summary of the Invention

[0004] In view of the above problems, the present disclosure provides a file transmission method, apparatus, device, medium and program product.

[0005] According to one aspect of the present disclosure, a file transmission method is provided, comprising:

[0006] Obtaining attribute information and transmission link information of the target file, wherein the transmission link information is determined based on the transmission path of the target file in the business operation and maintenance architecture; the attribute information of the target file includes content attribute information and data volume attribute information;

[0007] Input the data volume attribute information into the trained transmission prediction model, and output the information of the predicted transmission time interval for the target file to be transmitted on the transmission path;

[0008] Determine the priority information of the target file based on the transmission link information and content attribute information;

[0009] generating a transmission strategy for the target file based on the priority information when the actual transmission time interval of the target file is greater than the predicted transmission time interval; and

[0010] Execute the transfer operation for the target file according to the transfer policy.

[0011] According to an embodiment of the present disclosure, performing a transfer operation on a target file according to a transfer policy includes:

[0012] Determining a first node and a second node from a transmission path according to the transmission link information, wherein the target file is transmitted from the first node to the second node;

[0013] Acquire a to-be-received file in the second node, wherein the to-be-received file represents a file sent from another node to the second node;

[0014] If the priority of the target file is greater than that of the file to be received, closing the firewall restriction of the first node on the target database, where the target database represents a database for storing the target file;

[0015] A communication connection is established between the second node and the target database, and the target file is obtained from the target database using the second node.

[0016] According to an embodiment of the present disclosure, the above-mentioned file transmission method further includes:

[0017] When the priority of the target file is lower than that of other files to be received, stopping the transmission operation for the target file between the first node and the second node, and receiving other files to be received;

[0018] In case of receiving a transmission instruction from the first node to the second node for sending a target file, a transmission operation for the target file is performed between the first node and the second node.

[0019] According to an embodiment of the present disclosure, determining the priority information of a target file based on transmission link information and content attribute information includes:

[0020] Based on the transmission link information, determine the n transmission paths of the target file in the business operation and maintenance architecture;

[0021] For the i-th transmission path, determine the impact of the target file's transmission efficiency in the transmission path on business operations and maintenance based on the target file's content attribute information;

[0022] Generate priority information of the target file according to the degree of influence, where 1≤i≤n, and i and n are both positive integers.

[0023] According to an embodiment of the present disclosure, the above-mentioned file transmission method further includes:

[0024] Receiving transmission exception information sent by the monitoring platform, wherein the transmission exception information represents information about the cause of the target file transmission exception;

[0025] If the reason information indicates that abnormal data exists in the target file, the abnormal data in the target file is deleted, and the target file after the abnormal data is deleted is transmitted according to the transmission path;

[0026] When the cause information is that a node in the transmission path is abnormal, the node information in the transmission path is changed, a changed transmission link is generated, and a transmission operation for the target file is performed according to the changed transmission link.

[0027] According to an embodiment of the present disclosure, the above-mentioned file transmission method further includes:

[0028] Receive resource occupancy status information sent by the monitoring platform;

[0029] Based on the resource occupancy status information, hardware resources are allocated to the batch file transfer system for batch transfer of target files.

[0030] According to an embodiment of the present disclosure, a method for training a transmission prediction model includes:

[0031] Obtaining the transmission time interval information and data volume attribute information of historical files in the target transmission path within a preset period;

[0032] Generate a training sample set based on the transmission time interval information and data volume attribute information of the historical files;

[0033] The training sample set is used to train the initial preset model to obtain a trained transmission prediction model.

[0034] Another aspect of the present disclosure provides a file transmission device, comprising: an acquisition module, a prediction module, a determination module, a generation module and an execution module. The acquisition module is used to obtain the attribute information and transmission link information of the target file, wherein the transmission link information is determined based on the transmission path of the target file in the business operation and maintenance architecture; the attribute information of the target file includes content attribute information and data volume attribute information. The prediction module is used to input the data volume attribute information into a trained transmission prediction model, and output the information of the predicted transmission time interval for the target file to be transmitted on the transmission path. The determination module is used to determine the priority information of the target file based on the transmission link information and the content attribute information. The generation module is used to generate a transmission strategy for the target file based on the priority information when the actual transmission time interval of the target file is greater than the predicted transmission time interval. The execution module is used to perform the transmission operation for the target file according to the transmission strategy.

[0035] According to an embodiment of the present disclosure, the execution module includes a first determination unit, a first acquisition unit, a closing unit, and an establishment unit. The first determination unit is used to determine a first node and a second node from a transmission path, wherein the target file is transmitted from the first node to the second node. The first acquisition unit is used to obtain a file to be received in the second node, wherein the file to be received represents a file sent from other nodes to the second node. The closing unit is used to close the firewall restriction of the first node on the target database when the priority of the target file is greater than the priority of the file to be received, and the target database represents a database for storing the target file. The establishment unit is used to establish a communication connection between the second node and the target database, and use the second node to obtain the target file from the target database.

[0036] According to an embodiment of the present disclosure, the execution module further includes a stopping unit and an executing unit. The stopping unit is configured to stop the transfer operation for the target file between the first node and the second node and receive the other files to be received if the priority of the target file is lower than that of other files to be received. The executing unit is configured to execute the transfer operation for the target file between the first node and the second node upon receiving a transfer instruction from the first node to the second node to transmit the target file.

[0037] According to an embodiment of the present disclosure, the determination module includes a second determination unit, a third determination unit, and a first generation unit. The second determination unit is configured to determine, based on the transmission link information, n transmission paths for the target file in the business operation and maintenance architecture. The third determination unit is configured to determine, for the i-th transmission path, the degree of impact of the target file's transmission efficiency in the transmission path on business operation and maintenance based on the content attribute information of the target file. The first generation unit is configured to generate priority information for the target file based on the degree of impact, where 1≤i≤n, and i and n are both positive integers.

[0038] According to an embodiment of the present disclosure, the above-mentioned file transmission device also includes: a first receiving module, a deletion module and a change module. Among them, the first receiving module is used to receive transmission exception information sent by the monitoring platform, wherein the transmission exception information represents the cause information that causes the target file transmission exception. The deletion module is used to delete the abnormal data in the target file when the cause information is that there is abnormal data in the target file, and transmit the target file after the abnormal data is deleted according to the transmission path. The change module is used to change the node information in the transmission path when the cause information is that there is a node abnormality in the transmission path, generate a changed transmission link, and perform the transmission operation for the target file according to the changed transmission link.

[0039] According to an embodiment of the present disclosure, the file transfer device further includes a second receiving module and a dispatching module. The second receiving module is configured to receive resource occupancy status information sent by the monitoring platform. The dispatching module is configured to dispatch hardware resources to the batch file transfer system based on the resource occupancy status information for batch transfer of target files.

[0040] According to an embodiment of the present disclosure, the prediction module includes an acquisition unit, a second generation unit, and a training unit. The acquisition unit is configured to obtain transmission time interval information and data volume attribute information for historical files on a target transmission path within a preset period. The second generation unit is configured to generate a training sample set based on the transmission time interval information and data volume attribute information for the historical files. The training unit is configured to use the training sample set to train an initial preset model to obtain a trained transmission prediction model.

[0041] Another aspect of the present disclosure provides an electronic device, comprising: one or more processors; and a memory for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors execute the above method.

[0042] Another aspect of the present disclosure further provides a computer-readable storage medium having executable instructions stored thereon, which, when executed by a processor, causes the processor to perform the above method.

[0043] Another aspect of the present disclosure further provides a computer program product, including a computer program, which implements the above method when executed by a processor.

[0044] According to the file transfer method, apparatus, device, medium, and program product provided by the present disclosure, by obtaining the attribute information and transmission link information of the target file, the data volume attribute information is input into a trained transmission prediction model, and the information of the predicted transmission time interval for the target file to be transmitted on the transmission path is output. This achieves the ability to dynamically adjust the monitoring time interval for the actual transmission of the target file based on the data volume attribute information of the target file, thereby improving the accuracy of monitoring the actual transmission status of the target file. Since, when the actual transmission time interval of the target file is greater than the predicted transmission time interval, a transmission strategy for the target file is generated based on the priority information, and the transmission operation for the target file is executed according to the transmission strategy, this at least partially solves the problem in the related art of waiting for manual investigation of the cause of the abnormality when a file transmission anomaly occurs, resulting in the interruption of the transmission of batch files, and achieves the technical effect of automatically adjusting the transmission strategy in the event of a target file transmission anomaly to ensure the normal operation of the transaction link. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] The above contents and other objects, features and advantages of the present disclosure will become more apparent through the following description of the embodiments of the present disclosure with reference to the accompanying drawings, in which:

[0046] Figure 1 Schematically illustrates an application node scenario diagram of a file transmission method, apparatus, device, medium, and program product according to an embodiment of the present disclosure;

[0047] Figure 2 The following schematically shows a flow chart of a file transmission method according to an embodiment of the present disclosure;

[0048] Figure 3 Schematically shows a target file transmission link diagram according to an embodiment of the present disclosure;

[0049] Figure 4 Schematically illustrating a schematic diagram of transmitting a target file according to a transmission strategy according to some embodiments of the present disclosure;

[0050] Figure 5 A schematic diagram of transmitting a target file according to a transmission strategy according to other embodiments of the present disclosure is schematically shown.

[0051] Figure 6 A structural block diagram of a file transmission device according to an embodiment of the present disclosure is schematically shown; and

[0052] Figure 7 A block diagram of an electronic device suitable for implementing a file transmission method according to an embodiment of the present disclosure is schematically shown. DETAILED DESCRIPTION

[0053] Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are merely exemplary and are not intended to limit the scope of the present disclosure. In the detailed description below, for ease of explanation, many specific details are set forth to provide a comprehensive understanding of the embodiments of the present disclosure. However, it is apparent that one or more embodiments may also be implemented without these specific details. In addition, in the following description, descriptions of well-known structures and technologies are omitted to avoid unnecessary confusion of the concepts of the present disclosure.

[0054] The terms used herein are only for describing specific embodiments and are not intended to limit the present disclosure. The terms "comprise," "include," etc. used herein indicate the presence of the features, steps, operations, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, or components.

[0055] All terms used herein (including technical and scientific terms) have the meanings commonly understood by those skilled in the art unless otherwise defined. It should be noted that the terms used herein should be interpreted as having a meaning consistent with the context of this specification and should not be interpreted in an idealized or overly rigid manner.

[0056] When expressions such as "at least one of A, B and C, etc." are used, they should generally be interpreted in accordance with the meaning of the expression commonly understood by those skilled in the art (for example, "a system having at least one of A, B and C" should include but is not limited to a system having A alone, B alone, C alone, A and B, A and C, B and C, and / or A, B, C, etc.).

[0057] In the technical solution disclosed herein, the collection, storage, use, processing, transmission, provision, disclosure and application node processing of the data involved (including but not limited to user personal information) comply with the provisions of relevant laws and regulations, take necessary confidentiality measures, and do not violate public order and good morals.

[0058] Currently, when there are abnormalities in batch file interactions between application nodes, the normal operation of the entire transaction chain can only be restored by manually obtaining the cause of the abnormal message between systems and then manually repairing it.

[0059] Due to the fluctuation of transaction volume, the time for batch file generation is not fixed, which leads to the file transfer time between various application nodes in the business operation and maintenance architecture being also fixed. This creates certain difficulties in monitoring the actual file transmission situation and requires frequent manual adjustment of the monitoring threshold. Otherwise, abnormal alarms will be generated.

[0060] In view of this, an embodiment of the present disclosure provides a file transfer method, including: obtaining attribute information and transmission link information of a target file, wherein the transmission link information is determined based on the transmission path of the target file in a business operation and maintenance architecture; the attribute information of the target file includes content attribute information and data volume attribute information; inputting the data volume attribute information into a trained transmission prediction model, and outputting information on a predicted transmission time interval for the target file to be transmitted on the transmission path; determining priority information of the target file based on the transmission link information and the content attribute information; in a case where the actual transmission time interval of the target file is greater than the predicted transmission time interval, generating a transmission strategy for the target file based on the priority information; and performing a transmission operation for the target file according to the transmission strategy.

[0061] Figure 1 The application node scenario diagram of the file transmission method according to an embodiment of the present disclosure is schematically shown.

[0062] like Figure 1 As shown, the application node scenario 100 according to this embodiment may include a first terminal device 101, a second terminal device 102, a third terminal device 103, a network 104, and a server 105. The network 104 is used as a medium for providing a communication link between the first terminal device 101, the second terminal device 102, the third terminal device 103, and the server 105. The network 104 may include various connection types, such as wired or wireless communication links or optical fiber cables, etc.

[0063] The user may use at least one of the first terminal device 101, the second terminal device 102, and the third terminal device 103 to interact with the server 105 via the network 104 to receive or send messages, etc. Various communication client application nodes may be installed on the first terminal device 101, the second terminal device 102, and the third terminal device 103, such as shopping application nodes, web browser application nodes, search application nodes, instant messaging tools, email clients, social platform software, etc. (only as examples).

[0064] The first terminal device 101 , the second terminal device 102 , and the third terminal device 103 may be various electronic devices having display screens and supporting web browsing, including but not limited to smart phones, tablet computers, laptop computers, desktop computers, and the like.

[0065] The server 105 may be a server that provides various services, such as a background management server (for example only) that supports websites browsed by users using the first terminal device 101, the second terminal device 102, and the third terminal device 103. The background management server may analyze and process received user requests and other data, and feed back processing results (e.g., web pages, information, or data obtained or generated according to user requests) to the terminal devices.

[0066] It should be noted that the file transmission method provided in the embodiments of the present disclosure can generally be executed by the server 105. Accordingly, the file transmission apparatus provided in the embodiments of the present disclosure can generally be set in the server 105. The file transmission method provided in the embodiments of the present disclosure can also be executed by a server or server cluster that is different from the server 105 and can communicate with the first terminal device 101, the second terminal device 102, the third terminal device 103 and / or the server 105. Accordingly, the file transmission apparatus provided in the embodiments of the present disclosure can also be set in a server or server cluster that is different from the server 105 and can communicate with the first terminal device 101, the second terminal device 102, the third terminal device 103 and / or the server 105.

[0067] It should be understood that Figure 1 The number of terminal devices, networks and servers in the embodiment is merely illustrative. Any number of terminal devices, networks and servers may be provided as required.

[0068] The following will be based on Figure 1 The scene described by Figures 2 to 5 The file transmission method of the disclosed embodiment is described in detail.

[0069] Figure 2 The flowchart of the file transmission method according to the embodiment of the present disclosure is schematically shown.

[0070] like Figure 2 As shown, the file transmission method of this embodiment includes operations S210 to S250.

[0071] In operation S210, the attribute information and transmission link information of the target file are obtained, wherein the transmission link information is determined according to the transmission path of the target file in the business operation and maintenance architecture; the attribute information of the target file includes content attribute information and data volume attribute information.

[0072] In operation S220, the data volume attribute information is input into the trained transmission prediction model, and information of a predicted transmission time interval for transmitting the target file on the transmission path is output.

[0073] In operation S230, priority information of the target file is determined based on the transmission link information and the content attribute information.

[0074] In operation S240 , when the actual transmission time interval of the target file is greater than the predicted transmission time interval, a transmission strategy for the target file is generated according to the priority information.

[0075] In operation S250 , a transmission operation is performed on the target file according to the transmission policy.

[0076] Figure 3 A target file transmission link diagram according to an embodiment of the present disclosure is schematically shown.

[0077] like Figure 3 As shown, the target file can include three transmission paths in the business operation and maintenance architecture: the first transmission path can be application node A-application node B-application node C; the second transmission path can be application node A-application node D; the third transmission path can be application node A-application node E-application node F.

[0078] According to an embodiment of the present disclosure, the transmission link information may include information about any of the aforementioned transmission paths. The target file attribute information may include content attribute information and data volume attribute information. The content attribute information may be determined by the content information contained in the target file, such as the transaction content attributes of a fund product, a wealth management product, or a time deposit product. The data volume attribute information may be determined by the binary character count of the target file. For example, the data volume attribute of target file T may be 24M, 26M, 30M, and so on.

[0079] According to an embodiment of the present disclosure, the trained transmission prediction model can be a trained time recurrent neural network (LSTM). Inputting the data volume attribute information of the target file into the trained transmission prediction model can output information about the predicted transmission time interval for the target file to be transmitted on the transmission path, for example: the start transmission time t of the target file on the transmission path. b and the transmission end time t e .

[0080] According to embodiments of the present disclosure, in actual transaction scenarios, the same target file may receive different priorities in different transmission paths. For example, the priority of target file A in the transmission path of application node A, application node B, and application node C may be the highest priority. The priority of target file A in the transmission path of application node A, application node E, and application node F may be the lowest priority.

[0081] In an embodiment of the present disclosure, the priority of a target file can be annotated in the target file in the form of a label, so that during the transmission process, the transmission order of the target file at each node in the transmission path can be identified by the priority label. For example, file T1 and file T2 are both at application node A in the transmission path and need to be transmitted to the downstream node application node B. In the case where the priority of file T1 is higher than that of file T2, application node A will give priority to sending file T1 to application node B, which can avoid the problem of occasional loss of queue sending response signals caused by the unified sending of multiple files.

[0082] According to the embodiments of the present disclosure, a monitoring platform can be used to monitor the transmission time interval of the target file during the actual transmission process. For example, during the transmission of the target file T from application node A to application node B, the predicted transmission time interval is 14:15 to 14:50; however, during the actual transmission process, application node B did not receive the target file T at 14:50, indicating that a transmission abnormality occurred.

[0083] According to embodiments of the present disclosure, a transmission strategy for the target file T can be generated based on the priority tag of the target file T in application node B. For example, the transmission strategy may include continuing to wait for application node A to transmit the target file to application node B or pausing the wait and obtaining the target file from another node. The target file transmission operation is then executed according to the transmission strategy.

[0084] According to the file transfer method, apparatus, device, medium, and program product provided by the present disclosure, by obtaining the attribute information and transmission link information of the target file, the data volume attribute information is input into a trained transmission prediction model, and the information of the predicted transmission time interval for the target file to be transmitted on the transmission path is output. This achieves the ability to dynamically adjust the monitoring time interval for the actual transmission of the target file based on the data volume attribute information of the target file, thereby improving the accuracy of monitoring the actual transmission status of the target file. Since, when the actual transmission time interval of the target file is greater than the predicted transmission time interval, a transmission strategy for the target file is generated based on the priority information, and the transmission operation for the target file is executed according to the transmission strategy, this at least partially solves the problem in the related art of waiting for manual investigation of the cause of the abnormality when a file transmission anomaly occurs, resulting in the interruption of the transmission of batch files, and achieves the technical effect of automatically adjusting the transmission strategy in the event of a target file transmission anomaly to ensure the normal operation of the transaction link.

[0085] According to an embodiment of the present disclosure, performing a transfer operation on a target file according to a transfer policy includes:

[0086] Determining a first node and a second node from a transmission path according to the transmission link information, wherein the target file is transmitted from the first node to the second node;

[0087] Acquire a to-be-received file in the second node, wherein the to-be-received file represents a file sent from another node to the second node;

[0088] If the priority of the target file is greater than that of the file to be received, closing the firewall restriction of the first node on the target database, where the target database represents a database for storing the target file;

[0089] A communication connection is established between the second node and the target database, and the target file is obtained from the target database using the second node.

[0090] Figure 4 The following schematically illustrates a schematic diagram of transmitting a target file according to a transmission strategy according to some embodiments of the present disclosure.

[0091] like Figure 4 As shown, the first node determined from the transmission path may be application node A, the second node may be application node D, and the target file A is transmitted from application node A to application node D.

[0092] According to an embodiment of the present disclosure, in an actual transaction link, application node D not only needs to obtain the target file T from application node A, but also needs to obtain other files from other application nodes Q. These files sent from other application nodes to application node D can be used as files to be received by application node D, for example: they can be file T3 and file T4.

[0093] According to an embodiment of the present disclosure, the priority of the target file T is level one, the priority of file T3 is level three, and the priority of file T4 is level four. At this time, when the priority of the target file T is greater than the priority of the files T3 and T4 to be received, the application node D may no longer wait for the application node A to transmit the target file to the application node D, but directly receive the target file T from the database storing the target file A.

[0094] According to an embodiment of the present disclosure, application node D can disable the firewall restrictions between application node A and the target database. After opening the firewall restrictions, a communication connection is established between application node D and the target database. Application node D directly receives the target file T from the target database, processes the target file T in a timely manner, and loads it into the local database, triggering the file generation process. After completing the processing of the target file T, the communication connection with the target database is disconnected, thereby ensuring the normal operation of the transaction chain.

[0095] According to an embodiment of the present disclosure, since the second node downstream in the transmission path has not received the target file transmitted by the first node upstream within the predicted transmission time interval, and the priority of the target file is higher than the priority of other files to be received in the second node, it is possible to directly obtain the target file from the target database by establishing a communication connection with the target database, thereby overcoming the problem of waiting for the upstream node to send the file in the traditional file transmission process, which affects the file transmission efficiency.

[0096] According to an embodiment of the present disclosure, the above-mentioned file transmission method further includes:

[0097] When the priority of the target file is lower than that of other files to be received, stopping the transmission operation for the target file between the first node and the second node, and receiving other files to be received;

[0098] In case of receiving a transmission instruction from the first node to the second node for sending a target file, a transmission operation for the target file is performed between the first node and the second node.

[0099] Figure 5 A schematic diagram of transmitting a target file according to a transmission strategy according to some other embodiments of the present disclosure is schematically shown.

[0100] like Figure 5As shown, in this transmission path, target file A is transmitted from application node A to application node D. The priority of target file T is level 4, the priority of file T3 is level 2, and the priority of file T4 is level 1. If the priority of target file T is lower than the priorities of files T3 and T4 to be received, the transmission process of target file T from application node A to application node D can be suspended. Application node D first loads files T3 and T4 to be received. After loading is complete, it waits for application node A to send a transmission instruction for target file T to application node D before starting the file transmission process and receiving target file T.

[0101] According to an embodiment of the present disclosure, after application node D has completed loading the files T3 and T4 to be received, if it has not received a transmission instruction from application node A to send the target file T to application node D, the standby thread of application node A can be enabled to send the target file T to application node D.

[0102] According to an embodiment of the present disclosure, since when the priority of the target file is lower than that of other files to be received, the transmission operation for the target file between the first node and the second node is stopped, and other files to be received are received first. This can effectively avoid the problem that the downstream node waits for the file sent by the upstream node for a long time, resulting in the inability to transmit other files to be received normally, thereby affecting the file transmission efficiency on other transmission links.

[0103] According to an embodiment of the present disclosure, determining the priority information of a target file based on transmission link information and content attribute information includes:

[0104] Based on the transmission link information, determine the n transmission paths of the target file in the business operation and maintenance architecture;

[0105] For the i-th transmission path, determine the impact of the target file's transmission efficiency in the transmission path on business operations and maintenance based on the target file's content attribute information;

[0106] Generate priority information of the target file according to the degree of influence, where 1≤i≤n, and i and n are both positive integers.

[0107] According to the embodiments of the present disclosure, Figure 3 As shown, target file T has three transmission paths within the business operations architecture. The target file's content attributes represent fund transaction attributes, such as real-time fund transaction information. For the first transmission path, application node A to application node B to application node C, the target file's transmission efficiency significantly impacts business operations. For example, the transmission timeliness of a fund's real-time transaction interest rate directly impacts a user's willingness to purchase the fund's products. Target file T can be given the highest priority within this transmission path.

[0108] According to an embodiment of the present disclosure, the content attribute information of the target file T is information that does not affect real-time transactions, such as business personnel's professional experience information. The transmission timeliness of such files does not directly affect the user's willingness to purchase the related business or product. The target file T can be assigned a lower priority within this transmission path.

[0109] According to an embodiment of the present disclosure, the degree of impact of the transmission efficiency of the target file in the transmission path on business operations and maintenance can be configured in the system according to preset rules, and the content attributes of different target files and related transmission link information can be judged to determine the priority information of the target file on a certain transmission path.

[0110] According to an embodiment of the present disclosure, by determining the priority of the target file according to the degree of impact of the transmission efficiency of the target file in the transmission path on business operations and maintenance, the priority of the target file can be flexibly configured according to actual business needs to meet the business transaction link's requirements for file transmission timeliness.

[0111] According to an embodiment of the present disclosure, the above-mentioned file transmission method further includes:

[0112] Receiving transmission exception information sent by the monitoring platform, wherein the transmission exception information represents information about the cause of the target file transmission exception;

[0113] If the reason information indicates that abnormal data exists in the target file, the abnormal data in the target file is deleted, and the target file after the abnormal data is deleted is transmitted according to the transmission path;

[0114] When the cause information is that a node in the transmission path is abnormal, the node information in the transmission path is changed, a changed transmission link is generated, and a transmission operation for the target file is performed according to the changed transmission link.

[0115] According to an embodiment of the present disclosure, abnormal data may include data that does not meet the processing conditions of an application node. For example, when a target file T is transmitted from application node A to application node D, application node A can only process binary data. However, because target file T contains text data, application node A cannot process the target file, resulting in application node A being unable to send target file T to application node D in a timely manner. In this case, the abnormal data in the target file can be deleted, and then application node A can transmit target file T to application node D.

[0116] According to embodiments of the present disclosure, node anomalies can include those caused by issues with the transmission software or the system itself, such as a process crash. In this case, you can first attempt to restart the process of the abnormal node. If restarting the process fails, you can modify the node information in the transmission path, generate a modified transmission link, synchronize the data state of the abnormal node to the modified node, and then execute the transfer operation for the target file according to the modified transmission link.

[0117] According to an embodiment of the present disclosure, changing the node information in the transmission path may include changing the IP address, node name, port information, etc. of the node.

[0118] According to the embodiments of the present disclosure, the transmission exception information sent by the monitoring platform is used to automatically handle the abnormal situation according to the type of the transmission exception information, which at least overcomes the problem in the related art of transmission interruption caused by waiting for manual investigation of the transmission exception information.

[0119] According to an embodiment of the present disclosure, the above-mentioned file transmission method further includes:

[0120] Receive resource occupancy status information sent by the monitoring platform;

[0121] Based on the resource occupancy status information, hardware resources are allocated to the batch file transfer system for batch transfer of target files.

[0122] According to the embodiments of the present disclosure, since the number of batch file transfers in a transaction link fluctuates in actual transaction scenarios, the current resource utilization status can be monitored through a monitoring platform. When resource utilization is low, hardware resources are allocated to the batch file transfer system for batch transfer of target files.

[0123] According to the embodiments of the present disclosure, by monitoring resource occupancy status and automatically allocating hardware resources, the problem of excessive occupation of hardware resources by mechanical file transfer in related technologies can be effectively overcome.

[0124] Since different transaction volumes may affect the arrival time of files, the traditional method of setting a fixed monitoring time threshold is prone to false alarms and other alarm anomalies.

[0125] In view of this, the embodiments of the present disclosure use a transmission prediction model to predict the predicted transmission time interval of the target file, so that the monitoring time threshold of the file can be dynamically adjusted according to the predicted transmission time interval.

[0126] According to an embodiment of the present disclosure, a method for training a transmission prediction model includes:

[0127] Obtaining the transmission time interval information and data volume attribute information of historical files in the target transmission path within a preset period;

[0128] Generate a training sample set based on the transmission time interval information and data volume attribute information of the historical files;

[0129] The training sample set is used to train the initial preset model to obtain a trained transmission prediction model.

[0130] According to an embodiment of the present disclosure, the preset period can be one week, one month, etc., and the sampling period can be determined according to actual needs. The transmission time interval information and data volume attributes of the historical files in the target transmission path can be normalized and other preprocessing can be performed to generate a training sample set.

[0131] According to an embodiment of the present disclosure, the initial preset model may adopt an LSTM recurrent neural network, and the initial preset model may be trained using a training sample set to obtain a trained transmission prediction model.

[0132] According to the embodiments of the present disclosure, the transmission time interval and data volume attribute information of historical files are used to train an LSTM recurrent neural network to obtain a transmission prediction model, so that the monitoring time threshold of the file can be dynamically adjusted according to the predicted transmission time interval, effectively reducing false alarms and improving the stability of file transmission.

[0133] Based on the above file transmission method, the present disclosure also provides a file transmission device. Figure 6 The device is described in detail.

[0134] Figure 6 The structural block diagram of the file transmission device according to an embodiment of the present disclosure is schematically shown.

[0135] like Figure 6 As shown, the file transmission device 600 of this embodiment includes an acquisition module 610 , a prediction module 620 , a determination module 630 , a generation module 640 and an execution module 650 .

[0136] Acquisition module 610 is used to obtain attribute information and transmission link information of the target file. The transmission link information is determined based on the transmission path of the target file in the business operation and maintenance architecture. The attribute information of the target file includes content attribute information and data volume attribute information. In one embodiment, acquisition module 610 can be used to perform operation S210 described above, and will not be repeated here.

[0137] The prediction module 620 is used to input the data volume attribute information into the trained transmission prediction model and output information about the predicted transmission time interval of the target file on the transmission path. In one embodiment, the prediction module 620 can be used to perform the operation S220 described above, which will not be repeated here.

[0138] The determination module 630 is used to determine the priority information of the target file according to the transmission link information and the content attribute information. In one embodiment, the determination module 630 can be used to perform the operation S230 described above, which will not be repeated here.

[0139] The generation module 640 is used to generate a transmission strategy for the target file according to the priority information when the actual transmission time interval of the target file is greater than the predicted transmission time interval. In one embodiment, the generation module 640 can be used to perform the operation S240 described above, which will not be repeated here.

[0140] The execution module 650 is used to execute the transmission operation for the target file according to the transmission policy. In one embodiment, the execution module 650 can be used to execute the operation S240 described above, which will not be repeated here.

[0141] According to an embodiment of the present disclosure, the execution module includes a first determination unit, a first acquisition unit, a closing unit, and an establishment unit. The first determination unit is used to determine a first node and a second node from a transmission path, wherein the target file is transmitted from the first node to the second node. The first acquisition unit is used to obtain a file to be received in the second node, wherein the file to be received represents a file sent from other nodes to the second node. The closing unit is used to close the firewall restriction of the first node on the target database when the priority of the target file is greater than the priority of the file to be received, and the target database represents a database for storing the target file. The establishment unit is used to establish a communication connection between the second node and the target database, and use the second node to obtain the target file from the target database.

[0142] According to an embodiment of the present disclosure, the execution module further includes a stopping unit and an executing unit. The stopping unit is configured to stop the transfer operation for the target file between the first node and the second node and receive the other files to be received if the priority of the target file is lower than that of other files to be received. The executing unit is configured to execute the transfer operation for the target file between the first node and the second node upon receiving a transfer instruction from the first node to the second node to transmit the target file.

[0143] According to an embodiment of the present disclosure, the determination module includes a second determination unit, a third determination unit, and a first generation unit. The second determination unit is configured to determine, based on the transmission link information, n transmission paths for the target file in the business operation and maintenance architecture. The third determination unit is configured to determine, for the i-th transmission path, the degree of impact of the target file's transmission efficiency in the transmission path on business operation and maintenance based on the content attribute information of the target file. The first generation unit is configured to generate priority information for the target file based on the degree of impact, where 1≤i≤n, and i and n are both positive integers.

[0144] According to an embodiment of the present disclosure, the above-mentioned file transmission device also includes: a first receiving module, a deletion module and a change module. Among them, the first receiving module is used to receive transmission exception information sent by the monitoring platform, wherein the transmission exception information represents the cause information that causes the target file transmission exception. The deletion module is used to delete the abnormal data in the target file when the cause information is that there is abnormal data in the target file, and transmit the target file after the abnormal data is deleted according to the transmission path. The change module is used to change the node information in the transmission path when the cause information is that there is a node abnormality in the transmission path, generate a changed transmission link, and perform the transmission operation for the target file according to the changed transmission link.

[0145] According to an embodiment of the present disclosure, the file transfer device further includes a second receiving module and a dispatching module. The second receiving module is configured to receive resource occupancy status information sent by the monitoring platform. The dispatching module is configured to dispatch hardware resources to the batch file transfer system based on the resource occupancy status information for batch transfer of target files.

[0146] According to an embodiment of the present disclosure, the prediction module includes an acquisition unit, a second generation unit, and a training unit. The acquisition unit is configured to obtain transmission time interval information and data volume attribute information for historical files on a target transmission path within a preset period. The second generation unit is configured to generate a training sample set based on the transmission time interval information and data volume attribute information for the historical files. The training unit is configured to use the training sample set to train an initial preset model to obtain a trained transmission prediction model.

[0147] According to an embodiment of the present disclosure, any multiple modules among the acquisition module 610, the prediction module 620, the determination module 630, the generation module 640 and the execution module 650 can be combined into one module for implementation, or any one of the modules can be split into multiple modules. Alternatively, at least part of the functions of one or more of these modules can be combined with at least part of the functions of other modules and implemented in one module. According to an embodiment of the present disclosure, at least one of the acquisition module 610, the prediction module 620, the determination module 630, the generation module 640 and the execution module 650 can be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on a chip, a system on a substrate, a system on a package, an application specific integrated circuit (ASIC), or can be implemented by hardware or firmware such as any other reasonable way of integrating or packaging the circuit, or implemented in any one of the three implementation methods of software, hardware and firmware or in an appropriate combination of any of them. Alternatively, at least one of the acquisition module 610 , the prediction module 620 , the determination module 630 , the generation module 640 and the execution module 650 may be at least partially implemented as a computer program module, which may perform corresponding functions when executed.

[0148] Figure 7 A block diagram of an electronic device suitable for implementing a file transmission method according to an embodiment of the present disclosure is schematically shown.

[0149] like Figure 7 As shown, the electronic device 700 according to an embodiment of the present disclosure includes a processor 701, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 702 or a program loaded from a storage part 708 into a random access memory (RAM) 703. The processor 701 may, for example, include a general-purpose microprocessor (e.g., a CPU), an instruction set processor and / or a related chipset and / or a special-purpose microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. The processor 701 may also include an onboard memory for caching purposes. The processor 701 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of the present disclosure.

[0150] Various programs and data required for the operation of the electronic device 700 are stored in the RAM 703. The processor 701, ROM 702, and RAM 703 are connected to each other via a bus 704. The processor 701 executes the various operations of the method flow according to the embodiment of the present disclosure by executing the programs in the ROM 702 and / or RAM 703. It should be noted that the programs may also be stored in one or more memories other than the ROM 702 and RAM 703. The processor 701 may also execute the various operations of the method flow according to the embodiment of the present disclosure by executing the programs stored in the one or more memories.

[0151] According to an embodiment of the present disclosure, the electronic device 700 may further include an input / output (I / O) interface 705, which is also connected to the bus 704. The electronic device 700 may further include one or more of the following components connected to the I / O interface 705: an input portion 706 including a keyboard, a mouse, etc.; an output portion 707 including devices such as a cathode ray tube (CRT), a liquid crystal display (LCD), and a speaker; a storage portion 708 including a hard disk; and a communication portion 709 including a network interface card such as a LAN card or a modem. The communication portion 709 performs communication processing via a network such as the Internet. A drive 710 is also connected to the I / O interface 705 as needed. A removable medium 711, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed in the drive 710 as needed, so that a computer program read therefrom can be installed into the storage portion 708 as needed.

[0152] The present disclosure also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiments, or may exist independently and not be incorporated into the device / apparatus / system. The computer-readable storage medium carries one or more programs, and when executed, implements the method according to the embodiments of the present disclosure.

[0153] According to an embodiment of the present disclosure, a computer-readable storage medium may be a non-volatile computer-readable storage medium, for example, it may include but is not limited to: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, apparatus, or device. For example, according to an embodiment of the present disclosure, a computer-readable storage medium may include the ROM 702 and / or RAM 703 described above and / or one or more memories other than ROM 702 and RAM 703.

[0154] The embodiments of the present disclosure also include a computer program product, which includes a computer program containing program code for executing the method shown in the flowchart. When the computer program product is executed in a computer system, the program code is used to cause the computer system to implement the item recommendation method provided by the embodiments of the present disclosure.

[0155] The computer program executes the above functions defined in the system / device of the embodiment of the present disclosure when the processor 701 executes the computer program. According to the embodiment of the present disclosure, the system, device, module, unit, etc. described above can be implemented by a computer program module.

[0156] In one embodiment, the computer program may be stored on a tangible storage medium such as an optical storage device or a magnetic storage device. In another embodiment, the computer program may be transmitted and distributed in the form of a signal on a network medium, downloaded and installed via the communication portion 709, and / or installed from a removable medium 711. The program code contained in the computer program may be transmitted using any appropriate network medium, including but not limited to wireless, wired, or any suitable combination thereof.

[0157] In such an embodiment, the computer program can be downloaded and installed from a network via the communication section 709, and / or installed from a removable medium 711. When the computer program is executed by the processor 701, the above-described functions defined in the system of the embodiment of the present disclosure are performed. According to the embodiment of the present disclosure, the systems, devices, means, modules, units, etc. described above can be implemented by computer program modules.

[0158] According to an embodiment of the present disclosure, the program code for executing the computer program provided by the embodiment of the present disclosure can be written in any combination of one or more programming languages. Specifically, these computer programs can be implemented using high-level procedural and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages ​​include, but are not limited to, languages ​​such as Java, C++, python, "C" or similar programming languages. The program code can be executed entirely on the user computing device, partially on the user device, partially on a remote computing device, or entirely on a remote computing device or server. In cases involving a remote computing device, the remote computing device can be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computing device (for example, using an Internet service provider to connect via the Internet).

[0159] The flowcharts and block diagrams in the accompanying drawings illustrate the possible implementation architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flowchart or block diagram can represent a module, program segment, or a part of code, and the above-mentioned module, program segment, or a part of code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in an order different from that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flowchart, and the combination of boxes in the block diagram or flowchart, can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0160] Those skilled in the art will appreciate that the features described in the various embodiments and / or claims of this disclosure may be combined and / or coupled in various ways, even if such combinations and / or couplings are not explicitly described in this disclosure. In particular, the features described in the various embodiments and / or claims of this disclosure may be combined and / or coupled in various ways without departing from the spirit and teachings of this disclosure. All such combinations and / or couplings are intended to fall within the scope of this disclosure.

[0161] The embodiments of the present disclosure are described above. However, these embodiments are for illustrative purposes only and are not intended to limit the scope of the present disclosure. Although each embodiment has been described separately above, this does not mean that the measures in each embodiment cannot be used in combination to advantage. The scope of the present disclosure is defined by the appended claims and their equivalents. Without departing from the scope of the present disclosure, those skilled in the art may make various substitutions and modifications, which should all fall within the scope of the present disclosure.

Claims

1. A file transmission method, comprising: Acquire attribute information and transmission link information of a target file, wherein the transmission link information is determined based on a transmission path of the target file in a business operation and maintenance architecture; the attribute information of the target file includes content attribute information and data volume attribute information; Inputting the data volume attribute information into a trained transmission prediction model, and outputting information about a predicted transmission time interval for transmitting the target file on the transmission path; the trained transmission prediction model is a trained time series recurrent neural network; the predicted transmission time interval information includes: a start time and an end time of transmission of the target file on the transmission path; Determining priority information of the target file according to the transmission link information and the content attribute information; generating a transmission strategy for the target file according to the priority information when the actual transmission time interval of the target file is greater than the predicted transmission time interval; and Executing a transfer operation on the target file according to the transfer strategy; The performing of the transfer operation for the target file according to the transfer strategy includes: determining a first node and a second node from the transmission path according to the transmission link information, wherein the target file is transmitted from the first node to the second node; Acquire a file to be received in the second node, wherein the file to be received represents a file sent by other nodes to the second node; If the priority of the target file is greater than that of the to-be-received file, closing a firewall restriction of the first node on a target database, where the target database represents a database for storing the target file; A communication connection is established between the second node and a target database, and the target file is obtained from the target database using the second node.

2. The method according to claim 1, further comprising: If the priority of the target file is lower than that of other files to be received, stopping the transmission operation for the target file between the first node and the second node, and receiving the other files to be received; In case of receiving a transmission instruction of the target file from the first node to the second node, a transmission operation for the target file is performed between the first node and the second node.

3. The method according to claim 1, wherein The determining the priority information of the target file according to the transmission link information and the content attribute information includes: Determining n transmission paths of the target file in the business operation and maintenance architecture according to the transmission link information; For the i-th transmission path, determining, based on the content attribute information of the target file, the degree of influence of the transmission efficiency of the target file in the transmission path on the business operation and maintenance; The priority information of the target file is generated according to the impact degree, wherein 1≤i≤n, and i and n are both positive integers.

4. The method according to claim 1, further comprising: Receiving transmission exception information sent by the monitoring platform, wherein the transmission exception information represents information about a cause of the target file transmission exception; In a case where the reason information indicates that abnormal data exists in the target file, deleting the abnormal data in the target file, and transmitting the target file after the abnormal data is deleted according to the transmission path; When the cause information is that a node in the transmission path is abnormal, the node information in the transmission path is changed, a changed transmission link is generated, and a transmission operation for the target file is performed according to the changed transmission link.

5. The method according to claim 1, further comprising: Receive resource occupancy status information sent by the monitoring platform; According to the resource occupancy status information, hardware resources are allocated to the batch file transfer system for batch transmission of target files.

6. The method according to claim 1, wherein The training method of the transmission prediction model includes: Obtaining the transmission time interval information and data volume attribute information of historical files in the target transmission path within a preset period; Generating a training sample set according to the transmission time interval information and data volume attribute information of the historical file; The training sample set is used to train the initial preset model to obtain a trained transmission prediction model.

7. A file transmission device, comprising: An acquisition module, configured to acquire attribute information and transmission link information of a target file, wherein the transmission link information is determined based on a transmission path of the target file in a business operation and maintenance architecture; the attribute information of the target file includes content attribute information and data volume attribute information; a prediction module configured to input the data volume attribute information into a trained transmission prediction model and output information about a predicted transmission time interval for the target file to be transmitted along the transmission path; the trained transmission prediction model being a trained time series recurrent neural network; and the predicted transmission time interval information including: a start time and an end time of transmission of the target file along the transmission path; a determination module, configured to determine priority information of the target file based on the transmission link information and the content attribute information; a generating module, configured to generate a transmission strategy for the target file according to the priority information when the actual transmission time interval of the target file is greater than the predicted transmission time interval; and An execution module, configured to execute a transmission operation for the target file according to the transmission strategy; Wherein, the execution module includes: a first determining unit, configured to determine a first node and a second node from the transmission path according to the transmission link information, wherein the target file is transmitted from the first node to the second node; A first acquiring unit is configured to acquire a file to be received in the second node, wherein the file to be received represents a file sent by another node to the second node; a closing unit, configured to close a firewall restriction of the first node on a target database when the priority of the target file is greater than that of the file to be received, the target database representing a database for storing the target file; An establishing unit is configured to establish a communication connection between the second node and a target database, and obtain the target file from the target database using the second node.

8. An electronic device comprising: one or more processors; a storage device for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors are enabled to execute the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having executable instructions stored thereon, which, when executed by a processor, causes the processor to perform the method according to any one of claims 1 to 6.

10. A computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

Images