Data file transfer method and device, storage medium and program product

Abstract

The invention discloses a data file transmission method and device, a storage medium and a program product, and relates to the technical field of data processing.The data file transmission method comprises the steps that a data file and file configuration information of the data file are obtained, and a data conversion task of the data file is generated according to the file configuration information; performing data conversion on the data file according to the data conversion task to obtain a conversion file; and according to the file configuration information, sending a ready notification of the conversion file to a receiver, so that the receiver obtains the conversion file according to the ready notification. According to the method, data conversion is preposed, and data conversion and file transmission are processed in parallel, so that the data file transmission efficiency is improved.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to a data file transmission method, device, storage medium and program product. Background Art

[0002] With the rapid development of information technology, especially the widespread application of network technology and database technology, data file submission has gradually become electronic and automated. Financial institutions have begun to use special file transmission systems to safely and quickly transmit data files to financial regulatory agencies through the network.

[0003] At present, the commonly used file transfer scheme mainly includes the following steps: data extraction, data storage, data conversion and data transmission. However, when the file data volume is large and the types are various, this method of first extracting and storing uniformly and then converting the data not only increases the storage burden of the file transfer system and leads to the accumulation of redundant data, but also because the entire file transfer process presents the characteristics of serialization, that is, each processing link needs to be carried out in sequence, which is easy to cause network congestion in the case of large data volume, thereby prolonging the file transfer time. Summary of the invention

[0004] The main purpose of the present application is to provide a data file transmission method, device, storage medium and program product, aiming to solve the technical problem of low efficiency in data file transmission.

[0005] To achieve the above object, the present application proposes a data file transmission method, the method comprising: Acquire a data file and file configuration information of the data file, and generate a data conversion task for the data file according to the file configuration information; According to the data conversion task, the data file is converted to obtain a conversion file; According to the file configuration information, a ready notification of the converted file is sent to a receiver, so that the receiver obtains the converted file according to the ready notification.

[0006] In one embodiment, after the step of obtaining the conversion file, the method further includes: The conversion file is stored based on object storage technology and the file configuration information.

[0007] In one embodiment, after the step of generating the data conversion task of the data file, the method further includes: generating a file transfer record of the data file according to the data conversion task; The file transfer record is updated according to an operation execution result for the conversion file, wherein the operation execution result includes at least one of a data conversion execution result, a file storage execution result, and a notification sending execution result.

[0008] In one embodiment, after the step of generating the file transfer record of the data file, the method further includes: Reading the status code in the file transfer record; In the case where at least one of the status codes is a preset abnormal status code, outputting an alarm message, and determining the operation in which the abnormal status code appears for the first time as an abnormal operation; The abnormal operation and operations after the abnormal operation are re-executed.

[0009] In one embodiment, the data file transmission method further includes: The current network status is read, and the acquisition rate of the data file is adjusted according to the network status.

[0010] In one embodiment, after the step of obtaining the data file and the file configuration information of the data file, the method further includes: Verifying the data file according to the verification rules in the file configuration information; In the event of a verification failure, the data file is processed according to the exception processing rules in the file configuration information.

[0011] In one embodiment, before the step of obtaining the data file and the file configuration information of the data file, the method further includes: In the case where the type of the data file is a new type, access configuration is performed on the data file to determine file configuration information of the data file.

[0012] In addition, to achieve the above-mentioned purpose, the present application also proposes a data file transmission device, the data file transmission device comprising: A task generation module, used to obtain a data file and file configuration information of the data file, and generate a data conversion task for the data file according to the file configuration information; A data conversion module, used to perform data conversion on the data file according to the data conversion task to obtain a conversion file; The notification module is used to send a ready notification of the converted file to a receiver according to the file configuration information, so that the receiver can obtain the converted file according to the ready notification.

[0013] In addition, to achieve the above-mentioned purpose, the present application also proposes a data file transmission device, which includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the data file transmission method described above.

[0014] In addition, to achieve the above-mentioned purpose, the present application also proposes a storage medium, which is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, the steps of the data file transmission method described above are implemented.

[0015] In addition, to achieve the above-mentioned purpose, the present application also provides a computer program product, which includes a computer program, and when the computer program is executed by a processor, the steps of the data file transmission method described above are implemented.

[0016] One or more technical solutions proposed in the present application have at least the following technical effects: obtaining data files and file configuration information of data files, the file configuration information may contain key parameters such as data format, encoding and conversion rules, etc., which are very important for subsequent data conversion; then, data conversion tasks of data files can be generated according to the file configuration information, and this automated generation step reduces the possibility of human configuration errors, while improving the efficiency of data processing and data transmission; then, data conversion can be performed on the data files directly according to the data conversion tasks to obtain conversion files, reducing intermediate steps such as repeated parameter setting and manual preprocessing, speeding up data processing, and improving the efficiency of data transmission; according to the file configuration information, a ready notification of the conversion file is sent to the receiver to ensure that the receiver can timely understand the completion of the conversion file, so that the receiver can arrange the subsequent conversion file acquisition work according to this information. By placing data conversion before file storage, the present application can reduce redundant data, and at the same time decouple the data transmission and data conversion processes, without waiting for all data to be obtained before data conversion, and even in the case of a large amount of uploaded data, data conversion can also be performed normally, thereby improving data transmission efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0018] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0019] Figure 1 A flowchart of the first embodiment of the data file transmission method of the present application is provided; Figure 2 This is a general structural framework diagram of the data file transmission method provided in Example 2 of the present application; Figure 3 This is an overall interactive flow chart of the data file transmission method provided in Embodiment 2 of the present application; Figure 4 This is a schematic diagram of the module structure of the data file transmission device according to an embodiment of the present application; Figure 5 Schematic diagram of the device structure of the hardware operating environment involved in the data file transmission method in the embodiment of the present application. DETAILED DESCRIPTION

[0020] It should be understood that the specific embodiments described herein are only used to explain the technical solutions of the present application and are not used to limit the present application.

[0021] In order to better understand the technical solution of the present application, a detailed description will be given below in conjunction with the accompanying drawings and specific implementation methods.

[0022] In this embodiment, for the convenience of description, the following description is made with the terminal as the execution subject.

[0023] Conventional technologies usually require unified data extraction and storage, and then data conversion to meet the data format requirements of regulatory agencies. However, the serialized processing flow has obvious limitations. First, due to the need to store a large amount of intermediate data, redundant data increases, and the system storage pressure is high; second, the serialized processing flow is time-consuming. Once a problem occurs in a certain link, the entire file transfer process will be blocked, affecting the timeliness of data transmission. In addition, the existing data reporting system also has deficiencies in monitoring and management. It mainly focuses on monitoring the transmission results, while the status monitoring and exception handling capabilities during the transmission process are weak, resulting in the need for manual intervention and troubleshooting once an abnormality occurs, which seriously affects the efficiency of file transmission.

[0024] The present application provides a solution to obtain data files and file configuration information of data files. Before data conversion and transmission, the data files are verified according to the verification rules, which helps to find incompatible problems in data format, encoding or content in advance, so as to avoid error processing or data loss after the data reaches the receiving end; then, according to the file configuration information, a data conversion task of the data file is generated, and according to the data conversion task, the data file is converted to obtain a conversion file. This series of automated steps reduces the possibility of human error, while speeding up data processing and improving the efficiency of data transmission; finally, according to the file configuration information, a ready notification of the conversion file is sent to the receiver, ensuring that the receiver can obtain the latest data file in time, and improving the real-time performance of data exchange. Before a new type of data file is accessed, access configuration is required without the need to reconstruct the entire data processing process, which improves the flexibility and scalability of the system; during the file transmission process, the file configuration information and data conversion task records can be easily monitored. The file transmission process and audit data history are audited, which helps to track data changes and trace back the transmission process.

[0025] It should be noted that the execution subject of this embodiment may be a computing service device with data processing, network communication and program running functions, such as a tablet computer, a personal computer, a mobile phone, etc., or an electronic device capable of realizing the above functions, etc. The following takes the terminal as the execution subject as an example to illustrate this embodiment and the following embodiments.

[0026] Based on this, the embodiment of the present application provides a data file transmission method, referring to Figure 1 , Figure 1 This is a flow chart of the first embodiment of the data file transmission method of the present application.

[0027] In this embodiment, the data file transmission method includes steps S10 to S30: Step S10, obtaining a data file and file configuration information of the data file, and generating a data conversion task for the data file according to the file configuration information; It should be noted that data files refer to electronic documents containing specific data sets, which can be structured (such as database records) or unstructured (such as text, images); file configuration information refers to a series of rules and parameters that define how data files are processed and converted, including file verification rules, file transfer rules, file monitoring rules and exception handling rules.

[0028] In addition, it should be noted that a data conversion task refers to a process or task of performing a series of operations (such as format conversion, data cleaning, data mapping, etc.) on a data file based on the file configuration information. The task specifies that data of a certain format or type must be converted into another format or type, and usually involves reading the original data, applying a series of conversion rules or algorithms, and then outputting the converted data.

[0029] Exemplarily, the data file to be sent to the supervision system is downloaded from the business system, and the file configuration information corresponding to the data file is searched. The identifier of the business system to which the data file belongs (such as the business system number) is obtained from the file configuration information. Based on the identifier, the routing rules and data conversion rules of the data file of the business system are obtained to generate a data conversion task. The data conversion task contains information such as which supervision system the data file is to be sent to, as well as specific data conversion operations, so that subsequent nodes can clearly identify the content and parts to be processed.

[0030] Exemplarily, corresponding to the file configuration information containing the data conversion rules and routing rules of the data file, the conversion file can be generated directly according to the data conversion information, without the need to query according to the identifier area such as the system number.

[0031] It can be understood that data conversion tasks are automatically generated according to preset file configuration information to meet specific data processing requirements, which reduces manual intervention and improves the automation level of data file transmission.

[0032] Step S20, performing data conversion on the data file according to the data conversion task to obtain a converted file; It should be noted that the conversion file refers to a file generated after the data conversion task is processed, which contains the data converted according to specific rules. After obtaining the conversion file, the conversion file can be stored in the form of file storage or object storage, so that the subsequent recipient can obtain the conversion file according to an identifier or link.

[0033] For example, it is necessary to convert a JSON file containing customer account information exported from an internal database into an Excel format. The data conversion task determined according to the file configuration information of the JSON file may include: reading the data in the JSON file, applying rules such as date format conversion and value rounding to process the data, and storing it as an Excel file. Upon receiving a data conversion task containing conversion rules, the JSON file (data file) is read, the conversion rules are applied to process the relevant data, and the converted data is stored as an Excel file (conversion file).

[0034] Step S30: sending a conversion file readiness notification to a receiver according to the file configuration information, so that the receiver can obtain the conversion file according to the readiness notification.

[0035] It should be noted that the readiness notification refers to a message sent to the intended recipient after the conversion file is generated and stored, notifying the recipient that the conversion file is ready and can be obtained. The recipient refers to the computer system or user designated as the file recipient, which can be another program, a server or a specific user terminal.

[0036] For example, a customer's credit report needs to be converted from PDF format to XML format and sent to the regulatory agency. After the data conversion is completed, a readiness notification is sent to the designated mailbox of the regulatory agency (receiver) via email according to the sending settings in the file configuration information. After receiving the readiness notification, the regulatory agency can click the link in the email at any time to access the server and download the converted XML file (converted file).

[0037] This embodiment provides a data file transmission method, which places the data conversion step before the file storage step, so that the data conversion and file transmission are logically decoupled, thereby improving the flexibility and efficiency of data processing, realizing the independent operation of data conversion and file transmission, enhancing the maintainability and scalability of the system, simplifying the data processing process, and effectively improving the overall efficiency of data file transmission.

[0038] In a feasible implementation manner, after step S20, the method further includes: Step S21, storing the conversion file based on the object storage technology and the file configuration information.

[0039] It should be noted that object storage technology is a data storage technology that stores data in the form of objects in a flat data structure. Each object includes the data itself, metadata (such as tags, creation time, etc.) and a unique identifier.

[0040] Exemplarily, after the conversion file is generated, the object storage environment is prepared according to the file configuration information, including creating a storage bucket, configuring access permissions, etc.; the conversion file is then uploaded to the object storage server, and the metadata of the conversion file, such as the file name, description, and tags, etc., are set according to the file configuration information; the object storage server will automatically allocate a storage location for the uploaded conversion file and generate a unique access link or identifier; after confirming that the storage is successful, a broadcast notification is generated.

[0041] In this implementation, based on the metadata management, unique identifier and other functions provided by object storage technology, the stored conversion files can be managed and accessed more conveniently, thereby improving the access efficiency of the conversion files; moreover, object storage can decouple the file upload and file download processes, thereby improving the efficiency of data file transmission.

[0042] In a feasible implementation manner, after step S10, the method further includes: Step S11, generating a file transfer record of the data file according to the data conversion task; It should be noted that file transfer records refer to logs or data tables used to record relevant execution information of each process node during the data file transfer process, including but not limited to execution time, execution status, data file name and size, etc., which can be used to track and verify the transfer of data files, and facilitate finding abnormal points in the event of transmission errors.

[0043] Step S12: updating the file transfer record according to the operation execution result for the converted file, wherein the operation execution result includes at least one of: data conversion execution result, file storage execution result and notification sending execution result.

[0044] It should be noted that the execution result refers to the result of the operation on the conversion file (such as data conversion, file storage and notification sending, etc.), including success, failure or other status information. Update refers to the act of modifying or supplementing the file transfer record according to the operation execution result.

[0045] Exemplarily, after a data conversion task for a data file is generated, a file transfer record of the data file is generated while the data file is processed according to the data conversion task. After the processing (data conversion) is completed, a conversion file is generated, and the file transfer record is supplemented according to the execution result of the data conversion operation; then, the conversion file can be uploaded (stored) to the cloud server based on object storage technology, and the file transfer record is supplemented according to the upload result of the conversion file; after the upload is successful, a file ready notification is sent to the supervision system (receiver), and the notification result is supplemented in the file transfer record.

[0046] In this embodiment, by generating and updating file transfer records, the transfer process and status of data files can be clearly recorded; when a problem occurs in file transfer, the problem can be quickly located by querying the file transfer record and corresponding processing can be performed, thereby improving the efficiency of exception handling of file transfer.

[0047] In a feasible implementation manner, after step S11, the method further includes: Step S13, reading the status code in the file transfer record; It should be noted that the status code refers to a field in the file transfer record, which is used to indicate the success, failure or other specific status of each operation in the file transfer process. It can be a numeric code, or a predefined enumeration value or other form, which is used to quickly identify the operation result. The reading time of the status code can be set to be read at a scheduled time, or it can be configured according to the relevant monitoring information in the file configuration information.

[0048] It can be understood that by directly reading the status code, the status of the file transfer can be quickly identified, the efficiency of status identification can be improved, and the monitoring and auditing of the file transfer process can be facilitated.

[0049] Step S14: when at least one status code is a preset abnormal status code, output an alarm message, and determine the operation in which the abnormal status code appears for the first time as an abnormal operation; Step S15, re-execute the abnormal operation and the operation after the abnormal operation.

[0050] It should be noted that the abnormal status code refers to a predefined set of status codes that indicate abnormal status in the file transfer process, such as failure, error or other situations that require special attention; the alarm information refers to the notification output by the system when an abnormal status code is detected, which is used to remind users to pay attention to and handle abnormal situations.

[0051] In addition, it should be noted that an abnormal operation refers to an operation that causes the system or application to generate a preset abnormal status code.

[0052] For example, when a transaction data file is transmitted from a business system to a regulatory system, when it is detected that the status code of a certain upload operation is "upload failed" (one of the preset abnormal status codes), an alarm message is immediately output to the staff, and the upload operation and subsequent data conversion, object storage and other operations are re-executed to facilitate re-transmission.

[0053] Exemplarily, during the file transfer process, it is detected that the status codes of data conversion and object storage operations in the file transfer record are both failures, the data conversion operation is determined to be an abnormal operation, and the data conversion and subsequent object storage, notification sending and other operations are performed again.

[0054] In this embodiment, by timely outputting alarm information and re-executing the abnormal operation and its subsequent operations, the abnormalities or errors in the system can be discovered and repaired in time, thereby ensuring that each data file is transmitted normally, reducing the time for manual intervention, and improving file transmission efficiency.

[0055] Based on the first embodiment of the present application, in the second embodiment of the present application, the same or similar contents as those in the above-mentioned embodiment 1 can be referred to the above introduction, and will not be repeated in the following. On this basis, before step S10, it also includes: Step E10, reading the current network status, and adjusting the acquisition rate of the data file according to the network status.

[0056] It should be noted that the current network status refers to the operating status of the network system at a certain moment, including key performance indicators such as network bandwidth, latency, and packet loss rate, which can be monitored and obtained through specific protocols and algorithms.

[0057] In addition, it should be noted that the acquisition rate of data files refers to the speed of acquiring data files from the data source, such as 1 per second.

[0058] For example, before acquiring data files for data conversion, the network status is monitored, the current network status information is read, and it is determined whether the current network environment is suitable for high-speed acquisition of data files. If the network environment is good, the data file is acquired from the data source at a higher rate; if the network environment is not good, the data file is acquired from the data source at a lower rate.

[0059] In the process of acquiring data files, the network status information is continuously monitored, and the acquisition rate is dynamically adjusted according to changes in the network status.

[0060] In this embodiment, the acquisition rate can be dynamically adjusted to adapt to changes in the network environment, avoiding network congestion or network crash caused by network jitter, which in turn leads to file transfer interruption, ensuring that smooth data file transfer services can be provided in different network environments.

[0061] In a feasible implementation manner, the data file transmission method further includes: Step A10, real-time monitoring of the entire process of data file transmission; It should be noted that real-time monitoring refers to the uninterrupted observation and recording of each time point and status (the whole process) in the data file transmission process through a preset monitoring mechanism or algorithm; the whole process refers to the entire process from the triggering of the data file transmission to the successful reception by the receiving end, which may include the start of transmission, data conversion, file uploading, receiving and other processes.

[0062] Step A20, based on the file configuration information, detect whether each process node of the whole process is abnormal; It should be noted that process nodes refer to independent links or steps in the entire process of data file transmission, which are divided according to chronological order or functional characteristics; anomaly detection refers to monitoring each process node in the entire process of data file transmission to identify and report any deviation from expected behavior.

[0063] Exemplarily, according to the monitoring settings in the file configuration information, the process nodes and parameters that need to be monitored are determined, and each process node is monitored in real time; the monitoring data is compared with the preset exception processing rules to identify abnormal situations in the process; when an abnormality is detected, an abnormality report is generated and sent to relevant personnel or systems.

[0064] Exemplarily, the process of generating, uploading and sending data files is monitored in real time. When it is detected that the speed of uploading files is lower than the upload speed preset in the file configuration information, an abnormality report is automatically generated and the human cash is notified to handle any process.

[0065] It is understandable that through real-time monitoring and anomaly detection, problems in the transmission process can be discovered and responded to in a timely manner, thereby improving transmission efficiency.

[0066] Step A30: When at least one process node is abnormal, the abnormal process node is processed according to the exception processing rules in the file configuration information.

[0067] It should be noted that exception handling rules refer to a series of preset response measures or processing procedures for exceptions that may occur at process nodes. For example, for known exceptions, they are handled according to historical processing methods; for unknown exceptions, staff are notified to handle them.

[0068] For example, if an exception occurs during the process of obtaining a data file and a retry processing rule is configured, the data pulling task will be re-performed according to the configuration information. After multiple failures, the staff will be notified to perform manual processing and record the exception handling rules, gradually improving the exception handling mechanism to achieve automated processing of exception scenarios.

[0069] For example, when transferring transaction data from a local server to the cloud, a process node (such as a data encryption step) fails. After detecting this anomaly, the encryption operation is automatically retried three times according to the exception handling rules in the file configuration information. If it still fails, it switches to an alternative encryption service and notifies the staff to handle it.

[0070] In this embodiment, by automatically detecting and processing anomalies in process nodes, the system can promptly discover and solve problems, avoiding interruption of the entire business process due to anomalies in a single node, thereby saving manual inspection time and improving the efficiency of data file transmission.

[0071] In a feasible implementation manner, after the step of obtaining the data file and the file configuration information of the data file in step S10, the method further includes: Step E10, verifying the data file according to the verification rules in the file configuration information; It should be noted that verification rules refer to a set of logical judgments or conditional expressions used to verify whether the content of a data file meets expectations, including data format verification, data integrity verification, and business logic verification (such as whether the order amount is consistent with the payment amount, etc.), which can help identify and filter out data files that do not meet the requirements.

[0072] Exemplarily, when a data file transmitted by a business system is received, the verification rules in the file configuration information of the data file are read, such as verifying whether the date format, value range, etc. are standardized, and verifying whether the file is complete or has been tampered with.

[0073] Step E20: When the verification fails, the data file is processed according to the exception processing rules in the file configuration information.

[0074] For example, during the verification process, it is found that some necessary fields are missing in the data file, such as the file number, transmission object, etc. The missing fields can be marked as "pending correction" according to the exception handling rules, and the staff can be notified to perform manual inspection and correction.

[0075] In this implementation, data files that do not meet the requirements can be automatically identified and filtered out through verification rules, thereby reducing the steps of manual verification and improving the efficiency of file transmission.

[0076] In a feasible implementation manner, before step S10, the method further includes: Step S01: When the type of the data file is a new type, access configuration is performed on the data file to determine file configuration information of the data file.

[0077] It should be noted that the new type is mainly relative to the currently recognized or supported data file types, and refers to the data file types that have not yet been configured or have not yet been processed. Access configuration refers to the operation of defining and setting the necessary processing rules and parameters for new types of data files, including defining file types, setting parsing rules, allocating storage resources, etc., to ensure that new types of data files can be processed correctly.

[0078] For example, in a situation where a financial institution introduces a new transaction reporting format, before the business system uploads the relevant transaction data file, the access configuration process is started to define the necessary processing rules for this new format, including data parsing, verification and conversion rules, and these rules are determined as file configuration information for such transaction data files, and then this information is stored in the configuration database.

[0079] In this implementation, access configuration is used to ensure that subsequent processing flows can be executed normally; at the same time, automated access configuration reduces the need for manual settings and adjustments, thereby improving efficiency.

[0080] For example, to help understand the implementation process of the data file transmission method obtained by combining this embodiment with the above-mentioned embodiment 1, please refer to Figure 2 , Figure 2An overall structural framework diagram of a data file transmission method is provided, and the structure of the method may involve a file management system 100, a business system 200, a data exchange gateway 300, a file processing service cluster 400, an object storage 500, a data reporting gateway 600 and a supervision system 700.

[0081] Among them, the file transfer management system 100 mainly plays an overall configuration management and monitoring function. The business system 200, data exchange gateway 300, file processing service cluster 400, object storage 500 and data reporting gateway 600 all need to refer to the relevant file configuration information of the file transfer management system when performing related tasks.

[0082] The business system 200 transmits the data file to be uploaded to the data exchange gateway 300. The data exchange gateway 300 generates a data conversion task based on the received data file and related file configuration information, and asynchronously distributes it to the file service processing cluster 400 for data conversion.

[0083] The file service processing cluster 400 includes a plurality of different computer nodes that can process data conversion tasks in parallel. The computer node can be a physical server, a virtual machine, a container node, etc. After receiving a file transfer request, it configures the file transfer and saves the converted file to the object storage 500.

[0084] After receiving the notification that the converted file upload is complete, the data delivery gateway 600 sends a file ready notification to the supervision system 700 (receiver) according to the file configuration information.

[0085] The supervision system 700 obtains the required conversion file from the object storage 500 according to the received file readiness notification.

[0086] For details, please refer to Figure 3 , Figure 3 A general interactive diagram of a data file transmission method is provided, which can be mainly divided into four major interactive sections, specifically: The first part is configuration management. The file transfer management system 100 is responsible for unified management of file configuration information, but the business system 200 can also make corresponding adjustments to the configuration in the file transfer management system 100 through the configuration interface. When a new file type is connected, the file transfer management system 100 will perform access configuration for it. The file transfer management system 100 can extract file monitoring rules from the file configuration information to generate a monitoring rule table; extract file transfer rules to generate a file transfer rule table.

[0087] The second part is file conversion. After receiving the data file and file configuration information uploaded by the business system 200, the data exchange gateway 300 generates the corresponding data conversion task and distributes it to the processing node of the file processing service cluster 400, and adds a new file transfer record at the same time; the file processing service cluster 400 performs data conversion on the data file according to the received data conversion task and the data conversion rules in the file transfer rule table, obtains the converted file, updates the file transfer record, and uploads the converted file to the object storage 500; after the file upload is completed, the file upload result is broadcast and the file transfer record is updated.

[0088] The third part is the file notification. After receiving the notification of completion of file upload, the data reporting gateway 600 sends the file ready notification to the target supervision system according to the file transfer rules in the file configuration information; the supervision system 700 can obtain the converted data file (converted file) from the object storage 500 according to the received file ready notification.

[0089] The fourth part is monitoring and alarming, which runs through the entire file transfer process. The file transfer management system 100 can monitor the behaviors of the business system 200, the data exchange gateway 300 and the file processing service cluster 400 in real time according to the file monitoring rules in the monitoring rule table, and can also analyze the file processing results according to the file transfer records in the file transfer record table; and in the event of an abnormality, monitor and alarm the corresponding abnormality according to the exception handling rules in the file configuration information.

[0090] It should be noted that the above examples are only used to understand the present application and do not constitute a limitation on the data file transmission method of the present application. More simple transformations based on this technical concept are all within the scope of protection of the present application.

[0091] The present application also provides a data file transmission device, please refer to Figure 4 , the data file transmission device comprises: The task generation module 10 is used to obtain the data file and the file configuration information of the data file, and generate the data conversion task of the data file according to the file configuration information; The data conversion module 20 is used to perform data conversion on the data file according to the data conversion task to obtain a conversion file; The notification module 30 is used to send a conversion file readiness notification to a receiver according to the file configuration information, so that the receiver can obtain the conversion file according to the readiness notification.

[0092] The data file transmission device provided in the embodiment of the present application adopts the data file transmission method in the above embodiment, which can solve the technical problem of low efficiency of data file transmission. Compared with the prior art, the beneficial effects of the data file transmission device provided in the present application are the same as the beneficial effects of the data file transmission method provided in the above embodiment, and other technical features in the data file transmission device are the same as the features disclosed in the above embodiment method, which will not be repeated here.

[0093] An embodiment of the present application provides a data file transmission device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the data file transmission method in the above-mentioned embodiment one.

[0094] Reference below Figure 5 , which shows a schematic diagram of the structure of a data file transmission device suitable for implementing an embodiment of the present application. The data file transmission device in the embodiment of the present application may include but is not limited to mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Descriptions), PMPs (Portable Media Players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 5 The data file transmission device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0095] like Figure 5As shown, the data file transmission device may include a processing device 1001 (such as a central processing unit, a graphics processor, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM: Read Only Memory) 1002 or a program loaded from a storage device 1003 to a random access memory (RAM: Random Access Memory) 1004. In RAM1004, various programs and data required for the operation of the data file transmission device are also stored. The processing device 1001, ROM1002 and RAM1004 are connected to each other through a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Generally, the following systems can be connected to the I / O interface 1006: an input device 1007 including, for example, a touch screen, a touch pad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; an output device 1008 including, for example, a liquid crystal display (LCD: Liquid Crystal Display), a speaker, a vibrator, etc.; a storage device 1003 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 1009. The communication device 1009 can allow the data file transfer device to communicate with other devices wirelessly or by wire to exchange data. Although the data file transfer device with various systems is shown in the figure, it should be understood that it is not required to implement or have all the systems shown. More or fewer systems can be implemented or have alternatively.

[0096] In particular, according to the embodiments disclosed in the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present application include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device, or installed from a storage device 1003, or installed from a ROM 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the method of the embodiment disclosed in the present application are executed.

[0097] The data file transmission device provided in the embodiment of the present application adopts the data file transmission method in the above embodiment to solve the technical problem of data file transmission. Compared with the prior art, the beneficial effects of the data file transmission device provided in the present application are the same as the beneficial effects of the data file transmission method provided in the above embodiment, and other technical features in the data file transmission device are the same as the features disclosed in the method of the previous embodiment, which will not be repeated here.

[0098] It should be understood that the various parts disclosed in this application can be implemented by hardware, software, firmware or a combination thereof. In the description of the above embodiments, specific features, structures, materials or characteristics can be combined in any one or more embodiments or examples in a suitable manner.

[0099] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

[0100] An embodiment of the present application provides a computer-readable storage medium having computer-readable program instructions (ie, computer programs) stored thereon, wherein the computer-readable program instructions are used to execute the data file transmission method in the above-mentioned embodiment.

[0101] The computer-readable storage medium provided in the embodiment of the present application may be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems or devices, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM: Random Access Memory), a read-only memory (ROM: Read Only Memory), an erasable programmable read-only memory (EPROM: Erasable Programmable Read Only Memory or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM: CD-Read Only Memory), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system or device. The program code contained on the computer-readable storage medium may be transmitted using any appropriate medium, including but not limited to: wires, optical cables, RF (Radio Frequency: Radio Frequency), etc., or any suitable combination of the above.

[0102] The computer-readable storage medium may be included in the data file transmission device; or may exist independently without being assembled into the data file transmission device.

[0103] The computer-readable storage medium carries one or more programs. When the one or more programs are executed by a data file transmission device, the data file transmission device: obtains a data file and file configuration information of the data file, and generates a data conversion task for the data file according to the file configuration information; performs data conversion on the data file according to the data conversion task to obtain a converted file; and sends a ready notification of the converted file to a recipient according to the file configuration information, so that the recipient obtains the converted file according to the ready notification.

[0104] Computer program code for performing the operations of the present application may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0105] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present application. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0106] The modules involved in the embodiments described in this application may be implemented by software or hardware, wherein the name of the module does not constitute a limitation on the unit itself in some cases.

[0107] The readable storage medium provided in the embodiment of the present application is a computer-readable storage medium, which stores computer-readable program instructions (i.e., computer programs) for executing the above-mentioned data file transmission method, and can solve the technical problem of low efficiency of data file transmission. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided in the present application are the same as the beneficial effects of the data file transmission method provided in the above-mentioned embodiment, and will not be repeated here.

[0108] An embodiment of the present application also provides a computer program product, including a computer program, which implements the steps of the above-mentioned data file transmission method when executed by a processor.

[0109] The computer program product provided in the embodiment of the present application can solve the technical problem of low efficiency in data file transmission. Compared with the prior art, the beneficial effects of the computer program product provided in the present application are the same as the beneficial effects of the data file transmission method provided in the above embodiment, which will not be repeated here.

[0110] The above descriptions are only some embodiments of the present application, and are not intended to limit the patent scope of the present application. All equivalent structural changes made using the contents of the present application specification and drawings under the technical concept of the present application, or direct / indirect applications in other related technical fields are included in the patent protection scope of the present application.

Claims

1. A data file transmission method, characterized in that: The data file transmission method comprises: Acquire a data file and file configuration information of the data file, and generate a data conversion task for the data file according to the file configuration information; According to the data conversion task, the data file is converted to obtain a conversion file; According to the file configuration information, a ready notification of the converted file is sent to a receiver, so that the receiver obtains the converted file according to the ready notification.

2. The data file transmission method according to claim 1, characterized in that: After the step of obtaining the conversion file, the method further includes: The conversion file is stored based on object storage technology and the file configuration information.

3. The data file transmission method according to claim 2, characterized in that: After the step of generating the data conversion task of the data file, the method further includes: generating a file transfer record of the data file according to the data conversion task; The file transfer record is updated according to an operation execution result for the conversion file, wherein the operation execution result includes at least one of a data conversion execution result, a file storage execution result, and a notification sending execution result.

4. The data file transmission method according to claim 3, characterized in that: After the step of generating the file transfer record of the data file, the method further comprises: Reading the status code in the file transfer record; In the case where at least one of the status codes is a preset abnormal status code, outputting an alarm message, and determining the operation in which the abnormal status code appears for the first time as an abnormal operation; The abnormal operation and operations after the abnormal operation are re-executed.

5. The data file transmission method according to claim 1, characterized in that: Before the step of obtaining the data file and the file configuration information of the data file, the method further includes: The current network status is read, and the acquisition rate of the data file is adjusted according to the network status.

6. The data file transmission method according to claim 1, characterized in that: After the step of obtaining the data file and the file configuration information of the data file, the method further includes: Verifying the data file according to the verification rules in the file configuration information; In the event of a verification failure, the data file is processed according to the exception processing rules in the file configuration information.

7. The data file transmission method according to claim 1, characterized in that: Before the step of obtaining the data file and the file configuration information of the data file, the method further includes: In the case where the type of the data file is a new type, access configuration is performed on the data file to determine file configuration information of the data file.

8. A data file transmission device, characterized in that: The device comprises: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the data file transmission method according to any one of claims 1 to 7.

9. A storage medium, characterized in that: The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, the steps of the data file transmission method according to any one of claims 1 to 7 are implemented.

10. A computer program product, characterized in that The computer program product comprises a computer program, and when the computer program is executed by a processor, the steps of the data file transmission method according to any one of claims 1 to 7 are implemented.

Images