A data transmission method, apparatus, and computer-readable medium
By embedding the storage addresses of unstructured data within structured data, leveraging the advantages of HTTP and FTP protocols, and combining them with a retransmission mechanism, the problems of data loss in unstructured data transmission and low efficiency in structured data transmission are solved, achieving high efficiency, reliability, and security in data transmission.
Patent Information
- Application Number
- CN202411521180.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-10-29
- Publication Date
- 2025-11-14
- Estimated Expiration
- 2044-10-29
AI Technical Summary
In existing technologies, unstructured data transmission is prone to loss and stuttering, while structured data transmission is inefficient, resulting in poor timeliness and reliability of data transmission.
Structured data is received via an HTTP interface, parsed and processed to obtain the storage address of unstructured data, and unstructured data is retrieved from a third party via an FTP interface. Combined with a retransmission mechanism and result notification method, the reliability and efficiency of data transmission are ensured.
It enables efficient and reliable transmission of structured and unstructured data, improves the timeliness and security of data transmission, and avoids data loss and processing complexity.
Smart Images

Figure CN119402481B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of communication technology, and in particular to a data transmission method, apparatus, and computer-readable medium. Background Technology
[0002] Data transmission is the communication process of transferring data from one place to another. Data types include structured data and unstructured data. Structured data refers to data managed in the form of a relational database and is usually small in volume. Unstructured data refers to data with irregular or incomplete structures, without a predefined data model, and inconvenient to represent using a two-dimensional logical table in a database. It is usually large in volume, such as videos, PDF files, or various forms of images. However, in practical applications, the data transmitted is not necessarily of a single type; often, multiple types of data are transmitted together.
[0003] In existing technologies, protocols for data transmission include HTTP and FTP. Using HTTP for data transmission has the following advantages: First, HTTP is an application-layer protocol based on TCP, thus enabling reliable transmission; second, HTTP is a request-response protocol, where the client sends a request and the server returns a response, thereby improving the timeliness of data transmission.
[0004] The FTP protocol works by having an FTP client connect to an FTP server and authenticate using a username and password. Once authenticated, the client can browse files on the server and perform interactive operations, thus improving the security of data transmission.
[0005] In implementing the technical solutions in the embodiments of this application, the inventors discovered at least the following technical problems with the above-mentioned technology: 1. For unstructured data, if the HTTP protocol is used for transmission, data transmission loss may occur. This is mainly because the large size of unstructured data causes it to occupy a large amount of online resources during transmission, resulting in data transmission lag and subsequent loss of some transmitted data. In addition, data transmission lag also leads to low data transmission efficiency. 2. For structured data, if the FTP protocol is used for transmission, low transmission efficiency and high data processing load at the data receiving end may occur. This is mainly because the FTP protocol does not record the transmitted data. When the receiving end receives structured data, it needs to analyze and process the state of the structured data, which complicates the data processing process at the data receiving end and reduces the timeliness of data delivery. Summary of the Invention
[0006] To address the aforementioned problems in the prior art, embodiments of the present invention provide a data transmission method, apparatus, and computer-readable medium; the method enables efficient and reliable transmission of structured and unstructured data during data transmission.
[0007] According to a first aspect of the present invention, a data transmission method is provided, the method comprising: a first application and a second application; the method being applied to the first application; receiving structured data sent by the second application via an HTTP interface; parsing the structured data to obtain a parsed file; obtaining the storage address of unstructured data from the parsed file; and obtaining the unstructured data from a third party corresponding to the storage address via an FTP interface.
[0008] Optionally, the method further includes: monitoring the acquisition status of the unstructured data; if the monitoring result indicates that the acquisition status is acquired, storing the acquired unstructured data in a local database; if the monitoring result indicates that the acquisition status is not acquired, re-executing the acquisition operation according to a preset time; and counting the number of acquisition operations until the number of operations reaches a preset threshold without executing the acquisition operation, generating acquisition failure information and sending the acquisition failure information to the administrator.
[0009] Optionally, the method further includes: performing a local storage operation on the parsed file; if the result of the local storage operation indicates that the parsed file is stored in a local database, then storing the structured data in a backup directory; if the result of the local storage operation indicates that the parsed file is not stored in a local database, then re-performing the local storage operation on the parsed file; and counting the number of local storage operations; until the number of operations reaches a preset threshold and no local storage operation has been performed, then sending a storage failure message to the administrator.
[0010] Optionally, the method further includes: performing a feedback operation on the second application; the feedback operation includes feedback of successfully receiving unstructured data and / or feedback of successfully receiving structured data; if the feedback operation result indicates no feedback, the feedback operation is re-executed after a preset time; and the number of times the feedback operation is performed is counted; if no feedback operation is performed when the number of times reaches a preset threshold, a feedback failure message is sent to the administrator.
[0011] Optionally, before obtaining unstructured data from the third party corresponding to the storage address via the FTP interface, the method further includes: sending user identity information to the third party corresponding to the storage address so that the third party can verify the user identity information of the first application; if the verification is successful, sending an unstructured data request to the third party via the FTP interface.
[0012] Optionally, the structured data is transmitted in XML format.
[0013] According to a second aspect of the present invention, a data transmission system is also provided, the system comprising: a first application and a second application; the second application sending structured data to the first application via an HTTP interface; the first application parsing the structured data and obtaining the storage address of unstructured data from the obtained parsed file; the first application obtaining the unstructured data from a third party corresponding to the storage address via an FTP interface; and the first application sending a feedback result to the second application indicating successful receipt of the unstructured data.
[0014] According to a third aspect of the present invention, a data transmission apparatus is also provided, the apparatus comprising: a first application and a second application; the apparatus being applied to the first application; a receiving module for receiving structured data sent by a second node via an HTTP interface; a parsing module for parsing the structured data to obtain a parsed file; a first obtaining module for obtaining the storage address of unstructured data from the parsed file; and a second obtaining module for obtaining the unstructured data from a third party corresponding to the storage address via an FTP interface.
[0015] According to a fourth aspect of the present invention, an electronic device is also provided, the electronic device comprising: a processor; a memory for storing processor-executable instructions; the processor being configured to read the executable instructions from the memory and execute the instructions to implement the method as described in the first aspect.
[0016] According to a fifth aspect of the present invention, a computer-readable medium is also provided, on which a computer program is stored, wherein the program, when executed by a processor, implements the method described in the first aspect.
[0017] This invention provides a data transmission method, apparatus, and computer-readable medium. The method includes a first application and a second application. The method is applied to the first application. First, structured data sent by the second application is received via an HTTP interface. Second, the structured data is parsed to obtain a parsed file. Then, the storage address of unstructured data is obtained from the parsed file. Finally, the unstructured data is retrieved from a third party corresponding to the storage address via an FTP interface. This embodiment leverages the real-time nature of HTTP transmission to quickly acquire structured data, and after parsing the structured data, it retrieves the unstructured data completely from a third party corresponding to the unstructured data storage address via FTP transmission, thereby improving the reliability of unstructured data transmission. This enables efficient transmission of both structured and unstructured data. Attached Figure Description
[0018] The following sections will describe some specific embodiments of the invention in detail by way of example and not limitation, with reference to the accompanying drawings. The same reference numerals in the drawings denote the same or similar parts or portions. Those skilled in the art should understand that these drawings are not necessarily drawn to scale. In the drawings:
[0019] Figure 1 This is a schematic flowchart of a data transmission method provided in an embodiment of the present invention;
[0020] Figure 2 This is an interactive schematic diagram of a data transmission method provided in an embodiment of the present invention;
[0021] Figure 3 This is a schematic diagram of a data transmission device according to an embodiment of the present invention. Detailed Implementation
[0022] To make the objectives, features, and advantages of this invention more apparent and understandable, the technical solutions of the embodiments of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this invention, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.
[0023] Structured data can be transmitted using the HTTP protocol because of its small data size, fast transmission speed, and good timeliness; unstructured data can be transmitted using the FTP protocol.
[0024] Data transmission typically involves transmitting multiple data types simultaneously. To achieve efficient data transmission, the inventors of this application propose embedding unstructured data within structured data using storage addresses. The structured data is then transmitted from a second application to a first application via HTTP. The first application then parses the structured data and retrieves the storage address corresponding to the unstructured data from the parsing results. Finally, the first application obtains the structured data from the third party corresponding to the storage address. This not only enables real-time transmission of structured data but also improves the reliability of unstructured data transmission, solving the problems of poor timeliness or reliability caused by using the same transmission protocol to simultaneously transmit structured and unstructured data in existing technologies. This, in turn, enhances the efficiency of both structured and unstructured data transmission.
[0025] like Figure 1 The diagram shown is a flowchart illustrating a data transmission method according to an embodiment of the present invention.
[0026] A data transmission method includes a first application and a second application; the method is applied to the first application; and includes at least the following steps:
[0027] S101, receive structured data sent by the second application via the HTTP interface;
[0028] S102, Parse the structured data to obtain a parsed file;
[0029] S103, Obtain the storage address of unstructured data from the parsed file;
[0030] S104 retrieves unstructured data from a third party corresponding to the storage address via the FTP interface.
[0031] Here, the first application and the second application can correspond to different servers, or they can be different applications integrated on the same server. The first application communicates with the second application and a third party.
[0032] In S101, the structured data contains the storage addresses of unstructured data nested within it. In other words, the structured data is formed by first structured data representing a specific meaning and second structured data corresponding to the storage addresses of the unstructured data.
[0033] To effectively embed unstructured data within structured data, the structured data is transmitted in XML format. This allows the storage address of the unstructured data to be defined in the XML file, facilitating subsequent parsing and processing of the structured data and improving the efficiency of the first application's data processing.
[0034] The second application sends structured data in XML format to the first application via an HTTP interface; the first application receives the structured data sent by the second application via an HTTP interface.
[0035] In S102, before the structured data is parsed, the storage address of the unstructured data is not visible to either the first or second application.
[0036] The first application parses and processes structured data based on preset rules or a trained model, and outputs a parsed file.
[0037] In S103, semantic recognition is performed on the parsed file to obtain the storage address of the unstructured data; or the parsed file is segmented based on preset fields to output the storage address of the unstructured data.
[0038] In S104, the first application sends an unstructured data request to the third party corresponding to the storage address, and the third party sends the unstructured data corresponding to the unstructured data request to the first application. The third party can be a cloud platform or a file storage server, etc.
[0039] This embodiment leverages the real-time nature of HTTP transmission to quickly acquire structured data. After parsing and processing the structured data, it uses FTP transmission to completely retrieve the unstructured data from a third party corresponding to the unstructured data storage address, thereby improving the reliability of unstructured data transmission. This enables efficient transmission of both structured and unstructured data.
[0040] To avoid uncontrollable factors during data transmission, this embodiment employs a retransmission mechanism combined with result notification to improve the reliability of unstructured data transmission. In a preferred embodiment of this first embodiment, the method further includes: monitoring the acquisition status of the unstructured data; if the monitoring result indicates that the acquisition status is acquired, storing the acquired unstructured data in a local database; if the monitoring result indicates that the acquisition status is not acquired, re-executing the acquisition operation according to a preset time; and counting the number of acquisition operations until the number of operations reaches a preset threshold without executing an acquisition operation, generating acquisition failure information and sending the acquisition failure information to the administrator.
[0041] Here, regardless of whether the first application obtains unstructured data through administrator intervention or based on the retransmission mechanism, the first application marks the acquisition status as acquired. When the first application detects the acquired status, it stores the acquired unstructured data in the local database.
[0042] To avoid data loss due to database failure, this embodiment employs a retransmission mechanism combined with result notification to ensure effective storage of the parsed file, thereby improving data transmission reliability. In a preferred embodiment of this first embodiment, the method further includes: performing a local storage operation on the parsed file; if the result of the local storage operation indicates that the parsed file is stored in the local database, then storing the structured data in a backup directory; if the result of the local storage operation indicates that the parsed file is not stored in the local database, then re-performing the local storage operation on the parsed file; and counting the number of local storage operations; until the number of operations reaches a preset threshold without performing a local storage operation, then sending a storage failure message to the administrator.
[0043] Here, regardless of whether the first application performs local storage operations on the parsed file due to administrator intervention or based on the resend mechanism; when the first application detects that the parsed file is stored in the local database, it stores the structured data in XML format in the backup directory.
[0044] To avoid uncontrollable factors during data transmission, this embodiment employs a retransmission mechanism combined with result notification to improve the reliability of unstructured data transmission. In a preferred embodiment of this first embodiment, the method further includes: performing a feedback operation to the second application; the feedback operation includes feedback on successful reception of unstructured data and / or feedback on successful reception of structured data; if the feedback operation result indicates no feedback, the feedback operation is re-executed after a preset time; and the number of feedback operations is counted; if no feedback operation is performed when the number of times reaches a preset threshold, a feedback failure message is sent to the administrator.
[0045] Here, whether it's through administrator intervention or a retransmission mechanism that the first application receives the feedback result from the second application, the process is as follows:
[0046] To improve the security of unstructured data transmission, in a preferred embodiment of this first example, before obtaining unstructured data from the third party corresponding to the storage address via the FTP interface, the method further includes: sending user identity information to the third party corresponding to the storage address so that the third party can verify the user identity information of the first application; if the verification is successful, then sending an unstructured data request to the third party via the FTP interface.
[0047] For example: the first application sends user identity information to the third party corresponding to the storage address; the third party verifies the user identity information of the first application and sends the verification result back to the first application; the first application sends an unstructured data request to the third party; the third party sends the unstructured data corresponding to the unstructured data request back to the first application.
[0048] like Figure 2 The diagram shown is an interactive schematic of a data transmission method provided in an embodiment of the present invention.
[0049] A data transmission method, comprising: a first application and a second application;
[0050] S201, The second application sends structured data to the first application via an HTTP interface;
[0051] S202, the first application parses the structured data and obtains the storage address of the unstructured data from the parsed file;
[0052] S203, The first application sends an unstructured data request to the third party corresponding to the storage address via the FTP interface;
[0053] S204, The first application receives unstructured data corresponding to the unstructured data request from a third party via the FTP interface;
[0054] S205, the first application sends a feedback result to the second application via the HTTP interface, indicating that it has successfully received the unstructured data.
[0055] The data transmission method provided in this embodiment will be described in detail below with reference to specific application scenarios.
[0056] The case information, as structured data, includes basic case information and the storage address of legal documents. The second application sends the case information in XML format to the first application. The first application parses the case information to obtain the basic case information and the storage address of the legal documents. In order to obtain the legal documents, the first application needs to obtain the corresponding PDF file of the legal documents from the third party specified at the storage address. Thus, the transmission of structured and unstructured data between the two applications is realized, enabling the first application to obtain the basic case information and legal documents corresponding to the case information.
[0057] A data transmission method, comprising a first application and a second application; comprising at least the following steps:
[0058] S1, the second application sends structured data in XML format to the first application via an HTTP interface.
[0059] S2, the first application sends a feedback result of successfully receiving structured data to the second application via the HTTP interface.
[0060] S3, the first application, parses and processes the structured data to obtain a parsed file.
[0061] S4, the first application performs a local storage operation on the parsed file; if the result of the local storage operation indicates that the parsed file is stored in the first area of the local database, then the structured data is stored in the local backup directory; if the result of the local storage operation indicates that the parsed file is not stored in the first area of the local database, then the local storage operation is re-executed on the parsed file; and the number of local storage operations is counted; if the number of local storage operations has not been executed by the time the number of operations reaches a preset threshold, then the storage failure information is sent to the administrator so that the administrator can fix the fault and re-execute the local storage operation.
[0062] S5, the first application obtains the storage address of the unstructured data from the parsed file; and sends the user identity information to the third party corresponding to the storage address so that the third party can verify the user identity information of the first application; if the verification is successful, the first application sends an unstructured data request to the third party corresponding to the storage address through the FTP interface.
[0063] S6, the first application obtains unstructured data corresponding to the unstructured data request from a third party via an FTP interface; and monitors the acquisition status; if the monitoring result indicates that the acquisition status is acquired, the acquired unstructured data is stored in the second area of the local database; if the monitoring result indicates that the acquisition status is not acquired, the acquisition operation is re-executed according to a preset time; and the number of acquisition operations is counted until the number of operations reaches a preset threshold without the acquisition operation being executed, then acquisition failure information is generated and sent to the administrator so that the administrator can fix the fault and re-execute the acquisition operation.
[0064] S7, the first application sends a feedback result to the second application via an HTTP interface, indicating that it has successfully obtained unstructured data.
[0065] This embodiment combines the advantages of real-time HTTP transmission and the reliability of FTP transmission, which can maximize the efficiency and security of data transmission. Moreover, in response to uncontrollable factors that may occur during the data transmission process and the process of storing files after parsing, a retransmission mechanism is adopted to ensure the success rate of data transmission, thereby further improving the reliability of data transmission.
[0066] like Figure 3 The diagram shown is a structural schematic of a data transmission device provided in an embodiment of the present invention.
[0067] A data transmission device 300 includes: a first application and a second application; the device is applied to the first application; a receiving module 301, configured to receive structured data sent by a second node via an HTTP interface; a parsing module 302, configured to parse the structured data to obtain a parsed file; a first acquisition module 303, configured to acquire the storage address of unstructured data from the parsed file; and a second acquisition module 304, configured to acquire the unstructured data from a third party corresponding to the storage address via an FTP interface.
[0068] In a preferred embodiment of this invention, the device further includes: a first monitoring module for monitoring the acquisition status of the unstructured data; a first storage module for storing the acquired unstructured data in a local database if the monitoring result indicates that the acquisition status is acquired; and a first retransmission module for re-executing the acquisition operation according to a preset time if the monitoring result indicates that the acquisition status is not acquired; and counting the number of acquisition operations until the number of operations reaches a preset threshold without performing an acquisition operation, generating acquisition failure information and sending the acquisition failure information to the administrator.
[0069] In a preferred embodiment of this invention, the device further includes: a storage execution module, configured to perform a local storage operation on the parsed file; a second storage module, configured to store the structured data in a backup directory if the result of the local storage operation indicates that the parsed file is stored in a local database; a second resend module, configured to re-execute the local storage operation on the parsed file if the result of the local storage operation indicates that the parsed file is not stored in the local database; and to count the number of local storage operations; and to send a storage failure message to the administrator if the number of local storage operations has not been executed when the number of local storage operations reaches a preset threshold.
[0070] In a preferred embodiment of this invention, the device further includes: a feedback execution module, configured to perform a feedback operation on the second application; the feedback operation includes feedback of successfully receiving unstructured data and / or feedback of successfully receiving structured data; a third retransmission module, configured to re-execute the feedback operation according to a preset time if the feedback operation result indicates no feedback; and count the number of times the feedback operation is performed; until the number of times reaches a preset threshold and no feedback operation is performed, a feedback failure message is sent to the administrator.
[0071] In a preferred embodiment of this invention, the device further includes: a first sending module, configured to send user identity information to a third party corresponding to the storage address, so that the third party can verify the user identity information of the first application; and a second sending module, configured to send an unstructured data request to the third party via an FTP interface if the verification is successful.
[0072] In a preferred embodiment of this example, the structured data is transmitted in XML format.
[0073] The above-described apparatus can execute a data transmission method provided in an embodiment of the present invention, and has the functional modules and beneficial effects corresponding to executing a data transmission method. Technical details not described in detail in this embodiment can be found in a data transmission method provided in an embodiment of the present invention.
[0074] The present invention also provides an electronic device, comprising: a processor; a memory for storing executable instructions of the processor; the processor being configured to read the executable instructions from the memory and execute the instructions to implement a data transmission method according to the present invention.
[0075] In addition to the methods and apparatus described above, embodiments of this application may also be computer program products, which include computer program instructions that, when executed by a processor, cause the processor to perform the steps in the methods according to various embodiments of this application described in the "Exemplary Methods" section above.
[0076] The computer program product can be written in any combination of one or more programming languages to perform the operations of the embodiments of this application. The programming languages include object-oriented programming languages such as Java and C++, as well as conventional procedural programming languages such as C or similar languages. The program code can be executed entirely on the user's computing device, partially on the user's computing device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server.
[0077] Furthermore, embodiments of this application may also be computer-readable storage media storing computer program instructions thereon, which, when executed by a processor, cause the processor to perform the steps in the methods according to the following embodiments of this application described in the "Exemplary Methods" section above.
[0078] The computer-readable storage medium may be any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may, for example, include, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatuses, or devices, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: electrical connections having one or more wires, portable disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.
[0079] The basic principles of this application have been described above with reference to specific embodiments. However, it should be noted that the advantages, benefits, and effects mentioned in this application are merely examples and not limitations, and should not be considered as essential features of each embodiment of this application. Furthermore, the specific details disclosed above are for illustrative and facilitative purposes only, and are not limitations. These details do not limit the application to the necessity of employing the aforementioned specific details for implementation.
[0080] The block diagrams of devices, apparatuses, devices, and systems involved in this application are merely illustrative examples and are not intended to require or imply that they must be connected, arranged, or configured in the manner shown in the block diagrams. As those skilled in the art will recognize, these devices, apparatuses, devices, and systems can be connected, arranged, and configured in any manner. Words such as “comprising,” “including,” “having,” etc., are open-ended terms meaning “including but not limited to,” and are used interchangeably with them. The terms “or” and “and” as used herein refer to the terms “and / or,” and are used interchangeably with them unless the context clearly indicates otherwise. The term “such as” as used herein refers to the phrase “such as but not limited to,” and is used interchangeably with it.
[0081] It should also be noted that in the apparatus, equipment, and methods of this application, the components or steps can be disassembled and / or recombined. These disassemblies and / or recombinations should be considered as equivalent solutions of this application.
[0082] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use this application. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein can be applied to other aspects without departing from the scope of this application. Therefore, this application is not intended to be limited to the aspects shown herein, but rather to be accorded the widest scope consistent with the principles and novel features disclosed herein.
[0083] The above description has been given for purposes of illustration and description. Furthermore, this description is not intended to limit the embodiments of this application to the forms disclosed herein. Although numerous exemplary aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, alterations, additions, and sub-combinations thereof.
[0084] In the description of this specification, references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the present invention. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of those different embodiments or examples.
[0085] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this invention, "a plurality of" means two or more, unless otherwise explicitly specified.
[0086] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.
Claims
1. A data transmission method, characterized in that, include: First application and second application; the method is applied to the first application; The structured data sent by the second application is received via an HTTP interface; wherein the structured data is transmitted in XML format. The structured data is parsed to obtain a parsed file; Obtain the storage address of the unstructured data from the parsed file; Send user identity information to the third party corresponding to the storage address so that the third party can verify the user identity information of the first application; If the verification passes, an unstructured data request is sent to the third party via the FTP interface; Unstructured data is obtained from a third party corresponding to the storage address via an FTP interface; Monitor the acquisition status of the unstructured data; If the monitoring result indicates that the acquisition status is "acquired", the acquired unstructured data is stored in the local database; if the monitoring result indicates that the acquisition status is "not acquired", the acquisition operation is re-executed according to a preset time; and the number of acquisition operations is counted until the number of operations reaches a preset threshold without the acquisition operation being executed, then an acquisition failure message is generated and sent to the administrator. Perform a local storage operation on the parsed file; if the result of the local storage operation indicates that the parsed file is stored in the local database, then store the structured data in the backup directory; if the result of the local storage operation indicates that the parsed file is not stored in the local database, then re-perform the local storage operation on the parsed file; and count the number of times the local storage operation is performed; if the number of times reaches a preset threshold and no local storage operation is performed, then send a storage failure message to the administrator. Perform a feedback operation on the second application; the feedback operation includes feedback of successfully receiving unstructured data and / or feedback of successfully receiving structured data; if the feedback operation result indicates no feedback, then re-execute the feedback operation according to a preset time; and count the number of times the feedback operation is performed; if no feedback operation is performed when the number of times reaches a preset threshold, then send a feedback failure message to the administrator.
2. A data transmission system, capable of implementing the data transmission method as described in claim 1, characterized in that, include: First application and second application; The second application sends structured data to the first application via an HTTP interface; The first application parses the structured data and obtains the storage address of the unstructured data from the parsed file. The first application obtains unstructured data from a third party corresponding to the storage address via an FTP interface; The first application sends a feedback result to the second application, indicating that it has successfully received the unstructured data.
3. A data transmission device, capable of implementing the data transmission method as described in claim 1, characterized in that, include: First application and second application; The device is used in the first application; The receiving module is used to receive structured data sent by the second node via the HTTP interface; The parsing module is used to parse the structured data to obtain a parsed file; The first acquisition module is used to obtain the storage address of unstructured data from the parsed file; The second acquisition module is used to acquire unstructured data from a third party corresponding to the storage address via an FTP interface.
4. An electronic device, comprising: processor; Memory used to store the processor's executable instructions; The processor is configured to read the executable instructions from the memory and execute the instructions to implement the method as claimed in claim 1.
5. A computer-readable medium having a computer program stored thereon, the program being executed by a processor to implement the method of claim 1.
Citation Information
Patent Citations
Data processing method, device and equipment and computer readable storage medium
CN113986888A
Address information analysis method and device, equipment and medium
CN114861658A