Industrial-level data transmission method, device, equipment and medium based on dual link

Abstract

The present application discloses an industry-level data transmission method, device, equipment and medium based on dual links, and relates to the technical field of data transmission. By obtaining the pre-transmitted source data from the source data provider, the source data is synchronously input into the main link and the detection link; the transmission data is copied to the data backup area; the link and time where the transmission failure occurs are determined according to the number of messages and / or message reception time of the data transmitted by the main link and the detection link in a cycle under the set time window; the target data is obtained from the source data provider or the data backup area, and resent to the target location; the transmission data is received and stored, and waits for distribution. A transmission fault detection mechanism is established by using dual links, and the same data is transmitted to the two links at the same time. By comparing the differences between the two links for the same data, the link where the transmission failure occurs is detected, and the missing data is automatically resent, so as to realize the intelligent maintenance of data transmission failures and further improve the stability and intelligence of data transmission.

Description

Technical Field

[0001] The present application relates to the technical field of data transmission, and in particular to an industry-level data transmission method, device, equipment and medium based on dual links. Background Art

[0002] Industry-level data refers to large-scale data sets collected, organized, and analyzed in a specific industry. These data usually contain a large amount of industry-related information and can be used to gain insights into industry trends, market demand, competition, etc. This application uses the civil aviation field as an example to explain the transmission of industry-level data.

[0003] At present, the data in the civil aviation field is very rich, and the value of shared applications is gradually being reflected. In order to realize the sharing of civil aviation business data among different collaborative units, it is often necessary to connect different networks through dedicated lines, encrypted virtual private networks (VPNs), etc., and the traditional single-link strategy is usually adopted between different networks for data transmission, from the upstream data source to the data processing platform, and finally to the business application system, forming a complete data sharing service chain.

[0004] The above methods have the problems of long transmission links and high maintenance requirements. Single point failures will affect the normal data acquisition of downstream systems. They have low fault tolerance and poor stability, and require a lot of manpower and material resources to handle data authentication and other issues. It is urgent to improve the stability of industry-level data transmission and reduce manpower and material costs. Summary of the invention

[0005] The main purpose of this application is to provide an industry-level data transmission method, device, equipment and medium based on dual links, aiming to solve the problems of poor stability of industry-level data transmission and high manpower maintenance costs.

[0006] To achieve the above-mentioned purpose, in a first aspect, the present application provides an industry-level data transmission method based on dual links, the dual links including a main link and a detection link, and the method including: obtaining pre-transmitted source data from a source data provider, and synchronously inputting the source data into the main link and the detection link; copying the transmission data of the main link and the detection link to a data backup area; cyclically determining the link and time where a transmission failure occurs according to the number of messages and / or message reception time of the data transmitted by the main link and the detection link within a set time window; obtaining target data from the source data provider or the data backup area according to the link and time where the transmission failure occurs, and resending it to the target location; receiving and storing the transmission data of the main link and the detection link, and waiting for distribution.

[0007] Optionally, the link and time where the transmission failure occurs are determined based on the number of messages and / or the message reception time of the data transmitted by the main link and the detection link, including: calculating the difference in the number of messages of the data transmitted by the main link and the detection link; if the difference in the number of messages is greater than a message number threshold, determining that the link with the smaller number of messages has a transmission failure; and / or calculating the difference in reception time of the same message in the data transmitted by the main link and the detection link; if the reception time difference is greater than the time difference threshold, determining that the link where the message with the later reception time is located has a transmission failure.

[0008] Optionally, the link and time where the transmission failure occurs are determined based on the number of messages and / or message reception time of the data transmitted by the main link and the detection link, and also includes: separately counting the number of messages of the data transmitted by the main link and the detection link; and determining the link whose message number does not reach the set lower limit as the link where the transmission failure occurs.

[0009] Optionally, before copying the transmission data of the main link and the detection link to the data backup area, the method further includes: filtering the data transmitted by the main link and the detection link to remove duplicate data.

[0010] Optionally, the data transmitted by the main link and the detection link are filtered to remove duplicate data, including: receiving the transmission data of the main link and the detection link respectively; traversing the data cache to see whether the transmission data is stored; if so, the transmission data is determined to be duplicate data and is removed; if not, the transmission data is determined to be new data, released, and cached in the data cache.

[0011] Optionally, pre-transmitted source data is obtained from a source data provider, and the source data is synchronously input into a main link and a detection link, including: adding a first identification header to a message in the source data and inputting the main link, adding a second identification header to a message in the source data and inputting the detection link; storing the transmission data of the main link and the detection link, and also including: removing the first identification header and the second identification header to obtain sub-messages; filtering out messages to be processed from the sub-messages; and calling a corresponding parsing program to parse the messages to be processed according to the type of the messages to be processed.

[0012] Optionally, the method further includes establishing a communication channel to a preset target distribution object and blocking communication with a non-target distribution object.

[0013] In the second aspect, the present application provides an industry-level data transmission device based on dual links, the dual links include a main link and a detection link, the device includes a receiving module, a backup module, a detection module, a reissue module and a database module, the receiving module is used to obtain pre-transmitted source data from the source data provider, and synchronously input the source data into the main link and the detection link; the backup module is used to copy the transmission data of the main link and the detection link to the data backup area; the detection module is used to determine the link and time where the transmission failure occurs according to the number of messages and / or message reception time of the data transmitted by the main link and the detection link in a set time window; the reissue module is used to obtain the target data from the source data provider or the data backup area according to the link and time where the transmission failure occurs, and reissue it to the target location; the database module is used to receive and store the transmission data of the main link and the detection link, and wait for distribution.

[0014] Optionally, the detection module can be used to calculate the difference in the number of messages in the data transmitted by the main link and the detection link; if the difference in the number of messages is greater than a message number threshold, it is determined that a transmission failure has occurred in the link with fewer messages; and / or, calculate the difference in reception time of the same message in the data transmitted by the main link and the detection link; if the reception time difference is greater than a time difference threshold, it is determined that a transmission failure has occurred in the link where the message with the later reception time is located.

[0015] Optionally, the detection module may also be used to count the number of messages of data transmitted by the main link and the detection link respectively; and determine the link whose message number does not reach the set lower limit as the link with transmission failure.

[0016] Optionally, the industry-level data transmission device further includes a filtering module, which is used to filter the data transmitted by the main link and the detection link before copying the transmission data of the main link and the detection link to the data backup area to remove duplicate data.

[0017] Optionally, the filtering module can be used to receive the transmission data of the main link and the detection link respectively; traverse the data cache area to see whether the transmission data is stored; if so, the transmission data is determined to be duplicate data and filtered out; if not, the transmission data is determined to be new data, released, and cached in the data cache area.

[0018] Optionally, the receiving module can be used to add a first identification header to the message in the source data and input it into the main link, add a second identification header to the message in the source data and input it into the detection link; store the transmission data of the main link and the detection link, and also include: removing the first identification header and the second identification header to obtain sub-messages; filtering out the messages to be processed from the sub-messages; and calling the corresponding parsing program to parse the messages to be processed according to the type of the messages to be processed.

[0019] Optionally, the industry-level data transmission device further includes a security module, which is used to establish a communication channel to a preset target distribution object and block communication with a non-target distribution object.

[0020] In a third aspect, the present application provides an electronic device comprising: a memory, a processor; the memory is used to store program instructions; the processor is used to call the program instructions to execute any industry-level data transmission method provided in the first aspect above.

[0021] In a fourth aspect, the present application provides a computer-readable storage medium, in which computer execution instructions are stored. When the computer execution instructions are executed by a processor, they are used to implement any industry-level data transmission method provided in the first aspect above.

[0022] In a fifth aspect, the present application provides a computer program product, including a computer program; when the computer program is executed, it implements any industry-level data transmission method provided in the first aspect above.

[0023] The beneficial effects that can be achieved by this application are: by applying the dual-link data transmission method, the stability and reliability of the overall data transmission link are enhanced, which not only eliminates the impact of traditional single link failures on service availability, but also avoids the problem of requiring a large amount of human resources for maintenance and guarantee, as well as the resulting data authentication issues. In the field of civil aviation, the dual-link mode provides favorable support for data sharing between airports and airlines. At the same time, when upgrading, maintaining and switching the link, seamless switching can be achieved, thereby minimizing the impact of data interruptions on downstream airlines and airport systems.

[0024] The dual-link-based industry-level data transmission method, device, equipment and medium proposed in the present application obtain pre-transmitted source data from the source data provider, and synchronously input the source data into the main link and the detection link; copy the transmission data of the main link and the detection link to the data backup area; cycle within a set time window, and determine the link and time where the transmission failure occurs according to the number of messages and / or message reception time of the data transmitted by the main link and the detection link; obtain the target data from the source data provider or the data backup area according to the link and time where the transmission failure occurs, and resend it to the target location; receive and store the transmission data of the main link and the detection link, and wait for distribution, thereby realizing the automatic detection of industry-level data transmission failures and data resending. By establishing a dual-link data transmission system, the transmission fault tolerance rate is improved. When a transmission failure occurs in one link, the data can still be transmitted in place using the other link, which improves the data transmission stability to a certain extent. A transmission fault detection mechanism is established using the dual links. The same data is transmitted to both links at the same time. By comparing the transmission differences of the same data on the two links, the link where the transmission failure occurs is detected, and the missing data is automatically resent, realizing intelligent maintenance of data transmission failures, further improving the stability and intelligence of data transmission, reducing the cost of manual intervention, and improving user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 A schematic diagram of an application scenario provided for an embodiment of the present application;

[0026] Figure 2 Schematic diagram of the process of the industry-level data transmission method based on dual links provided in the embodiment of the present application Figure 1 ;

[0027] Figure 3 Schematic diagram of the process of the industry-level data transmission method based on dual links provided in the embodiment of the present application Figure 2 ;

[0028] Figure 4 Schematic diagram of the functional modules of the dual-link industry-level data transmission device provided in the embodiment of the present application Figure 1 ;

[0029] Figure 5 Schematic diagram of the functional modules of the dual-link industry-level data transmission device provided in the embodiment of the present application Figure 2 ;

[0030] Figure 6 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application.

[0031] The realization of the purpose, functional features and advantages of this application will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0032] It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0033] The main solution of the embodiment of the present application is to establish a data transmission fault detection mechanism based on dual links, and to obtain the location of the transmission fault by comparing the transmission differences of the dual links for the same source data, so as to automatically resend the missing data.

[0034] In the single-link scenario adopted by existing technologies, network failures, equipment failures, etc. may become the source of single-point failures. Once a failure occurs, it will affect the normal data acquisition of downstream systems. Especially for some important production systems, the impact of data loss and data delay caused by link failures is quite extensive, which will have a great impact on the normal flight operation guarantee. At the same time, for a single link, more operation and maintenance resources need to be added, and the operation and maintenance cost is high. At the same time, link upgrades and replacements require the suspension of existing data transmission, which will have a data interruption impact on downstream systems. It is impossible to achieve a senseless upgrade, which is also a pain point that has not been solved for a long time.

[0035] The present application provides a solution to automate data transmission fault detection and data resending, thereby reducing the manpower and time costs of manual maintenance and improving data transmission stability.

[0036] Figure 1 This is a schematic diagram of an application scenario provided by an embodiment of the present application. Figure 1 As shown, the application scenario includes network 101 , network 102 and transmission link 103 .

[0037] Network 101 and network 102 refer to the transmission source and transmission destination of data to be transmitted, which can be a local area network (LAN) or a wide area network (WAN). The interaction between terminals is realized through the data transmission between network 101 and network 102. Figure 1 As shown, the above-mentioned terminal can be a data center, a server, a base station or a computer in a broad sense. Taking the civil aviation field as an example, the data service platform provides stable and reliable airport support plan data and advance flight plan data to the service terminal based on data such as flight information, weather, departure place, destination, etc.

[0038] The transmission link 103 carries the task of data transmission between the network 101 and the network 102, and needs to ensure the integrity and timeliness of the transmitted data. In the technical solution of the present application, the transmission link 103 is a dual link, and the two links transmit the same data simultaneously.

[0039] Figure 2Schematic diagram of the process of the industry-level data transmission method based on dual links provided in the embodiment of the present application Figure 1 .like Figure 2 As shown, the dual link includes a main link and a detection link, and the method includes:

[0040] S201: Acquire pre-transmitted source data from a source data provider, and synchronously input the source data into a primary link and a detection link.

[0041] Source data refers to the data that needs to be transmitted. The source data provider is the source of data transmission, such as a server terminal, which aggregates and packages the data to be transmitted. The source data includes multiple messages, which form a message queue during the transmission process. Each message contains information such as message content, timestamp, and message unique identifier. The same message has the same message content, timestamp, and message unique identifier.

[0042] The dual link includes a main link and a detection link. The two links are used to transmit the same data in parallel / synchronously. If there is no transmission failure in the two links during the transmission process, the same message transmitted in the two links should arrive synchronously; if there is a transmission failure in any of the two links, the same message transmitted by the two links will not arrive synchronously.

[0043] S202: Copy the transmission data of the main link and the detection link to the data backup area.

[0044] The data backup area can be set in a hardware device such as a memory or in a virtual storage location to back up the complete data to be transmitted, so as to obtain the target data when the data is reissued. The copied transmission data may be consistent with the source data, or there may be differences.

[0045] S203: cyclically determining the link and time where the transmission failure occurs according to the number of messages of data transmitted by the main link and the detection link and / or the message receiving time in the set time window.

[0046] The set time window refers to a set time period, for example, 10 minutes, 5 minutes or 40 seconds. The length of the time period can be preset according to the need for fault detection accuracy. The shorter the time period, the higher the fault detection accuracy. The detection of transmission faults is cyclical, and the time interval between the previous set time window and the next set time window can also be preset. The shorter the interval, the higher the fault detection accuracy.

[0047] If there is no transmission failure on the main link or the detection link, the data transmitted by the dual link should be consistent or have small differences. Specifically, the messages transmitted by the dual link in the same time period should be consistent or have small differences, and the link positions where the same message arrives at the same time node during the dual link transmission process should be consistent or have small differences. Therefore, based on the difference in data transmitted by the main link and the detection link, it can be compared to determine that the link with the larger difference has a transmission failure in the set time window.

[0048] Exemplarily, the link and time where the transmission failure occurs can be determined based on the number of messages of the data transmitted by the main link and the detection link. The number of messages refers to the number of messages contained in the data transmitted during the above time period. Within the set time window, if the main link or the detection link does not have a transmission failure, the number of messages transmitted by the two should be consistent or have a small difference.

[0049] Exemplarily, the link and time where the transmission failure occurs can be determined based on the message reception time of the data transmitted by the main link and the detection link. The message reception time refers to the time when the message arrives at the data receiving point. The data receiving point can be preset or it can be the time point when it arrives at the starting point or end point of the set time window. It is not limited here. If the main link or the detection link has no transmission failure, the message reception time of the same message transmitted by the two should be consistent or have a small difference. Therefore, the link and time with the transmission failure can be compared based on the performance difference of the main link or the detection link in transmitting data within the set time window.

[0050] In another embodiment, the link and time where the transmission failure occurs may be determined based on the number of messages of data transmitted by the main link and the detection link and the message receiving time, so as to improve the detection accuracy.

[0051] S204: Obtain target data from the source data provider or data backup area according to the link and time where the transmission failure occurs, and resend the target data to the target location.

[0052] The target data refers to the data missing when the link fails, and the target location refers to the link and time node where the failure occurs. The target data can be obtained from the source data provider or data backup area. The target data corresponding to the time node where the failure occurs can be identified based on the timestamp of the message, or the message content of the corresponding target data can be obtained near the time node based on another link where no transmission failure occurs. The source data provider or data backup area obtains the target data.

[0053] In some embodiments, after determining the link and time of the transmission failure, the detection result can also be written into the database in real time, and an alarm prompt can be issued through the operation and maintenance interface. When manual intervention is required, targeted maintenance can be performed based on the detection result.

[0054] S205: receiving and storing transmission data of the main link and the detection link, and waiting for distribution.

[0055] After the data retransmission is completed, the data transmitted by the dual link reaches the transmission destination. At this time, the transmitted data is persistently stored in the database, and the data is routed and distributed in a targeted manner according to the data requirements of different distribution objects (for example, various business units).

[0056] Taking the civil aviation field as an example, the specific information is to obtain the flight departure station, landing station, and airline in the airport support plan message, and forward the flight message to the queue of the corresponding departure and landing airport according to the pre-configured airline-airport message routing rules. For messages that do not comply with the airline-airport message routing rules, they are logged and skipped. For messages that need to be stored, they are sent to the entry queue of the message storage program.

[0057] The embodiment of the present application obtains the pre-transmitted source data from the source data provider, and synchronously inputs the source data into the main link and the detection link; copies the transmission data of the main link and the detection link to the data backup area; circulates in the set time window, and determines the link and time where the transmission failure occurs according to the number of messages and / or message reception time of the data transmitted by the main link and the detection link; obtains the target data from the source data provider or the data backup area according to the link and time where the transmission failure occurs, and reissues it to the target location; receives and stores the transmission data of the main link and the detection link, and waits for distribution. The embodiment of the present application improves the transmission fault tolerance rate by establishing a dual-link data transmission system. When a transmission failure occurs in one link, the data can still be transmitted in place using another link, thereby improving the data transmission stability to a certain extent; establishes a transmission fault detection mechanism using dual links, transmits the same data to the two links at the same time, detects the link where the transmission failure occurs by comparing the transmission difference of the two links for the same data, and automatically reissues the missing data, thereby realizing intelligent maintenance of data transmission failure, further improving the stability and intelligence of data transmission, reducing the cost of manual intervention, and improving the user experience.

[0058] On the basis of the above embodiment, optionally, the link and time where the transmission failure occurs can be determined based on the number of messages and / or the message reception time of the data transmitted by the main link and the detection link. Exemplarily, the link and time where the transmission failure occurs can be determined based on the difference in the number of messages, the difference in reception time, or the number of messages within a set time window.

[0059] Figure 3 Schematic diagram of the process of the industry-level data transmission method based on dual links provided in the embodiment of the present application Figure 2 .like Figure 3As shown, in some embodiments, determining the link and time where the transmission failure occurs according to the number of messages and / or the message receiving time of the data transmitted by the active link and the detection link includes:

[0060] S301: Calculate the difference in the number of messages of the data transmitted by the main link and the detection link; S302: If the difference in the number of messages is greater than the message number threshold, it is determined that the link with the smaller number of messages has a transmission failure. The message number threshold can be determined and preset based on experience, and the message number threshold affects the fault detection accuracy to a certain extent. When two links transmit the same data synchronously, the number of messages passing through the two links should be consistent or have a small difference in the same time period. If the difference is large, it means that the link with the slower transmission has a transmission failure and the link is missing a certain number of messages.

[0061] In some other embodiments, determining the link and time at which the transmission failure occurs according to the number of messages of data transmitted by the active link and the detection link and / or the message receiving time includes:

[0062] S303: Calculate the reception time difference of the same message in the data transmitted by the main link and the detection link; S304: If the reception time difference is greater than the time difference threshold, it is determined that the link where the message received later has a transmission failure. Among them, the time difference threshold can be determined and preset based on experience, and the time difference threshold affects the fault detection accuracy to a certain extent. When two links transmit the same data synchronously, for the same message, the reception time arriving at the same monitoring point in the two links should be consistent or have a small difference. If the difference is large, it means that the link with the later reception time has a transmission failure. For example, in a 5-minute time window, for the same message 3, it has passed through the main link, but has not been received in the detection link, which means that the detection link has a transmission failure. The number of messages passing through the two links should be consistent or have a small difference. If the difference is large, it means that the link with the slower transmission has a transmission failure and the link is missing a certain number of messages.

[0063] In some other embodiments, determining the link and time where the transmission failure occurs according to the number of messages and / or the message reception time of the data transmitted by the main link and the detection link includes:

[0064] S305: Count the number of messages transmitted by the main link and the detection link respectively; S306 determines the link whose number of messages does not reach the set lower limit as the link with transmission failure. Two links transmit data synchronously. In a certain time window, if a transmission failure occurs, the number of messages passed by each link should be certain. For example, about 70 (±10) messages can be passed in 3 minutes. If the link only passes 20 messages in these 3 minutes, data loss has obviously occurred. The lower limit value can be set based on experience, for example, using a historical average. In this embodiment, independent detection of a single link can be achieved based on whether the number of messages passed in a fixed time window of each link reaches the lower limit, which can prevent the situation where transmission failures occur on two links at the same time and no results can be obtained through comparative detection, further improving the accuracy of transmission fault detection.

[0065] In some embodiments, the above three methods can also be used in combination and synchronously, for example, the difference in the number of messages and the difference in the receiving time of the data transmitted by the main link and the detection link can be monitored simultaneously, or the number of messages of the transmitted data and the difference in the number of messages can be monitored simultaneously, which is not limited here. The above message number threshold, time difference threshold and set lower limit value can be combined with data statistical characteristics to set parameter values ​​that are more accurate and consistent with actual fault scenarios, thereby reducing the workload of manual evaluation of parameter settings.

[0066] Optionally, the transmission data of the main link and the detection link are copied to the data backup area, and the method also includes: filtering the data transmitted by the main link and the detection link to remove duplicate data. The received source data may contain some duplicate data, for example, duplication occurs when the source data provider receives data from other nodes, or the source data itself contains duplicate data. The transmission of duplicate data wastes channel resources. In this embodiment, the duplicate data is firstly filtered out, and then dual-link transmission and backup are performed to reduce the waste of channel resources and improve data transmission efficiency.

[0067] Optionally, filtering the data transmitted by the main link and the detection link to remove duplicate data includes: receiving the transmission data of the main link and the detection link respectively; traversing the data buffer to see whether the transmission data is stored; if yes, determining that the transmission data is duplicate data and removing it; if no, determining that the transmission data is new data, releasing it and caching it in the data buffer. In this embodiment, the data buffer is used to filter the transmission data.

[0068] Exemplarily, the cache is designed as follows: the key is the unique identifier of the message, and the value is empty; the processing flow is: when the program starts, the cache will query the unique identifier of the processed data in the past hour from the original message history database by default for initialization. When the data queue receives a new message, the unique identifier in the message is first extracted, and the cache is queried based on the identifier. If a hit occurs, it means that the message has been processed, and the message is ignored. Otherwise, it means that the message is processed for the first time, and the message is pushed to the switch of the message queue, and a record of the message is added to the cache.

[0069] On the basis of the above embodiment, optionally, the pre-transmitted source data is obtained from the source data provider, and the source data is synchronously input into the main link and the detection link, including: adding a first identification header to the message in the source data and inputting it into the main link, adding a second identification header to the message in the source data and inputting it into the detection link; storing the transmission data of the main link and the detection link, and also including: removing the first identification header and the second identification header to obtain a sub-message; filtering out the message to be processed from the sub-message; and calling a corresponding parsing program to parse the message to be processed according to the type of the message to be processed. Adding an identification header to the message makes it easy to identify whether the message belongs to the main link or the detection link.

[0070] Taking the civil aviation field as an example, a header information identification layer is added to messages with different link data to mark the link status (for example, from link A, from link B). When filtering data, the transmission data of different links can be identified and filtered separately. For example, with the help of cache, the same message is compared (determined by the unique identifier of the message) to see if it is processed for the first time, and the first arrival link data of each message is processed and recorded in the cache. The second arrival message will be ignored based on the cache judgment that it has been processed.

[0071] On the basis of the above-mentioned embodiment, optionally, the dual-link-based industry-level data transmission method further includes: establishing a communication channel to a preset target distribution object, and blocking communication with a non-target distribution object. In this embodiment, a security network gate is set between the service platform and the data source network environment for selective isolation, thereby enhancing the security between different connected networks, and only configuring a connection channel for authorized business access (i.e., the target distribution object), thereby blocking potential risk attacks from the data source side and the platform side.

[0072] The above embodiments have provided a detailed description of the dual-link based industry-level data transmission provided by the present application. The following will specifically explain the dual-link based industry-level data transmission device, electronic device, storage medium and program product provided by the embodiments of the present application.

[0073] Figure 4 Schematic diagram of the functional modules of the dual-link industry-level data transmission device provided in the embodiment of the present application Figure 1 .like Figure 4 As shown, the dual link includes a main link and a detection link, and the industry-level data transmission device 400 includes: a receiving module 401, a backup module 402, a detection module 403, a reissue module 404 and a database module 405. The receiving module 401 is used to obtain the pre-transmitted source data from the source data provider, and synchronously input the source data into the main link and the detection link; the backup module 402 is used to copy the transmission data of the main link and the detection link to the data backup area; the detection module 403 is used to determine the link and time where the transmission failure occurs according to the number of messages and / or message reception time of the data transmitted by the main link and the detection link in a cycle within a set time window; the reissue module 404 is used to obtain the target data from the source data provider or the data backup area according to the link and time where the transmission failure occurs, and reissue it to the target location; the database module 405 is used to receive and store the transmission data of the main link and the detection link, and wait for distribution.

[0074] Optionally, the detection module 403 can be used to calculate the difference in the number of messages in the data transmitted by the main link and the detection link; if the difference in the number of messages is greater than a message number threshold, it is determined that a transmission failure has occurred in the link with fewer messages; and / or, calculate the difference in reception time of the same message in the data transmitted by the main link and the detection link; if the reception time difference is greater than a time difference threshold, it is determined that a transmission failure has occurred in the link where the message with the later reception time is located.

[0075] Optionally, the detection module 403 may also be configured to count the number of messages of data transmitted by the main link and the detection link respectively; and determine the link whose message number does not reach a set lower limit as a link with a transmission failure.

[0076] Optionally, the industry-level data transmission device 400 further includes a filtering module, which can be used to filter the data transmitted by the main link and the detection link to filter out duplicate data.

[0077] Optionally, the filtering module can also be used to receive the transmission data of the main link and the detection link respectively; traverse the data cache area to see whether the transmission data is stored; if so, the transmission data is determined to be duplicate data and filtered out; if not, the transmission data is determined to be new data, released, and cached in the data cache area.

[0078] Optionally, the receiving module 401 can also be used to add a first identification header to the message in the source data and input it into the main link, add a second identification header to the message in the source data and input it into the detection link; store the transmission data of the main link and the detection link, and also include: removing the first identification header and the second identification header to obtain sub-messages; filtering out the messages to be processed from the sub-messages; and calling the corresponding parsing program to parse the messages to be processed according to the type of the messages to be processed.

[0079] Optionally, the industry-level data transmission device 400 further includes a security module, which is used to establish a communication channel to a preset target distribution object and block communication with a non-target distribution object.

[0080] Figure 5 Schematic diagram of the functional modules of the dual-link industry-level data transmission device provided in the embodiment of the present application Figure 2 .like Figure 5 As shown, the device includes a receiving module 501, a filtering module 502, a backup module 503, a detection module 504, a re-issuing module 505, a database module 506 and a source data provider.

[0081] This embodiment takes the civil aviation field as an example to illustrate the basic working process of an industry-level data transmission device.

[0082] The receiving module 501 configures a connection of a multi-environment message queue to the source data provider to realize the simultaneous reception of data from two links, and at the same time adds a header information identification layer to the data of different links to mark the link status (for example: link A, from link B). With the help of the filtering module 502 cache, the same message is compared (based on the unique identifier of the message) to see if it is being processed for the first time, and the first arrival line data of each message is processed and recorded in the cache. The second arrival message will be ignored based on the cache judgment that it has been processed.

[0083] The backup module 503 backs up and stores the filtered transmission data / messages, waiting for the resending module 505 to obtain the target data. The backup module 503 receives the message data, binds it to the original message storage queue, combines the original data query scenario and the data resending scenario, adds a meta-information storage field, and stores the data source system identifier, data receiving time and other information.

[0084] The detection module 504 is used to identify link failures, detect data interruptions, trigger alarm prompts, and promptly notify the operation and maintenance personnel to troubleshoot and take recovery measures. The reissue module 505 provides a reissue restful API, which queries data from the source data provider or the backup module 503 for reissue according to conditions such as airport identification and time interval, wherein the restful API (Representational State Transfer Application Programming Interface) is a way to expose resources through the network, communicate through HTTP requests, and perform standard database functions, such as creating, reading, updating, and deleting records in resources. The reissue scenarios include: the first, the upstream data source fails, resulting in data loss; or the two links fail at the same time (less likely). After the data interruption is found according to the monitoring alarm, the reissue module promptly contacts the source data provider according to the time period of data loss (for example: data between 7 and 8 o'clock) to reissue and re-push the missing data. In the scenario where two links are interrupted, the confirmation mechanism of the message queue and the data serialization function can be combined to ensure that the data will not be lost once received, so there is no need to reissue. The second is that one of the two links fails, resulting in data loss. The failed link is identified, and the data of the normal link stored in the backup module 503 is reissued. The third is that the downstream has a data reissue requirement. According to the data stored in the backup module 503 or the source data message of the source data provider, according to the data range conditions of the target distribution object, the original message database is queried to complete data assembly and reissue.

[0085] The database module 506 is used to parse and store the received transmission data / message according to the business data protocol, extract and save the business information. The distribution module filters the data and distributes and pushes it in a targeted manner according to the shared data permissions of the downstream data demander (target distribution object). Specifically, the database module 506 performs parsing and processing according to the table design according to the business data protocol and persists it in the database for storage. When a message is received from the entry queue, the header information (identification header) in the message is read to obtain the message subtype, and it is determined whether the message subtype is the message type to be processed. If it is, the parsing program of the corresponding message is called to parse it and generate the corresponding database object, and then write it into the corresponding table of the database. If it is not one of the messages, it is logged and skipped. The database module 506 performs targeted data routing distribution according to the data requirements of different business units. Specifically, the flight departure station, landing station, and airline company in the airport support plan message are obtained, and the message of the flight is forwarded to the queue of the corresponding departure and landing airport according to the pre-configured airline-airport message routing rules. For messages that do not comply with the airline-airport message routing rules, they are logged and skipped. For messages that need to be stored, they are sent to the entry queue of the message storage program.

[0086] The device provided in the embodiment of the present application can be used to execute the above-mentioned industry-level data transmission method, and its implementation method and technical effect are similar, which will not be repeated here.

[0087] Figure 6 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. Figure 6 As shown, the electronic device 600 includes: a processor 601 and a memory 602 .

[0088] The processor 601 and the memory 602 can be connected to each other through a communication bus 603. The communication bus 603 is a common communication trunk for transmitting information between various functional components, and constitutes a communication link between the processor 601 and the memory 602. The communication bus 603 can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus. The communication bus 603 can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 6 Only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus. The memory 602 is used to store computer execution instructions, the communication interface is used to communicate with other devices, and the processor 601 is used to execute computer execution instructions to execute the solution of the industry-level data transmission method based on dual links in the above method embodiment.

[0089] Specifically, the processor 601 may include one or more processing units, for example, the processor 601 may be a CPU, or a digital signal processing (DSP), an application specific integrated circuit (ASIC), etc. A general processor may be a microprocessor or the processor may be any conventional processor, etc. The steps of the method disclosed in the application may be directly embodied as being executed by a hardware processor, or may be executed by a combination of hardware and software modules in the processor.

[0090] The memory 602 can be used to store program instructions. The memory 602 may include a program storage area and a data storage area. Among them, the program storage area may store an operating system, an application required for at least one function (such as a sound playback function, etc.), etc. The data storage area may store data created during the use of the electronic device 600 (such as audio data, etc.), etc. In addition, the memory 602 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (Universal Flash Storage, referred to as UFS), etc. The processor 601 executes various functional applications and data processing of the electronic device 600 by running the program instructions stored in the memory 602.

[0091] It should be noted that the number of the memory 602 and the processor 601 is not limited in the embodiment of the present application, and they can be one or more. Figure 6 Take one as an example for illustration; the memory 602 and the processor 601 can be connected by wire or wirelessly in various ways, such as through a bus connection. In practical applications, the electronic device 600 can be various forms of computers or mobile terminals. Among them, the computer is, for example, a laptop computer, a desktop computer, a workbench, a server, a blade server, a mainframe computer, etc.; the mobile terminal is, for example, a personal digital assistant, a cellular phone, a smart phone, a wearable device, and other similar computing devices.

[0092] The electronic device of this embodiment can be used to execute the technical solution in the above method embodiment. Its implementation principle and technical effect are similar and will not be described in detail here.

[0093] An embodiment of the present application also provides a computer-readable storage medium, in which computer-executable instructions are stored. When the computer-executable instructions are executed by a processor, they are used to implement the solution of the dual-link-based industry-level data transmission method in the above method embodiment.

[0094] The embodiment of the present application also provides a computer program product, including a computer program; when the computer program is executed, the solution of the industry-level data transmission method based on dual links in the above method embodiment is implemented.

[0095] The above are only preferred embodiments of the present application, and are not intended to limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. An industry-level data transmission method based on dual links, characterized in that: The dual link includes a main link and a detection link, and the method includes: Acquire pre-transmitted source data from a source data provider, and synchronously input the source data into the active link and the detection link; Copying the transmission data of the main link and the detection link to the data backup area; Cyclic in a set time window, according to the number of messages of data transmitted by the main link and the detection link and / or the message receiving time, determine the link and time where the transmission failure occurs; According to the link and time where the transmission failure occurs, the target data is obtained from the source data provider or the data backup area, and resent to the target location; The transmission data of the main link and the detection link are received and stored, waiting for distribution.

2. The industry-level data transmission method according to claim 1, characterized in that: The determining, according to the number of messages and / or the message receiving time of the data transmitted by the main link and the detection link, the link and time at which the transmission failure occurs includes: Calculating the difference between the number of messages of data transmitted by the main link and the detection link; If the difference in the number of messages is greater than the message number threshold, it is determined that a transmission failure occurs on the link with fewer messages; and / or, Calculating a reception time difference of the same message in the data transmitted by the active link and the detection link; If the reception time difference is greater than the time difference threshold, it is determined that a transmission failure occurs on the link where the message with the later reception time is located.

3. The industry-level data transmission method according to claim 2, characterized in that: The determining, according to the number of messages and / or the message receiving time of the data transmitted by the main link and the detection link, the link and time at which the transmission failure occurs, further comprises: Counting the number of messages of data transmitted by the main link and the detection link respectively; The link whose number of messages does not reach the set lower limit is determined as a link where a transmission failure occurs.

4. The industry-level data transmission method according to claim 2, characterized in that: The step of copying the transmission data of the main link and the detection link to the data backup area also includes: The data transmitted by the main link and the detection link are filtered to remove duplicate data.

5. The industry-level data transmission method according to claim 4, characterized in that: The filtering of the data transmitted by the primary link and the detection link to remove duplicate data includes: Receiving transmission data of the main link and the detection link respectively; Check whether the data buffer stores the transmission data; If yes, the transmitted data is considered as duplicate data and is filtered out; If not, the transmission data is deemed to be new data, released, and cached in the data cache area.

6. The industry-level data transmission method according to any one of claims 1 to 5, characterized in that: The obtaining of the pre-transmitted source data from the source data provider and synchronously inputting the source data into the active link and the detection link comprises: adding a first identification header to a message in the source data and inputting the message into the active link, and adding a second identification header to a message in the source data and inputting the message into the detection link; The storage of the transmission data of the main link and the detection link also includes: removing the first identification header and the second identification header to obtain a sub-message; filtering out the message to be processed from the sub-message; and calling a corresponding parsing program to parse the message to be processed according to the type of the message to be processed.

7. The industry-level data transmission method according to any one of claims 1 to 5, characterized in that: The method also includes establishing a communication channel to a preset target distribution object and blocking communication with a non-target distribution object.

8. An industry-level data transmission device based on dual links, characterized in that: The dual link includes a main link and a detection link, and the device includes a receiving module, a backup module, a detection module, a re-issuing module and a database module. The receiving module is used to obtain pre-transmitted source data from a source data provider, and synchronously input the source data into the main link and the detection link; The backup module is used to copy the transmission data of the main link and the detection link to the data backup area; The detection module is used to determine the link and time where the transmission failure occurs according to the number of messages and / or message reception time of the data transmitted by the main link and the detection link in a cycle within a set time window; The resending module is used to obtain the target data from the source data provider or the data backup area according to the link and time where the transmission failure occurs, and resend it to the target location; The database module is used to receive and store the transmission data of the main link and the detection link, waiting for distribution.

9. An electronic device, characterized in that: include: Memory, processor; The memory is used to store program instructions; The processor is used to call the program instructions to execute the industry-level data transmission method according to any one of claims 1-7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the industry-level data transmission method according to any one of claims 1 to 7.

Images