ADS-B (Automatic Dependent Surveillance-Broadcast) data connection and processing method and device, equipment and storage medium

Abstract

The invention discloses an ADS-B (Automatic Dependent Surveillance-Broadcast) data connection and processing method, device and equipment and a storage medium, and relates to the field of aviation, and the method comprises the steps: carrying out the connection of ADS-B data in an ADS-B message, and stamping a timestamp to obtain message data; forwarding the message data to a Broker of the Kafka message middleware so as to divide the message data, and distributing a plurality of pieces of divided message data to a plurality of different first partitions; respectively consuming each first partition to analyze an ICAO address corresponding to the divided message data in each first partition, and respectively distributing all the divided message data with the same ICAO address in all the first partitions to the same second partition; and consumption is carried out on each second partition, analysis processing is carried out on each divided message data in each second partition, and post-processing operation is carried out by using the analyzed message data. According to the method, distributed connection can be carried out on a large amount of ADS-B data, and analysis processing is carried out while connection is carried out, so that the message processing efficiency is improved.

Description

Technical Field

[0001] The present application relates to the field of aviation, and in particular to an ADS-B data connection and processing method, device, equipment and storage medium. Background Art

[0002] The ADS-B (Automatic Dependent Surveillance-Broadcast) system, comprised of multiple ground and airborne stations, enables two-way data communication in a meshed, multipoint-to-multipoint fashion. It primarily implements air-to-air surveillance. Without requiring manual operation or query, the ADS-B system automatically obtains information such as its own position, altitude, speed, heading, and identification number from relevant airborne equipment and broadcasts it to other aircraft or ground stations, allowing air traffic controllers, pilots, and ground surveillance personnel to monitor aircraft status.

[0003] However, in a satellite-based ADS-B aviation surveillance system designed for global coverage, the data received by the surveillance system ports may originate from forwarding from the system's operations control center and satellite ground stations. Each operations control center receives data from global ADS-B raw messages, potentially outputting tens of thousands, or even hundreds of thousands, of ADS-B messages per second. However, current ADS-B routing services are unable to simultaneously process and analyze this massive amount of data, resulting in significant packet loss and latency. Furthermore, due to the unique format and parsing methods of ADS-B data, individual ADS-B messages cannot be disassembled. Furthermore, due to the interlinked relationships between ADS-B messages, different messages from the same flight or aircraft cannot be individually disassembled and sent to different service programs. Current distributed ADS-B routing solutions, such as those based on microservice frameworks like Spring Cloud, are also unable to route and process ADS-B data with these special interlinked relationships.

[0004] In summary, when a port receives massive amounts of ADS-B data, how to connect and process the massive amounts of ADS-B data is a problem that still needs to be further solved in this field. Summary of the Invention

[0005] In view of this, the purpose of this application is to provide an ADS-B data ingestion and processing method, apparatus, device, and storage medium that can perform distributed ingestion of large amounts of ADS-B data and simultaneously parse and process it, thereby improving the efficiency of ADS-B message processing and avoiding packet loss and delay. The specific solution is as follows:

[0006] In a first aspect, the present application discloses an ADS-B data induction and processing method, which is applied to the ADS-B system, comprising:

[0007] When an ADS-B message is received, the ADS-B data in the ADS-B message is linked and timestamped to obtain message data;

[0008] Forwarding the message data to a Broker of the Kafka message middleware to divide the message data to obtain a plurality of divided message data, and allocating the plurality of divided message data to a plurality of different first partitions;

[0009] Consuming each of the first partitions respectively to resolve the ICAO address corresponding to the divided message data in each of the first partitions, and respectively allocating all of the divided message data having the same ICAO address in all of the first partitions to the same second partition;

[0010] Each of the second partitions is consumed respectively to parse and process each of the divided message data in each of the second partitions to obtain parsed message data, and then perform post-processing operations using the parsed message data.

[0011] Optionally, the linking and timestamping the ADS-B data in the ADS-B message to obtain message data includes:

[0012] The ADS-B data in the ADS-B message is accessed and timestamped by the distribution service to obtain message data.

[0013] Optionally, forwarding the message data to a Broker of a Kafka message middleware to divide the message data to obtain a plurality of divided message data, and allocating the plurality of divided message data to a plurality of different first partitions includes:

[0014] The message data is forwarded to the Broker of the Kafka message middleware by random partitioning, so as to divide the message data into a plurality of divided message data, and the plurality of divided message data are randomly distributed to a preset number of different first partitions.

[0015] Optionally, consuming each of the first partitions separately includes:

[0016] Each of the first partitions is consumed respectively by using a pre-created pre-parsing service program corresponding to each of the first partitions.

[0017] Optionally, consuming each of the second partitions separately includes:

[0018] Each of the second partitions is consumed respectively by using a pre-created real-time parsing service program corresponding to each of the second partitions.

[0019] Optionally, the ADS-B data connection and processing method further includes:

[0020] Determining the number of the pre-analysis service programs based on the message output frequency and the computing capability of the pre-analysis service programs;

[0021] The number of the real-time analysis service programs is determined based on the message output frequency and the computing capability of the real-time analysis service programs.

[0022] Optionally, the respectively allocating all the divided message data having the same ICAO address in all the first partitions to the same second partition includes:

[0023] Based on the principle of average distribution, all the divided message data with the same ICAO address in all the first partitions are respectively distributed to the same second partition.

[0024] In a second aspect, the present application discloses an ADS-B data connection and processing device, which is applied to the ADS-B system, comprising:

[0025] A data connection module is used to connect and timestamp the ADS-B data in the ADS-B message when the ADS-B message is received, so as to obtain the message data;

[0026] A data forwarding module, configured to forward the message data to a Broker of the Kafka message middleware to divide the message data to obtain a plurality of divided message data, and to distribute the plurality of divided message data to a plurality of different first partitions;

[0027] a first partition consumption module, configured to consume each of the first partitions respectively, to parse the ICAO address corresponding to the divided message data in each of the first partitions, and to distribute all the divided message data having the same ICAO address in all the first partitions to the same second partition;

[0028] The second partition consumption module is used to consume each of the second partitions respectively, to parse and process each of the divided message data in each of the second partitions respectively, to obtain parsed message data, and to perform post-processing operations using the parsed message data.

[0029] In a third aspect, the present application discloses an electronic device comprising a processor and a memory; wherein, when the processor executes a computer program stored in the memory, the aforementioned ADS-B data connection and processing method is implemented.

[0030] In a fourth aspect, the present application discloses a computer-readable storage medium for storing a computer program; wherein, when the computer program is executed by a processor, the aforementioned ADS-B data connection and processing method is implemented.

[0031] It can be seen that when the present application receives an ADS-B message, it first links and timestamps the ADS-B data in the ADS-B message to obtain message data, and then forwards the message data to the Broker of the Kafka message middleware to divide the message data to obtain multiple divided message data, and allocate the multiple divided message data to multiple different first partitions, and then consume each of the first partitions respectively to parse the ICAO address corresponding to the divided message data in each of the first partitions, and allocate all the divided message data with the same ICAO address in all the first partitions to the same second partition, and then consume each of the second partitions respectively to parse and process each of the divided message data in each of the second partitions to obtain parsed message data, and use the parsed message data to perform post-processing operations. Before parsing ADS-B messages, this application first partitions the messages through Kafka message middleware and assigns messages from the same object to the same partition. It can distribute and receive a large amount of ADS-B data and parse and process it while receiving, thereby improving the efficiency of ADS-B message processing and avoiding packet loss and delay. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are merely embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without any creative work.

[0033] Figure 1 A flowchart of an ADS-B data connection and processing method disclosed in this application;

[0034] Figure 2 A specific ADS-B data connection and processing flow chart disclosed in this application;

[0035] Figure 3 A flowchart of a specific ADS-B data connection and processing method disclosed in this application;

[0036] Figure 4 This is a schematic diagram of the structure of an ADS-B data connection and processing device disclosed in this application;

[0037] Figure 5 This is a structural diagram of an electronic device disclosed in this application. DETAILED DESCRIPTION

[0038] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0039] The present invention discloses an ADS-B data connection and processing method, which is applied to the ADS-B system. Figure 1 As shown, the method includes:

[0040] Step S11: When an ADS-B message is received, the ADS-B data in the ADS-B message is linked and timestamped to obtain message data.

[0041] It should be noted that the ADS-B data concatenation and processing solution proposed in this application is applied to an ADS-B system, which can be composed of multiple ground stations and airborne stations and can achieve two-way data communication in a meshed, multi-point-to-multi-point manner, and can be primarily used for air-to-air surveillance. When the ADS-B system receives massive ADS-B messages, such as DF17, DF18, and DF19, forwarded by terminals such as an Operation Control Center (OCC) or a satellite ground station, it first concatenates and timestamps the ADS-B data contained in the aforementioned ADS-B messages to obtain the corresponding message data.

[0042] Specifically, the step of extracting and timestamping the ADS-B data in the ADS-B message to obtain the message data may include extracting and timestamping the ADS-B data in the ADS-B message through a distribution service to obtain the message data. Figure 2 As shown in the figure, when receiving massive ADS-B messages from the operation control center, a distribution service, Dispenser, can be used to connect and timestamp the ADS-B data in the ADS-B messages, thereby obtaining the corresponding ADS-B message data. It should be noted that Disperser is only used for message forwarding and does not perform any substantive data processing operations.

[0043] Step S12: forwarding the message data to the Broker of the Kafka message middleware to divide the message data to obtain a plurality of divided message data, and allocating the plurality of divided message data to a plurality of different first partitions.

[0044] In this embodiment, after the ADS-B data in the ADS-B message is linked and timestamped to obtain the message data, the above message data is further forwarded to the Broker of the Kafka (a high-throughput distributed publish-subscribe messaging system) message middleware. When the Broker receives the above message data, it first divides the above message data to obtain multiple divided message data, and then distributes the multiple divided message data to multiple different first partitions. It can be understood that the Kafka middle message ware is a multi-partition, multi-replica distributed messaging system based on ZooKeeper coordination, and has many characteristics such as high throughput, persistence, horizontal scalability, and support for stream data processing. The Kafka middle message ware can specifically include multiple Producers, multiple Brokers, and multiple Consumers. Among them, the Producer is responsible for sending messages to the Broker, and the Consumer is responsible for subscribing to messages from the Broker and consuming the messages. In addition, the message needs to be defined by a Topic. The Producer and Consumer can identify the message based on the Topic, and the Topic can also be further divided into multiple Partitions by the user, and then multiple Consumers perform task assignments, each responsible for several Partitions, and a Partition can only be consumed by one Consumer at most. In addition, the Kafka middle message ware supports custom Partitions. Considering that after receiving massive amounts of ADS-B data, a single service cannot handle the parsing and processing of a large amount of ADS-B data, and cannot implement a safe redundant design, and the Kafka message middleware has a partitioning mechanism, the partitioning mechanism of the Kafka message middleware can be used to classify and reorganize ADS-B message data.

[0045] In a specific embodiment, the message data is forwarded to the Broker of the Kafka message middleware to divide the message data to obtain a plurality of divided message data, and the plurality of divided message data are distributed to a plurality of different first partitions. Specifically, it may include: forwarding the message data to the Broker of the Kafka message middleware by random partitioning to divide the message data to obtain a plurality of divided message data, and randomly distributing the plurality of divided message data to a preset number of multiple different first partitions. In this embodiment, the message data can be forwarded to the Broker of the Kafka message middleware by random partitioning, so that the message data can be divided by the Broker to obtain a plurality of divided message data, and then the plurality of divided message data are randomly distributed to a preset number n of multiple different first partitions; wherein the preset number n can be selected according to actual application requirements.

[0046] Step S13: Consume each of the first partitions respectively to parse the ICAO address corresponding to the divided message data in each of the first partitions, and distribute all the divided message data with the same ICAO address in all the first partitions to the same second partition respectively.

[0047] In this embodiment, when designing the distributed structure of the Kafka message middleware, it is considered that the messages of the same flight or the same aircraft must be transmitted to the same parsing service program for processing, and the messages of different flights or different aircraft can be handed over to different parsing service programs for processing, so the message data with the same address (i.e. the same data source) can be allocated to the same partition. Specifically, after allocating a plurality of the divided message data to a plurality of different first partitions, each of the above-mentioned first partitions is consumed respectively, thereby parsing out the ICAO (International Civil Aviation Organization, International Civil Aviation Organization) address corresponding to the divided message data in each of the above-mentioned first partitions, and then all the divided message data with the same ICAO address in all the above-mentioned first partitions are allocated to the same second partition, that is, the message data is partitioned twice. Among them, the number m of the second partitions can be set according to the actual application requirements, and can be dynamically expanded horizontally according to the number of messages to be processed.

[0048] Specifically, the consuming of each first partition may include: using a pre-created pre-analysis service program corresponding to each first partition to consume each first partition. Figure 2As shown, the pre-parsing service program (PreParser) created in advance and corresponding to each first partition (Partition_1) can be used to consume each first partition (Partition_1) respectively, such as using PreParser_1 to consume Partition_1_1, using PreParser_2 to consume Partition_1_2, etc. It should be noted that there is a one-to-one correspondence between the pre-parsing service program and the first partition (Partition_1). In addition, the Topic (topic) corresponding to the first partition needs to be set, such as setting it to Kafka_Topic1:RAW_DATA. After consuming each first partition (Partition_1) through the pre-parsing service program (PreParser), the ICAO address corresponding to the message data in each first partition (Partition_1) can be parsed, that is, the ICAO address of the ADS-B message can be parsed, and then all message data with the same ICAO address in all first partitions (Partition_1) can be assigned to the same second partition (Partition_2). In addition, you also need to set the Topic corresponding to the second partition, such as setting it to Kafka_Topic2:RAW_DATA_SORTED.

[0049] In a specific embodiment, when Partition_2 is divided based on ICAO address, in order to ensure that the same ICAO is assigned to the same Partition_2, the message can be partitioned into partitions with area code (A%m); where A is the ICAO address of a message and m is the total number of Partition_2.

[0050] Step S14: Consume each of the second partitions respectively to parse and process each of the divided message data in each of the second partitions respectively to obtain parsed message data, and perform post-processing operations using the parsed message data.

[0051] In this embodiment, after all the divided message data having the same ICAO address in all the first partitions (Partition_1) are allocated to the same second partition (Partition_2), each of the above-mentioned second partitions (Partition_2) can be consumed separately, thereby parsing and processing each of the divided message data in each of the above-mentioned second partitions (Partition_2) to obtain corresponding parsed message data. Furthermore, the above-mentioned parsed message data can also be used to perform corresponding post-processing operations according to actual project requirements.

[0052] Specifically, the consuming of each second partition may include: using a pre-created real-time parsing service program corresponding to each second partition to consume each second partition. Figure 2 As shown, the pre-created real-time parsing service program (Parser) can be used to consume each second partition (Partition_2) respectively, such as using Parser_1 to consume Partition_2_1, using Parser_2 to consume Partition_2_2, etc. It should be pointed out that there is a one-to-one correspondence between the real-time parsing service program and the second partition (Partition_2).

[0053] In a specific embodiment, it may also include: determining the number of the pre-parsing service programs based on the message output frequency and the computing power of the pre-parsing service program; determining the number of the real-time parsing service programs based on the message output frequency and the computing power of the real-time parsing service program. For example, when the message output frequency is 100,000 messages / second and the computing power of the pre-parsing service program (i.e., PreParser) is 20,000 messages / second, then n=5; when the message output frequency is 100,000 messages / second and the computing power of the real-time parsing service program (i.e., Parser) is 20,000 messages / second, then m=5. It should be pointed out that in actual applications, the number m of real-time parsing service programs (i.e., Parsers) can be less than but not greater than the number of Partition_2. When the number m of Parsers is small, there may be a situation where a single Parser consumes multiple Partition_2s, which may cause insufficient consumption capacity. Therefore, this application adopts the number m of Parsers to be consistent with the number of Partition_2.

[0054] Also, see Figure 2As shown, the pre-parse service program PreParser is responsible for receiving and processing messages with the topic RAW_DATA. In order to quickly receive and process messages, it is preferred that the number of pre-parse service program PreParser be set to be greater than or equal to the number n of RAW_DATA partitions Partition_1. This is because when the number of pre-parse service program PreParser is less than n, some pre-parse service program PreParser will bear more processing tasks; when the number of pre-parse service program PreParser is greater than n, some pre-parse service program PreParser will be idle and can be used as backup services. When some pre-parse service program PreParser services are unavailable or abnormal, these backup services will immediately enter working mode to perform corresponding reception and processing operations. Similarly, the number of real-time analysis service program Parser should be greater than or equal to the number m of RAW_DATA_SORTED partitions Partition_2. It should be noted that each PreParser should meet the processing capacity of (total number of messages / n) / second, and each Parser should meet the processing capacity of (total number of messages / m) / second.

[0055] It can be seen that when the embodiment of the present application receives an ADS-B message, the ADS-B data in the ADS-B message is first linked and timestamped to obtain message data, and then the message data is forwarded to the Broker of the Kafka message middleware to divide the message data to obtain multiple divided message data, and the multiple divided message data are allocated to multiple different first partitions, and then each of the first partitions is consumed respectively to parse the ICAO address corresponding to the divided message data in each first partition, and all the divided message data with the same ICAO address in all the first partitions are allocated to the same second partition respectively, and then each of the second partitions is consumed respectively to parse and process each divided message data in each second partition respectively to obtain parsed message data, and the parsed message data is used to perform post-processing operations. Before parsing ADS-B messages, the embodiment of the present application first partitions the messages through the Kafka message middleware and assigns messages from the same object to the same partition. This allows for distributed routing of large amounts of ADS-B data and allows for parsing and processing of the data while routing, thereby improving the efficiency of ADS-B message processing and avoiding packet loss and delay.

[0056] The present application discloses a specific ADS-B data connection and processing method, which is applied to the ADS-B system. Figure 3 As shown, the method includes:

[0057] Step S21: When an ADS-B message is received, the ADS-B data in the ADS-B message is accessed and timestamped by the distribution service to obtain message data.

[0058] Step S22: forwarding the message data to the Broker of the Kafka message middleware by random partitioning, so as to divide the message data into a plurality of divided message data, and randomly distribute the plurality of divided message data into a preset number of different first partitions.

[0059] Step S23: Utilize the pre-created pre-parsing service program corresponding to each first partition to consume each first partition respectively, so as to parse the ICAO address corresponding to the divided message data in each first partition, and distribute all the divided message data with the same ICAO address in all the first partitions to the same second partition based on the principle of average distribution.

[0060] In this embodiment, after randomly distributing the plurality of divided message data to a preset number of different first partitions, a pre-created pre-parsing service program (i.e., PreParser) corresponding to each of the first partitions (Partition_1) can be used to consume each of the first partitions (Partition_1) respectively, thereby parsing the ICAO address corresponding to the divided message data in each of the first partitions (Partition_1), and then, based on the principle of average distribution, all the divided message data with the same ICAO address in all the first partitions (Partition_1) are evenly distributed to the same second partition (Partition_2), that is, the message data with the same ICAO address are evenly distributed to the same partition.

[0061] Step S24: Use the pre-created real-time parsing service program corresponding to each second partition to consume each second partition respectively, so as to parse and process each divided message data in each second partition respectively, obtain parsed message data, and use the parsed message data to perform post-processing operations.

[0062] For more specific processing procedures of the above steps S21, S22, and S24, reference may be made to the corresponding contents disclosed in the aforementioned embodiments, which will not be repeated here.

[0063] It can be seen that in the embodiment of the present application, after receiving the ADS-B message, the ADS-B data in the ADS-B message is first linked and timestamped to obtain the message data, and then input into the Broker of the Kafka message middleware to divide the message data and randomly distribute it to n first partitions, and then use the pre-created pre-parsing service program (i.e., PreParser) corresponding to each first partition to consume each first partition respectively, so as to divide the message data in the n partitions into m second partitions based on the principle of average distribution and according to the ICAO address, and then use the pre-created real-time parsing service program (i.e., Parser) corresponding to each second partition to consume each second partition respectively, so as to parse and process the message data in each second partition respectively, thereby obtaining the parsed message data, and performing post-processing operations using the parsed message data. Before parsing ADS-B messages, this application first partitions the messages through Kafka message middleware and evenly distributes messages from the same object to the same partition. It can distribute and receive a large amount of ADS-B data, parse and process it while receiving it, and then use the parsed message data for post-processing operations, thereby improving the efficiency of ADS-B message processing and avoiding packet loss and delay.

[0064] Accordingly, the present application also discloses an ADS-B data connection and processing device, which is applied to the ADS-B system, see Figure 4 As shown, the device includes:

[0065] The data connection module 11 is used to connect the ADS-B data in the ADS-B message and add a timestamp to obtain the message data when the ADS-B message is received;

[0066] A data forwarding module 12 is configured to forward the message data to a broker of a Kafka message middleware to divide the message data into a plurality of divided message data, and distribute the plurality of divided message data into a plurality of different first partitions;

[0067] The first partition consumption module 13 is configured to consume each of the first partitions respectively, resolve the ICAO address corresponding to the divided message data in each of the first partitions, and respectively distribute all the divided message data having the same ICAO address in all the first partitions to the same second partition;

[0068] The second partition consumption module 14 is used to consume each of the second partitions respectively, to parse and process each of the divided message data in each of the second partitions respectively, to obtain parsed message data, and to perform post-processing operations using the parsed message data.

[0069] Among them, the specific work processes of the above modules can refer to the corresponding contents disclosed in the aforementioned embodiments, which will not be repeated here.

[0070] It can be seen that in an embodiment of the present application, when an ADS-B message is received, the ADS-B data in the ADS-B message is first linked and timestamped to obtain message data, and then the message data is forwarded to the Broker of the Kafka message middleware to divide the message data to obtain multiple divided message data, and the multiple divided message data are allocated to multiple different first partitions, and then each of the first partitions is consumed respectively to parse the ICAO address corresponding to the divided message data in each first partition, and all the divided message data with the same ICAO address in all the first partitions are allocated to the same second partition respectively, and then each of the second partitions is consumed respectively to parse and process each divided message data in each second partition respectively to obtain parsed message data, and the parsed message data is used to perform post-processing operations. Before parsing ADS-B messages, the embodiment of the present application first partitions the messages through the Kafka message middleware and assigns messages from the same object to the same partition. This allows for distributed routing of large amounts of ADS-B data and allows for parsing and processing of the data while routing, thereby improving the efficiency of ADS-B message processing and avoiding packet loss and delay.

[0071] In some specific embodiments, the data connection module 11 may specifically include:

[0072] The data induction unit is used to induct the ADS-B data in the ADS-B message through the distribution service and add a time stamp to obtain the message data.

[0073] In some specific embodiments, the data forwarding module 12 may specifically include:

[0074] A data forwarding unit is used to forward the message data to the Broker of the Kafka message middleware by random partitioning, so as to divide the message data into multiple divided message data, and randomly distribute the multiple divided message data to a preset number of multiple different first partitions.

[0075] In some specific embodiments, the first partition consumption module 13 may specifically include:

[0076] The first partition consumption unit is configured to consume each of the first partitions respectively by using a pre-created pre-parsing service program corresponding to each of the first partitions.

[0077] In some specific embodiments, the second partition consumption module 14 may specifically include:

[0078] The second partition consumption unit is configured to consume each of the second partitions respectively by using a pre-created real-time parsing service program corresponding to each of the second partitions.

[0079] In some specific embodiments, the ADS-B data connection and processing device may further include:

[0080] a first quantity determining unit, configured to determine the quantity of the pre-parsing service programs based on a message output frequency and a computing capability of the pre-parsing service programs;

[0081] The second quantity determining unit is configured to determine the quantity of the real-time analysis service programs based on the message output frequency and the computing capability of the real-time analysis service programs.

[0082] In some specific embodiments, the first partition consumption module 13 may specifically include:

[0083] The data distribution unit is configured to distribute all the divided message data having the same ICAO address in all the first partitions to the same second partition based on an average distribution principle.

[0084] Furthermore, the embodiment of the present application also discloses an electronic device, Figure 5 This is a structural diagram of an electronic device 20 according to an exemplary embodiment. The content in the diagram should not be considered as any limitation to the scope of application of the present application.

[0085] Figure 5 This is a schematic diagram of the structure of an electronic device 20 provided in an embodiment of the present application. The electronic device 20 may specifically include: at least one processor 21, at least one memory 22, a power supply 23, a communication interface 24, an input / output interface 25, and a communication bus 26. The memory 22 is used to store a computer program, which is loaded and executed by the processor 21 to implement the relevant steps of the ADS-B data access and processing method disclosed in any of the aforementioned embodiments. Furthermore, the electronic device 20 in this embodiment may specifically be an electronic computer.

[0086] In this embodiment, the power supply 23 is used to provide operating voltage for each hardware device on the electronic device 20; the communication interface 24 can create a data transmission channel between the electronic device 20 and the external device. The communication protocol it follows is any communication protocol that can be applied to the technical solution of this application and is not specifically limited here; the input and output interface 25 is used to obtain external input data or output data to the outside world. Its specific interface type can be selected according to specific application needs and is not specifically limited here.

[0087] In addition, the memory 22, as a carrier for resource storage, can be a read-only memory, random access memory, disk or CD, etc. The resources stored thereon can include an operating system 221, a computer program 222, etc., and the storage method can be temporary storage or permanent storage.

[0088] The operating system 221 is used to manage and control the hardware devices and computer program 222 on the electronic device 20, and can be Windows Server, NetWare, Unix, Linux, etc. In addition to including computer programs capable of implementing the ADS-B data interface and processing method disclosed in any of the aforementioned embodiments and executed by the electronic device 20, the computer program 222 may further include computer programs capable of performing other specific tasks.

[0089] Furthermore, this application discloses a computer-readable storage medium for storing a computer program. When executed by a processor, the computer program implements the aforementioned ADS-B data ingestion and processing method. The specific steps of this method can be found in the aforementioned embodiments and are not further detailed here.

[0090] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from the other embodiments. Reference can be made to the descriptions of the identical or similar parts between the various embodiments. For the devices disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the descriptions are relatively simple, and the relevant parts can be referred to the descriptions of the methods.

[0091] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the above description has generally described the components and steps of each example according to their functions. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0092] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein may be implemented directly using hardware, a software module executed by a processor, or a combination of the two. The software module may be placed in a random access memory (RAM), internal memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

[0093] Finally, it should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of additional identical elements in the process, method, article, or device comprising the element.

[0094] The above is a detailed introduction to the ADS-B data connection and processing method, device, equipment and storage medium provided by the present application. Specific examples are used herein to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method and core ideas of the present application. At the same time, for those skilled in the art, according to the ideas of the present application, there may be changes in the specific implementation methods and application scopes. In summary, the content of this specification should not be understood as limiting the present application.

Claims

1. A method for ingesting and processing ADS-B data, characterized in that: Applied to ADS-B system, including: When an ADS-B message is received, the ADS-B data in the ADS-B message is linked and timestamped to obtain message data; Forwarding the message data to a Broker of the Kafka message middleware to divide the message data to obtain a plurality of divided message data, and allocating the plurality of divided message data to a plurality of different first partitions; Consuming each of the first partitions respectively to resolve the ICAO address corresponding to the divided message data in each of the first partitions, and respectively allocating all of the divided message data having the same ICAO address in all of the first partitions to the same second partition; Each of the second partitions is consumed respectively to parse and process each of the divided message data in each of the second partitions to obtain parsed message data, and then perform post-processing operations using the parsed message data.

2. The ADS-B data connection and processing method according to claim 1, characterized in that: The ADS-B data in the ADS-B message is accessed and timestamped to obtain message data, including: The ADS-B data in the ADS-B message is accessed and timestamped by the distribution service to obtain message data.

3. The ADS-B data connection and processing method according to claim 1, characterized in that: The forwarding of the message data to the Broker of the Kafka message middleware to divide the message data to obtain a plurality of divided message data, and allocating the plurality of divided message data to a plurality of different first partitions includes: The message data is forwarded to the Broker of the Kafka message middleware by random partitioning, so as to divide the message data into a plurality of divided message data, and the plurality of divided message data are randomly distributed to a preset number of different first partitions.

4. The ADS-B data connection and processing method according to claim 1, characterized in that: The consuming each of the first partitions separately includes: Each of the first partitions is consumed respectively by using a pre-created pre-parsing service program corresponding to each of the first partitions.

5. The ADS-B data connection and processing method according to claim 4, characterized in that: The consuming each of the second partitions separately includes: Each of the second partitions is consumed respectively by using a pre-created real-time parsing service program corresponding to each of the second partitions.

6. The ADS-B data connection and processing method according to claim 5, characterized in that: Also includes: Determining the number of the pre-analysis service programs based on the message output frequency and the computing capability of the pre-analysis service programs; The number of the real-time analysis service programs is determined based on the message output frequency and the computing capability of the real-time analysis service programs.

7. The ADS-B data connection and processing method according to any one of claims 1 to 6, characterized in that: The respectively allocating all the divided message data having the same ICAO address in all the first partitions to the same second partition includes: Based on the principle of average distribution, all the divided message data with the same ICAO address in all the first partitions are respectively distributed to the same second partition.

8. An ADS-B data connection and processing device, applied to the ADS-B system, characterized in that: include: A data connection module is used to connect and timestamp the ADS-B data in the ADS-B message when the ADS-B message is received, so as to obtain the message data; A data forwarding module, configured to forward the message data to a Broker of the Kafka message middleware to divide the message data to obtain a plurality of divided message data, and to distribute the plurality of divided message data to a plurality of different first partitions; a first partition consumption module, configured to consume each of the first partitions respectively, to parse the ICAO address corresponding to the divided message data in each of the first partitions, and to distribute all the divided message data having the same ICAO address in all the first partitions to the same second partition; The second partition consumption module is used to consume each of the second partitions respectively, to parse and process each of the divided message data in each of the second partitions respectively, to obtain parsed message data, and to perform post-processing operations using the parsed message data.

9. An electronic device, characterized in that: The method comprises a processor and a memory; wherein, when the processor executes the computer program stored in the memory, the ADS-B data connection and processing method according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium, characterized in that Used to store a computer program; wherein, when the computer program is executed by a processor, it implements the ADS-B data connection and processing method according to any one of claims 1 to 7.

Images