Flight data change message forwarding processing method, system, equipment and medium
By performing timing and repeated filtering of flight data change messages, distributing them to message queues and processing them in parallel, the problem of inefficiency in handling flight data changes is solved, and efficient and reliable flight data synchronization is achieved.
Patent Information
- Application Number
- CN202411884541.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-12-19
- Publication Date
- 2025-04-04
AI Technical Summary
When airlines and airports process flight data changes, the existing technology is inefficient, prone to errors and lags behind the development of modern technology, and urgently needs methods to automatically process large amounts of flight data changes.
Provide a method for forwarding flight data change packets, including obtaining flight data change packets, performing message timing filtering and repeated filtering, deleting abnormal packets, and distributing them to the corresponding message queue through a preset relationship mapping table. The first thread is started for the message queue for parallel processing, and a message sending pool is created through the second thread, and the processed message is sent to the server.
It realizes efficient and reliable flight change data processing, improves the synchronization efficiency of flight data changes, avoids packet accumulation, and ensures real-time and accuracy of data.
Smart Images
Figure CN119763373A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the technical field of flight messages, and in particular, relates to a method, system, electronic device and storage medium for forwarding and processing flight data change messages. Background Art
[0002] As the aviation industry continues to expand and develop, the number of flights and route networks are also increasing, which has led to airlines and airports having to deal with more flight data changes. When airlines or airports make changes to flight data, they need to synchronize the flight change data to the departure system of the travel information.
[0003] Airlines or airport departure control rooms generally modify flight data by manually inputting black screen instructions to synchronize to the departure system of the aviation information system. This method is not only inefficient and prone to errors, but also lags behind the level of modern technological development. In this context, the civil aviation information industry has an increasingly urgent need for automated processing of flight data changes, hoping to efficiently process a large number of flight data changes through automation. Summary of the invention
[0004] To solve the above problems, the present invention provides a flight data change message forwarding processing method, system, electronic device and storage medium. This solution provides an efficient and reliable flight change data processing system for airlines and airports, so that airlines or airports can synchronize flight change data to the departure system in a timely manner when changing flight data.
[0005] In order to solve the above technical problems, the first aspect of the present invention proposes a method for forwarding and processing a flight data change message, the method comprising:
[0006] Acquire flight data change messages, perform message timing filtering and message duplication filtering on the flight data change messages to determine abnormal messages, and delete the abnormal messages;
[0007] Distribute the flight data change message that deletes the abnormal message to the corresponding message queue according to the preset relationship mapping table;
[0008] Starting corresponding first threads for the message queues respectively, and each of the first threads obtains the flight data change messages from the message queues in parallel for processing;
[0009] Message sending pools are created respectively by multiple second threads corresponding to the message queues, and the flight data change messages processed by the corresponding first threads are sent to the server based on each message sending pool; the start time of the second thread is earlier than the start time of the first thread.
[0010] According to a preferred embodiment of the present invention, the processing method comprises:
[0011] Performing format verification on the flight data change message to determine whether the empty fields and non-empty fields in the flight data change message meet the format requirements; if the empty fields or non-empty fields in the flight data change message do not meet the format requirements, deleting the flight data change message;
[0012] The message type of the flight data change message is obtained, and it is determined whether the message type is a message type that can be processed. If not, the flight data change message is deleted.
[0013] According to a preferred embodiment of the present invention, the processing method further comprises:
[0014] The flight data change message is parsed and converted into a flight data change message in a preset data format; if the conversion fails, the flight data change message is deleted.
[0015] According to a preferred embodiment of the present invention, the flight data change message is subjected to message timing filtering and message duplication filtering to determine abnormal messages, including:
[0016] Using the characteristic data in the flight data change message as the first characteristic data; using the characteristic data of the received flight data change message as the second characteristic data;
[0017] Determining whether there is second feature data that is consistent with the first feature data;
[0018] If there is a second characteristic data that is consistent with the first characteristic data; determining whether the message generation time of the flight data change message corresponding to the first characteristic data is less than the message generation time of the flight data change message corresponding to the second characteristic;
[0019] If the message generation time of the flight data change message corresponding to the first feature data is less than the message generation time of the flight data change message corresponding to the second feature, then the flight data change message corresponding to the first feature data has a timing error and is an abnormal message;
[0020] If the message generation time of the flight data change message corresponding to the first feature data is greater than or equal to the message generation time of the flight data change message corresponding to the second feature, determining whether the message content of the flight data change message corresponding to the first feature data is consistent with the message content of the flight data change message corresponding to the second feature;
[0021] If the message content of the flight data change message corresponding to the first characteristic data is consistent with the message content of the flight data change message corresponding to the second characteristic data, then the flight data change message corresponding to the first characteristic data has a duplication error and is an abnormal message;
[0022] If the message content of the flight data change message corresponding to the first feature data is inconsistent with the message content of the flight data change message corresponding to the second feature data, the flight data change message corresponding to the first feature data is normal;
[0023] If there is no second characteristic data that is consistent with the first characteristic data, the flight data change message corresponding to the first characteristic data is normal.
[0024] According to a preferred embodiment of the present invention, the preset relationship mapping table includes: a first thread table, a second thread table and a message request relationship mapping table;
[0025] The first thread table includes: a message queue name, a first thread ID, a message queue address, and a number of threads;
[0026] The second thread table includes: a message queue name, a second thread ID message queue address, and a thread number;
[0027] The message request relationship mapping table includes: a mapping relationship between the first thread and the second thread and a mapping relationship between different flight data change messages and the message queues.
[0028] According to a preferred embodiment of the present invention, the processing method further comprises:
[0029] When the number of flight data change messages sent by any data source is greater than a first preset number, a mapping relationship between the flight data change messages of the data source and at least one specific message queue is added, and the specific message queue corresponding to the same data source is deployed in the same data processing cluster;
[0030] When the number of flight data change messages sent by any data source is less than a second preset number, the message queues having a mapping relationship with the flight data change messages sent by the data source are deployed in the same data processing cluster.
[0031] According to a preferred embodiment of the present invention, the obtaining of the flight data change message from the message queue for processing includes:
[0032] Processing the flight data change message and receiving the processing result;
[0033] If the processing result is failure, the flight data change message is processed again until the number of times the flight data change message is processed reaches a preset number, and the flight data change message is deleted.
[0034] In order to solve the above technical problems, the second aspect of the present invention proposes a flight data change message forwarding processing system, the processing system comprising:
[0035] A message processing module, used for acquiring flight data change messages, performing message timing filtering and message duplication filtering on the flight data change messages to determine abnormal messages, and deleting the abnormal messages;
[0036] A message distribution module, used for distributing the flight data change message of deleting the abnormal message to the corresponding message queue according to the preset relationship mapping table;
[0037] A message consumption module, used for starting corresponding first threads for the message queues respectively, and each of the first threads obtains flight data change messages from the message queues in parallel for processing;
[0038] The message production module is used to create message sending pools through multiple second threads corresponding to the message queue, and send the flight data change messages processed by the corresponding first threads to the server based on each message sending pool; the start time of the second thread is earlier than the start time of the first thread.
[0039] In order to solve the above technical problem, the third aspect of the present invention provides an electronic device, comprising:
[0040] Processor; and
[0041] A memory storing computer executable instructions, wherein when the computer executable instructions are executed, the processor executes the method described in any one of the above embodiments.
[0042] In order to solve the above technical problems, the fourth aspect of the present invention proposes a computer storage medium, wherein the computer storage medium stores one or more programs, and when the one or more programs are executed by a processor, the method described in any one of the above embodiments is implemented.
[0043] Compared with the prior art, the present invention has the following advantages: when receiving a flight data change message, the present invention removes abnormal messages in the flight data change message, and then distributes the verified flight data change message to the corresponding message queue through a preset relationship mapping table, starts the corresponding first thread for each message queue, obtains the flight data change message from the message queue based on each first thread for processing, and then creates a message sending pool through the second thread corresponding to the message queue, and sends the flight data change message to the server. Each message has its own processor and process chain, which realizes the decoupling between messages and will not affect each other, improves the message processing speed and sending efficiency, and avoids message accumulation. This solution provides an efficient and reliable flight change data processing system for airlines and airports, so that airlines or airports can synchronize flight change data to the departure system in a timely manner when changing flight data.
[0044] Other features and advantages of the present disclosure will be described in the following description, and partly become apparent from the description, or be understood by implementing the present disclosure. The purpose and other advantages of the present disclosure can be realized and obtained by the structures pointed out in the description, claims and drawings. BRIEF DESCRIPTION OF THE DRAWINGS
[0045] In order to more clearly illustrate the embodiments of the present disclosure or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.
[0046] Figure 1 A first flow chart of a method for forwarding a flight data change message according to an embodiment of the present disclosure is shown;
[0047] Figure 2 A second flowchart of a method for forwarding a flight data change message according to an embodiment of the present disclosure is shown;
[0048] Figure 3 A schematic diagram of a message queue link diagram process according to an embodiment of the present disclosure is shown;
[0049] Figure 4 A third flowchart of a method for forwarding a flight data change message according to an embodiment of the present disclosure is shown;
[0050] Figure 5 A schematic diagram of a message timing filtering and repeated filtering process according to an embodiment of the present disclosure is shown;
[0051] Figure 6A block diagram of a flight data change message forwarding processing system according to an embodiment of the present disclosure is shown;
[0052] Figure 7 A schematic diagram of the structure of an electronic device according to an embodiment of the present disclosure is shown. DETAILED DESCRIPTION
[0053] In order to make the purpose, technical solution and advantages of the embodiments of the present disclosure clearer, the technical solution in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are part of the embodiments of the present disclosure, not all of the embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present disclosure.
[0054] The same reference numerals in the drawings represent the same or similar elements, components or parts, and thus the repeated description of the same or similar elements, components or parts may be omitted below. It should also be understood that although the first, second, third and other attributives representing the numbers may be used herein to describe various devices, elements, components or parts, these devices, elements, components or parts should not be limited by these attributives. In other words, these attributives are only used to distinguish one from another. For example, the first device may also be called the second device, but it does not deviate from the essential technical solution of the present invention. In addition, the terms "and / or" and "and / or" refer to all combinations including any one or more of the listed items.
[0055] See also Figure 1 , Figure 1 FIG. 1 is a flow chart of a method for forwarding a flight data change message provided by the present invention. Figure 1 As shown, the method includes:
[0056] S11. Obtain flight data change messages, perform message timing filtering and message duplication filtering on the flight data change messages to determine abnormal messages, and delete the abnormal messages.
[0057] In this embodiment, the flight data change message is a standardized message format used to describe flight information changes, which usually includes basic information of the flight and the specific content of the change.
[0058] In this embodiment, message timing filtering: ensure that the backend subsystem obtains the latest message. For example, a user sends flight number CA123 at 10:31:01 to change the boarding gate to A2, and then sends flight number CA123 at 10:31:10 to change the boarding gate to A3; but due to unknown reasons, such as network delay, the flight number CA123 at 10:31:10 is first received to change the boarding gate to A3, and then the flight number CA123 at 10:31:01 is received to change the boarding gate to A2. The backend subsystem of the departure system needs the latest flight status information, so the message at 10:31:01 can be determined as an abnormal message and discarded.
[0059] In this embodiment, message duplication filtering is performed to ensure that the backend subsystem does not receive duplicate messages; for example, a user sends a message at 11:31:01 to change the boarding gate to A3 with flight number CA123. Due to unknown reasons, the customer sends multiple identical messages in succession. For example, the airline or airport staff sends multiple messages in succession, while the backend subsystem of the departure system only needs one message. The redundant duplicate messages can be determined as abnormal messages and discarded.
[0060] S12. Distribute the flight data change message that deletes the abnormal message to the corresponding message queue according to the preset relationship mapping table.
[0061] In this embodiment, a message queue is a mechanism for transmitting messages between different programs or processes in a computer system. It allows producers (the party that sends messages) and consumers (the party that receives messages) to communicate in an asynchronous manner, thereby decoupling their execution and dependencies. Message queues are widely used in various distributed systems to improve the scalability, reliability and flexibility of the system.
[0062] In this embodiment, flight data change messages are distributed to different message queues through a preset relationship mapping table. When the number of flight data change messages sent by any data source is greater than a first preset number, a mapping relationship between the flight data change message of the data source and at least one specific message queue is added, and the specific message queues corresponding to the same data source are deployed in the same data processing cluster; for example, the preset relationship mapping table can bind the airline and the message queue, that is, the flight data change messages belonging to the same airline are distributed to the same message queue, so as to decouple the messages of different airlines. Through such a solution, when an abnormality occurs in the message of any airline, it will only affect the processing progress of the flight data change message in one message queue without affecting other message queues, and the specific message queues of the same data source are deployed in the same data processing cluster, thereby achieving decoupling at the message queue level, reducing the impact range of the failure, and improving resource utilization.
[0063] In this embodiment, the producers, consumers and message queues can also be bound in the preset relationship mapping table. After any flight data change message is obtained, it is processed by the set producers, consumers and message queues, thereby ensuring that each message has its own processor and process chain, realizing decoupling between messages and preventing them from affecting each other.
[0064] In this embodiment, when the number of flight data change messages sent by any data source is less than the second preset number, the message queue that has a mapping relationship with the flight data change messages sent by the data source is deployed in the same data processing cluster. Specifically, the preset relationship mapping table can also bind different airlines to the same message queue according to the number of flight data change messages sent by the airline. For example, if a certain airline has fewer flights, then the flight data change messages it sends are also relatively small. At this time, several airlines with fewer flight data change messages can be bound to the same message queue, so that the flight data change messages of these airlines are distributed to this corresponding message queue, and the same message queue is used to process the flight data change messages of several different airlines. As above, the message queue that has a mapping relationship with the flight data change messages sent by the data source is deployed in the same data processing cluster, which can also reduce the impact range of the failure and improve resource utilization.
[0065] S13. Start corresponding first threads for each message queue, and each first thread obtains flight data change messages from the message queue in parallel for processing.
[0066] In this embodiment, a corresponding first thread is started for each message queue, that is, the consumption thread of the above-mentioned consumer. The flight data change messages in the message queue are processed in parallel by the threads corresponding to each message queue, so that each message queue is decoupled. When an exception occurs in any message queue, it will not affect the processing of other message queues, thereby improving data processing efficiency and facilitating the staff to handle exceptions in the message queue in a timely manner.
[0067] In this embodiment, one message queue may correspond to multiple first threads, and the same message queue may be processed by multiple first threads to further improve data processing efficiency. At the same time, one message queue may have multiple first threads. When any flight data change message blocks the first thread, other first threads may continue to process the flight data change messages in the message queue to avoid blocking of the message queue.
[0068] In this embodiment, when processing the flight data change message, the processing result needs to be fed back. If the result is failure, the message will be consumed again, up to 3 times. Specifically, the flight data change message is processed and the processing result is received; if the processing result is failure, the flight data change message is processed again until the number of times the flight data change message is processed reaches the preset number, and the flight data change message is deleted.
[0069] S14. Create message sending pools respectively through multiple second threads corresponding to the message queues, and send the flight data change messages processed by the corresponding first threads to the server based on each message sending pool; the start time of the second thread is earlier than the start time of the first thread.
[0070] In this embodiment, after the flight data change message is processed, multiple second threads corresponding to the message queue respectively create message sending pools to send the flight data change message to the server, that is, to the background subsystem of the departure system. The second thread is the production thread of the above-mentioned producer. This solution adopts the strategy of establishing a producer pool, and realizes parallel sending of messages by producers through pooling operations, thereby effectively improving the message processing capability.
[0071] In this embodiment, the concept of pooling is introduced during the producer startup process. The program will start multiple producers, each producer will start multiple instances, and each producer will create a pool to store instance objects. When the producer needs to send a message, you only need to "borrow" an instance from the pool and "return" it to the pool after use. The program uses LinkedList as a pool because LinkedList is a linked list structure with dynamic expansion capabilities, low query performance, high insertion and deletion performance, high performance in accessing and deleting elements at any position, and thread safety. When inserting and deleting, you only need to operate the pointer, and the efficiency will be very high.
[0072] In this embodiment, the scheme is set to start the producer first. This is because if the consumer is started first, the program will consume the message immediately, but the producer has not been started at this time and cannot be sent to the corresponding message queue, which will cause the consumed messages to end processing because the producer cannot be found, and eventually cause the message to be lost. Finally, according to the mapping relationship configuration, the consumer and producer will be put into the Map, and the corresponding producer can be matched through the consumer.
[0073] In this embodiment, the preset relationship mapping table includes: a first thread table, a second thread table and a message request relationship mapping table; the first thread table includes: a message queue name, a first thread ID, a message queue address and a number of threads; the second thread table includes: a message queue name, a second thread ID message queue address and a number of threads; the message request relationship mapping table includes: a mapping relationship between the first thread and the second thread and a mapping relationship between different flight data change messages and message queues.
[0074] Specifically, the first thread table is the consumer table, and the second thread table is the producer table. The relationship table can be split according to the two dimensions of airlines and airports. The consumer table and the producer table mainly manage the message queue configuration, including the message queue name, consumer or producer ID, environment ID, queue address, number of threads, key and other information; the message request relationship mapping table is used to maintain the mapping relationship between consumers and producers, and to determine which message queue the currently consumed message needs to be sent to through the mapping relationship. When the project starts, the consumer configuration table and the producer configuration table will be queried, the configuration information will be read, the consumer and producer objects will be constructed, and the API provided by the message queue will be called to start the consumer and producer. When the status code 200 is returned, it means that the startup is successful.
[0075] When receiving a flight data change message, the present invention removes abnormal messages in the flight data change message, and then distributes the verified flight data change message to the corresponding message queue through a preset relationship mapping table, starts a corresponding first thread for each message queue, obtains the flight data change message from the message queue based on each first thread for processing, and then creates a message sending pool through the second thread corresponding to the message queue, and sends the flight data change message to the server. Each message has its own processor and process chain, which realizes decoupling between messages and does not affect each other, improves the message processing speed and sending efficiency, and avoids message accumulation.
[0076] See also Figure 2 , Figure 2 FIG. 2 is a flow chart of a method for forwarding a flight data change message provided by the present invention. Figure 2 As shown, the method includes:
[0077] S21. Perform format verification on the flight data change message to determine whether the empty fields and non-empty fields in the flight data change message meet the format requirements; if the empty fields or non-empty fields in the flight data change message do not meet the format requirements, delete the flight data change message.
[0078] In this embodiment, the legitimacy of the flight data change message is verified, and non-compliant flight data change messages are filtered out, thereby improving the efficiency and effectiveness of flight data change message processing and reducing the pressure on the background software. It mainly includes security verification and format verification. Verify whether the format and content of the header field meet the requirements.
[0079] Specifically, for format validation, custom annotations are used in combination with aspects. First, two enumeration classes are created, one to define the fields that require non-empty validation by default, and the other to enumerate the fields that require regular validation and the regular expressions for validation. For example, if the flight number field needs to be non-empty validated, CURRENT_FLIGHT_NO ("FlightNo", true) is defined, indicating that the FlightNo field requires non-empty validation by default. ID ("ID", "XXX") indicates that ID needs to be regularly validated, and "XXX" is the corresponding regular expression. At the same time, an annotation is defined, which contains a collection of 3 enumeration types: notNulls, nullables, and patternHeaderEnums. notNulls indicates the fields that need to be checked for non-nullness in addition to the default non-null fields; nullables indicates that some fields in the default non-null fields are allowed to be empty. This is because the non-null fields of most messages are the same, but a few are different, so the default non-null fields are defined for most messages. For a few messages, nullables can be used to exclude the fields that are allowed to be empty in the default non-null fields; patternHeaderEnums indicates which fields in addition to the default need to be checked for regular expressions. If the defined fields are in the default fields, the regular expressions in the default fields will be overwritten. By using the aspect to cut into the annotation, you can achieve complete format verification, and this logic will not affect other functions.
[0080] S22. Obtain the message type of the flight data change message, and determine whether the message type is a message type that can be processed. If not, delete the flight data change message.
[0081] In this embodiment, security verification is implemented in combination with a message processor. Each type of message has a processor. If the current message type does not match the processor, it means that the current message is not within the supported range and the current message is discarded.
[0082] In this embodiment, the scheme also needs to parse and convert the flight data change message, and if there is a problem, the current message is discarded. Specifically, the flight data change message is parsed and converted into a flight data change message in a preset data format. If the conversion fails, the flight data change message is deleted.
[0083] Through the solution of the above embodiment, airlines or airport users only need to pay attention to whether the sent message is a dynamic message or a full message, without considering the service type, thereby improving the customer experience. Figure 3 As shown in the message queue link diagram, the flight data change messages sent by the airline and airport system include dynamic messages and full messages. The dynamic messages can be distributed to message queue 1, and the full messages can be distributed to message queue 2. After the flight data change messages in the message queues are subjected to security verification, format verification, parsing and conversion, message timing filtering, and message duplicate filtering, the flight data change messages in message queues 1 and message queues 2 are processed by the producers and consumers in the above-mentioned embodiments, and then sent to the background subsystem of the departure system.
[0084] See also Figure 4 , Figure 4 The third flowchart of the method for forwarding and processing flight data change messages provided by the present invention is as follows: Figure 4 As shown, the method includes:
[0085] S31. Using characteristic data in the flight data change message as first characteristic data; using characteristic data in the received flight data change message as second characteristic data.
[0086] In this embodiment, the received flight data change message is a message in the message queue that has not been processed and sent to the background subsystem of the departure system or a message to be distributed that has not been distributed to the message queue. For the message that has been processed or sent to the background subsystem of the departure system, the message is ready to be released or is being released. At this time, there is no need to perform time sequence filtering or repeated filtering, and only the newly received flight safety data change message needs to be processed.
[0087] In this embodiment, the data categories of the first characteristic data and the second characteristic data are consistent, and the characteristic data may be the flight number, flight date, message type, data source, message generation time, etc. of the flight data.
[0088] S32: Determine whether there is second feature data that is consistent with the first feature data.
[0089] In this embodiment, when there is second characteristic data that is consistent with the first characteristic data, it means that there is a flight data change message with consistent characteristics. In order to avoid duplicate messages or messages with disordered timing, it is necessary to compare the flight data change messages corresponding to the first characteristic data and the second characteristic data.
[0090] S33a. If there is second characteristic data that is consistent with the first characteristic data, determine whether the message generation time of the flight data change message corresponding to the first characteristic data is less than the message generation time of the flight data change message corresponding to the second characteristic data.
[0091] S34a. If the message generation time of the flight data change message corresponding to the first characteristic data is less than the message generation time of the flight data change message corresponding to the second characteristic data, the flight data change message corresponding to the first characteristic data has a timing error and is an abnormal message.
[0092] In this embodiment, the message generation time of the two flight data change messages is compared. If the currently received flight data change message is earlier than the received flight data change message, then there is a timing error at this time. The currently received flight data change message is directly treated as an abnormal message and deleted later.
[0093] S34b. If the message generation time of the flight data change message corresponding to the first characteristic data is greater than or equal to the message generation time of the flight data change message corresponding to the second characteristic, determine whether the message content of the flight data change message corresponding to the first characteristic data is consistent with the message content of the flight data change message corresponding to the second characteristic.
[0094] In this embodiment, if the currently received flight data change message is later than the received flight data change message, then there is a timing error. The message contents are further compared for consistency. If the messages are consistent, it indicates that duplicate messages have occurred.
[0095] S35a. If the message content of the flight data change message corresponding to the first characteristic data is consistent with the message content of the flight data change message corresponding to the second characteristic data, a duplication error occurs in the flight data change message corresponding to the first characteristic data, and the message is an abnormal message.
[0096] S35b. If the message content of the flight data change message corresponding to the first characteristic data is inconsistent with the message content of the flight data change message corresponding to the second characteristic data, the flight data change message corresponding to the first characteristic data is normal.
[0097] In this embodiment, when the message contents are consistent, the currently received flight data change message is regarded as an abnormal message and is subsequently directly deleted.
[0098] S33b. If there is no second characteristic data that is consistent with the first characteristic data, the flight data change message corresponding to the first characteristic data is normal.
[0099] In this embodiment, if the second characteristic data that is consistent with the first characteristic data does not exist, the currently received flight data change message can be processed normally.
[0100] like Figure 5 As shown, in this embodiment, the schematic diagram of the message timing filtering and message duplication filtering process is implemented through a distributed lock, with the flight number + flight date + message type + data source as the key and the message body as the content. Each time a new message is received, the message content will be parsed to extract the flight number, flight date, message type, data source and message generation time, and then the key composed of the current message flight number, flight date, message type and data source will be used to query whether the key exists. If it exists, the current message generation time is compared to see if it is less than the existing message generation time. If it is less, the current message has a timing error and is discarded. Otherwise, the message content is compared. If they are the same, the current message has a message duplication error and the current message is discarded. Otherwise, the message is forwarded to the background subsystem and the message is updated. If the key does not exist, the current message is forwarded and stored. Ensure that the background subsystem obtains the latest and non-duplicate messages. After the processor processes each message, it needs to feedback the processing result. If the feedback result is a failure, the message will be consumed again, up to 3 times.
[0101] See also Figure 6 , Figure 6 This is a block diagram of a flight data change message forwarding processing system provided by the present invention. Figure 6 As shown, the processing system includes: a message processing module 11, a message distribution module 12, a message consumption module 13 and a message production module 14.
[0102] In this embodiment, the message processing module 11 is used to obtain flight data change messages, perform message timing filtering and message duplication filtering on the flight data change messages to determine abnormal messages, and delete the abnormal messages.
[0103] In this embodiment, the message distribution module 12 is used to distribute the flight data change message of deleting the abnormal message to the corresponding message queue according to the preset relationship mapping table.
[0104] In this embodiment, the message consumption module 13 is used to start corresponding first threads for message queues respectively, and each first thread obtains flight data change messages from the message queues in parallel for processing.
[0105] In this embodiment, the message production module 14 is used to create message sending pools through multiple second threads corresponding to the message queues, and send the processed flight data change messages processed by the corresponding first threads to the server based on each message sending pool; the start time of the second thread is earlier than the start time of the first thread.
[0106] In this embodiment, the processing system also includes: a data verification module, which is used to perform format verification on the flight data change message to determine whether the empty fields and non-empty fields in the flight data change message meet the format requirements; if the empty fields or non-empty fields in the flight data change message do not meet the format requirements, the flight data change message is deleted; the message type of the flight data change message is obtained to determine whether the message type is a message type that can be processed, and if not, the flight data change message is deleted.
[0107] In this embodiment, the message processing module 11 is further used to parse the flight data change message and convert it into a flight data change message in a preset data format. If the conversion fails, the flight data change message is deleted.
[0108] In this embodiment, the message processing module 11 is specifically used to use the characteristic data in the flight data change message as the first characteristic data; use the characteristic data of the received flight data change message as the second characteristic data; determine whether there is a second characteristic data consistent with the first characteristic data; if there is a second characteristic data consistent with the first characteristic data; then determine whether the message generation time of the flight data change message corresponding to the first characteristic data is less than the message generation time of the flight data change message corresponding to the second characteristic; if the message generation time of the flight data change message corresponding to the first characteristic data is less than the message generation time of the flight data change message corresponding to the second characteristic, then the flight data change message corresponding to the first characteristic data has a timing error and is an abnormal message; if the message generation time of the flight data change message corresponding to the first characteristic data is less than the message generation time of the flight data change message corresponding to the second characteristic, then the flight data change message corresponding to the first characteristic data has a timing error and is an abnormal message; If the generation time is greater than or equal to the message generation time of the flight data change message corresponding to the second feature, it is determined whether the message content of the flight data change message corresponding to the first feature data is consistent with the message content of the flight data change message corresponding to the second feature; if the message content of the flight data change message corresponding to the first feature data is consistent with the message content of the flight data change message corresponding to the second feature, then the flight data change message corresponding to the first feature data has a duplication error and is an abnormal message; if the message content of the flight data change message corresponding to the first feature data is inconsistent with the message content of the flight data change message corresponding to the second feature, the flight data change message corresponding to the first feature data is normal; if there is no second feature data consistent with the first feature data, the flight data change message corresponding to the first feature data is normal.
[0109] In this embodiment, the preset relationship mapping table includes: a first thread table, a second thread table and a message request relationship mapping table; the first thread table includes: a message queue name, a first thread ID, a message queue address and a number of threads; the second thread table includes: a message queue name, a second thread ID message queue address and a number of threads; the message request relationship mapping table includes: a mapping relationship between the first thread and the second thread and a mapping relationship between different flight data change messages and message queues.
[0110] In this embodiment, the processing system also includes: a message queue link management module, which is used to add a mapping relationship between the flight data change messages of the data source and at least one specific message queue when the number of flight data change messages sent by any data source is greater than a first preset number, and deploy the specific message queues corresponding to the same data source in the same data processing cluster; when the number of flight data change messages sent by any data source is less than a second preset number, the message queues that have a mapping relationship with the flight data change messages sent by the data source are deployed in the same data processing cluster.
[0111] In this embodiment, the message processing module 11 is also used to process the flight data change message and receive the processing result; if the processing result is failure, the flight data change message is processed again until the number of times the flight data change message is processed reaches a preset number, and the flight data change message is deleted.
[0112] like Figure 7 As shown, an embodiment of the present invention provides an electronic device, including a processor 1110, a communication interface 1120, a memory 1130 and a communication bus 1140, wherein the processor 1110, the communication interface 1120, and the memory 1130 communicate with each other through the communication bus 1140;
[0113] Memory 1130, used for storing computer programs;
[0114] The processor 1110 is used to implement any of the above processing methods when executing the program stored in the memory 1130.
[0115] In the electronic device provided by the embodiment of the present invention, the processor 1110 obtains the flight data change message by executing the program stored in the memory 1130, performs message timing filtering and message duplication filtering on the flight data change message to determine the abnormal message, and deletes the abnormal message; distributes the flight data change message to the corresponding message queue according to the preset relationship mapping table; starts the corresponding first thread for the message queue respectively, and each first thread obtains the flight data change message from the message queue in parallel for processing; creates message sending pools respectively through multiple second threads corresponding to the message queue, and sends the corresponding processed flight data change message to the server based on each message sending pool; the start time of the second thread is earlier than the start time of the first thread.
[0116] The communication bus 1140 mentioned in the above electronic device can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The communication bus 1140 can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus.
[0117] The communication interface 1120 is used for communication between the above electronic device and other devices.
[0118] The memory 1130 may include a random access memory 1130 (RAM), or may include a non-volatile memory 1130 (non-volatile memory), such as at least one disk memory 1130. Optionally, the memory 1130 may also be at least one storage device located away from the processor 1110.
[0119] The above-mentioned processor 1110 can be a general-purpose processor 1110, including a central processing unit 1110 (CPU), a network processor 1110 (NP), etc.; it can also be a digital signal processor 1110 (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components.
[0120] An embodiment of the present invention provides a computer-readable storage medium, which stores one or more programs. The one or more programs can be executed by one or more processors 1110 to implement the processing method of any of the above embodiments.
[0121] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function according to the embodiment of the present invention is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website site, a computer, a server or a data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server or data center. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or a data center that includes one or more available media integrated. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state hard disk SolidStateDisk (SSD)), etc.
[0122] Although the present disclosure has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present disclosure.
Claims
1. A method for forwarding and processing flight data change messages, characterized in that: The processing method comprises: Acquire flight data change messages, perform message timing filtering and message duplication filtering on the flight data change messages to determine abnormal messages, and delete the abnormal messages; Distribute the flight data change message that deletes the abnormal message to the corresponding message queue according to the preset relationship mapping table; Starting corresponding first threads for the message queues respectively, and each of the first threads obtains the flight data change messages from the message queues in parallel for processing; Message sending pools are created respectively by multiple second threads corresponding to the message queues, and the flight data change messages processed by the corresponding first threads are sent to the server based on each message sending pool; the start time of the second thread is earlier than the start time of the first thread.
2. The processing method according to claim 1, characterized in that: The processing method comprises: Performing format verification on the flight data change message to determine whether the empty fields and non-empty fields in the flight data change message meet the format requirements; if the empty fields or non-empty fields in the flight data change message do not meet the format requirements, deleting the flight data change message; The message type of the flight data change message is obtained, and it is determined whether the message type is a message type that can be processed. If not, the flight data change message is deleted.
3. The processing method according to claim 1, characterized in that: The processing method also includes: The flight data change message is parsed and converted into a flight data change message in a preset data format; if the conversion fails, the flight data change message is deleted.
4. The processing method according to claim 1, characterized in that: Performing message timing filtering and message duplication filtering on the flight data change message to determine abnormal messages includes: Using the characteristic data in the flight data change message as the first characteristic data; using the characteristic data of the received flight data change message as the second characteristic data; Determining whether there is second feature data that is consistent with the first feature data; If there is a second characteristic data that is consistent with the first characteristic data; determining whether the message generation time of the flight data change message corresponding to the first characteristic data is less than the message generation time of the flight data change message corresponding to the second characteristic; If the message generation time of the flight data change message corresponding to the first feature data is less than the message generation time of the flight data change message corresponding to the second feature, then the flight data change message corresponding to the first feature data has a timing error and is an abnormal message; If the message generation time of the flight data change message corresponding to the first feature data is greater than or equal to the message generation time of the flight data change message corresponding to the second feature, determining whether the message content of the flight data change message corresponding to the first feature data is consistent with the message content of the flight data change message corresponding to the second feature; If the message content of the flight data change message corresponding to the first characteristic data is consistent with the message content of the flight data change message corresponding to the second characteristic data, then the flight data change message corresponding to the first characteristic data has a duplication error and is an abnormal message; If the message content of the flight data change message corresponding to the first feature data is inconsistent with the message content of the flight data change message corresponding to the second feature data, the flight data change message corresponding to the first feature data is normal; If there is no second characteristic data consistent with the first characteristic data, the flight data change message corresponding to the first characteristic data is normal.
5. The processing method according to claim 1, characterized in that: The preset relationship mapping table includes: a first thread table, a second thread table and a message request relationship mapping table; The first thread table includes: a message queue name, a first thread ID, a message queue address, and a number of threads; The second thread table includes: a message queue name, a second thread ID message queue address, and a thread number; The message request relationship mapping table includes: a mapping relationship between the first thread and the second thread and a mapping relationship between different flight data change messages and the message queues.
6. The processing method according to claim 5, characterized in that: The processing method also includes: When the number of flight data change messages sent by any data source is greater than a first preset number, a mapping relationship between the flight data change messages of the data source and at least one specific message queue is added, and the specific message queue corresponding to the same data source is deployed in the same data processing cluster; When the number of flight data change messages sent by any data source is less than a second preset number, the message queues having a mapping relationship with the flight data change messages sent by the data source are deployed in the same data processing cluster.
7. The processing method according to any one of claims 1 to 6, characterized in that: The obtaining the flight data change message from the message queue for processing includes: Processing the flight data change message and receiving the processing result; If the processing result is failure, the flight data change message is processed again until the number of times the flight data change message is processed reaches a preset number, and the flight data change message is deleted.
8. A flight data change message forwarding and processing system, characterized in that: The processing system comprises: A message processing module, used for acquiring flight data change messages, performing message timing filtering and message duplication filtering on the flight data change messages to determine abnormal messages, and deleting the abnormal messages; A message distribution module, used for distributing the flight data change message of deleting the abnormal message to the corresponding message queue according to the preset relationship mapping table; A message consumption module, used for starting corresponding first threads for the message queues respectively, and each of the first threads obtains flight data change messages from the message queues in parallel for processing; A message production module is used to create message sending pools through multiple second threads corresponding to the message queue, and send the flight data change messages processed by the corresponding first threads to the server based on each message sending pool; the start time of the second thread is earlier than the start time of the first thread.
9. An electronic device, characterized in that: include: processor; as well as A memory storing computer executable instructions which, when executed, cause the processor to perform the method according to any one of claims 1-7.
10. A computer storage medium, characterized in that: in, The computer storage medium stores one or more programs, and when the one or more programs are executed by a processor, the method of any one of claims 1 to 7 is implemented.