Civil aviation big language model-based reply information acquisition method

Abstract

The invention provides a reply information acquisition method based on a civil aviation big language model. The reply information acquisition method comprises the following steps: acquiring to-be-replied information input by a user; if the target intention is a civil aviation knowledge type intention, determining a target knowledge type intention from a preset knowledge type intention tree; if the target knowledge type intention carries the target identifier, calling a real-time information acquisition port corresponding to the current target knowledge type intention according to account information of the user to acquire target real-time information corresponding to the user; and splicing the to-be-replied information, the target real-time information and the target civil aviation knowledge information, and inputting the spliced information into the civil aviation large language model to obtain reply information output by the civil aviation large language model. According to the reply information acquisition method based on the civil aviation large language model, when the determined target knowledge type intention carries the target identifier, the target real-time information is queried according to the real-time information acquisition port, and the target knowledge type intention is determined to carry the target identifier. Therefore, the civil aviation large language model can further accurately determine the real intention of the user according to the target real-time information.

Description

Background Art

[0002] Due to the different language expression habits of different users, many users will encounter problems such as "missing subject" and "ambiguous expression" when asking questions. These problems are generally minor in general knowledge question-and-answer scenarios. However, after the staff studied historical data and conducted necessary user surveys, they learned that in many scenarios in the civil aviation field, major problems will arise, making the civil aviation large language model's answers not meet user expectations. For example, when a user inputs "how to change a flight ticket", the user's intention may be a civil aviation knowledge intention, that is, the user wants to understand the ticket change process and related regulations. However, a considerable number of users' true intentions are civil aviation business intentions, that is, users want to change the tickets they have purchased. It can be seen that when there is a deviation in user intent recognition, there is also a large deviation in the content of the answer. Summary of the invention

[0003] In view of this, the present application provides a method for obtaining reply information based on a large civil aviation language model, which at least partially solves the problems existing in the prior art.

[0004] In one aspect of an embodiment of the present application, a method for obtaining reply information based on a civil aviation large language model is provided, comprising the following steps:

[0005] Get the information to be replied to entered by the user;

[0006] If the target intent corresponding to the information to be replied is a civil aviation knowledge intent, the target knowledge intent is determined from a preset knowledge intent tree; the target knowledge intent is any leaf node in the knowledge intent tree; at least some of the leaf nodes in the knowledge intent tree are provided with a target identifier; the leaf node provided with a target identifier has a corresponding real-time information acquisition port; if the leaf node is provided with a target identifier, it indicates that the intent corresponding to the information to be replied has a probability of being a civil aviation business intent;

[0007] If the target knowledge class intent carries a target identifier, the real-time information acquisition port corresponding to the current target knowledge class intent is called according to the user's account information to obtain the target real-time information corresponding to the user; and the target knowledge class intent and the information to be replied are input into the preset civil aviation knowledge database query port to obtain the target civil aviation knowledge information returned by the civil aviation knowledge database query port;

[0008] The information to be replied, the target real-time information and the target civil aviation knowledge information are spliced ​​and input into the civil aviation large language model to obtain reply information output by the civil aviation large language model.

[0009] Beneficial effects:

[0010] An embodiment of the present application provides a method for obtaining reply information based on a civil aviation large language model. By setting a target identifier on a specific leaf node in a civil aviation knowledge class intent tree, when a determined target knowledge class intent carries a target identifier, the method does not directly reply with the civil aviation knowledge class intent, but instead queries the user's related target real-time information according to the real-time information acquisition port corresponding to the target knowledge class intent, thereby enabling the civil aviation large language model to further accurately determine the user's true intention based on the target real-time information. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0012] Figure 1 A flowchart of a method for obtaining reply information based on a civil aviation large language model provided in an embodiment of the present application. DETAILED DESCRIPTION

[0013] The embodiments of the present application are described in detail below with reference to the accompanying drawings.

[0014] It should be noted that the following embodiments and features in the embodiments may be combined with each other in the absence of conflict; and, based on the embodiments in the present disclosure, all other embodiments obtained by ordinary technicians in the field without making any creative work are within the scope of protection of the present disclosure.

[0015] It should be noted that various aspects of the embodiments within the scope of the appended claims are described below. It should be apparent that the aspects described herein may be embodied in a wide variety of forms, and any specific structure and / or function described herein is merely illustrative. Based on the present disclosure, it should be understood by those skilled in the art that an aspect described herein may be implemented independently of any other aspect, and two or more of these aspects may be combined in various ways. For example, any number of aspects described herein may be used to implement the device and / or practice the method. In addition, other structures and / or functionalities other than one or more of the aspects described herein may be used to implement this device and / or practice this method.

[0016] Glossary:

[0017] Root node: The top node of the tree. It has no parent node and is the starting node of the tree.

[0018] Leaf node: A node without child nodes is called a leaf node, also called a terminal node.

[0019] Parent node: The immediate superior node of a node is called its parent node.

[0020] Child node: The direct subordinate node of a node is called its child node.

[0021] Please refer to Figure 1 As shown, a method for obtaining reply information based on a civil aviation large language model provided in an embodiment of the present application specifically includes the following steps:

[0022] S100, obtaining information to be replied input by a user. Specifically, the user may input the information to be replied through a preset interactive interface, and the information to be replied may include at least one of the following information forms: text, picture, video, and audio.

[0023] S200, if the target intent corresponding to the information to be replied is a civil aviation knowledge intent, the target knowledge intent is determined from a preset knowledge intent tree. The target knowledge intent is any leaf node in the knowledge intent tree, and at least some of the leaf nodes in the knowledge intent tree are provided with a target identifier; the leaf node provided with a target identifier has a corresponding real-time information acquisition port. If the leaf node is provided with a target identifier, it means that the intent corresponding to the information to be replied is likely to be a civil aviation business intent.

[0024] It is worth noting that, in this embodiment, the sub-node described here can be understood as any node other than the root node in the knowledge-based intent tree.

[0025] S300, if the target knowledge class intent carries a target identifier, then the real-time information acquisition port corresponding to the current target knowledge class intent is called according to the user's account information to obtain the target real-time information corresponding to the user; and the target knowledge class intent and the information to be replied are input into the preset civil aviation knowledge database query port to obtain the target civil aviation knowledge information returned by the civil aviation knowledge database query port. Specifically, the civil aviation knowledge database query port corresponds to a preset civil aviation knowledge database. Specifically, the civil aviation knowledge database can be a graph database, a table database, a vector database, etc. Or it can be composed of multiple databases of the same type or at least partially different types.

[0026] S400, splicing the information to be replied, the target real-time information and the target civil aviation knowledge information and inputting them into a civil aviation large language model to obtain reply information output by the civil aviation large language model.

[0027] In some exemplary embodiments of the present application, the reply information includes knowledge text and / or a page jump hyperlink; the page jump hyperlink is used to jump to the corresponding civil aviation business processing page.

[0028] In this embodiment, by setting a target identifier on a specific leaf node in the civil aviation knowledge class intent tree, when the determined target knowledge class intent carries a target identifier, it does not directly reply with the civil aviation knowledge class intent, but instead queries the user's related target real-time information based on the real-time information acquisition port corresponding to the target knowledge class intent, thereby enabling the civil aviation large language model to further accurately determine the user's true intention based on the target real-time information.

[0029] Taking the above scenario as an example, the user inputs "how to change the ticket", and the knowledge intent located in the civil aviation knowledge intent tree may be "change process regulations". In this embodiment, the "change process regulations" are set with a target identifier, and the real-time information acquisition port set for the "change process regulations" is the user itinerary acquisition port. At this time, the user itinerary acquisition port will be called according to the user's account information to obtain the user itinerary information returned by the port. At the same time, the knowledge related to the change process (i.e., the target civil aviation knowledge information) will also be obtained through the civil aviation knowledge database query port.

[0030] After the above information is input into the civil aviation language model, if the user's itinerary information indicates that the user has an unexecuted itinerary, it will be processed according to the civil aviation business intent, and relevant operation instructions and / or a hyperlink to jump to the rebooking interface for the unexecuted itinerary can be output. If the user's itinerary information indicates that the user has no unexecuted itinerary, it will be processed according to the civil aviation knowledge intent, and a reply will be made based on the knowledge related to the rebooking process obtained.

[0031] Therefore, the solution provided by this embodiment, when the user's input information to be replied is ambiguous or corresponds to both civil aviation knowledge intent and civil aviation business intent. According to the real-time information acquisition port corresponding to the knowledge intent with the target identifier in the knowledge intent tree, some background knowledge (such as user itinerary, flight dynamics, user portrait) can be actively acquired to assist the civil aviation large language model in processing, so that the civil aviation large language model can output reply information that better meets the user's needs.

[0032] In some exemplary embodiments of the present application, the civil aviation large language model is configured to give priority to outputting the reply information as a civil aviation business processing task when the input information contains both the target real-time information and the target civil aviation knowledge information.

[0033] In practice, if the user has an unexecuted trip, the questions he raises are most likely about the trip. Therefore, when identifying intent, if the user inputs both civil aviation business intent and civil aviation knowledge intent, he probably wants to perform business processing on it. Therefore, in the embodiment, the civil aviation large language model is configured to output the reply information with priority to civil aviation business processing tasks when the input information contains both the target real-time information and the target civil aviation knowledge information.

[0034] It is worth noting that in this embodiment, civil aviation business processing tasks are different from civil aviation business intentions, and the two can be in a corresponding relationship. For example, if the civil aviation large language model learns that the current user's intention is a civil aviation business intention, it can be executed as a civil aviation business processing task.

[0035] At the same time, in this embodiment, when the input information includes both the target real-time information and the target civil aviation knowledge information, the reply information is not necessarily output as a civil aviation business processing task. Instead, the command of the civil aviation large language model is fine-tuned so that the probability of processing as a civil aviation business processing task is higher than the probability of processing as a civil aviation knowledge output task in this case. The reason is that in actual practice, not all users tend to perform business processing.

[0036] Accordingly, in this embodiment, when the target knowledge class intent carries a target identifier, the target knowledge class intent will not be Figure 1 The target real-time information and target civil aviation knowledge information are input into the civil aviation language model. The purpose is to avoid the input intention (target knowledge type intention) affecting the user intention judgment of the civil aviation language model, so as to give full play to the inherent reasoning ability of the civil aviation language model.

[0037] Of course, in some exemplary embodiments of the present application, the target knowledge class intent and the possible civil aviation business class intent corresponding to the target knowledge class intent can also be simultaneously Figure 1 The target real-time information and target civil aviation knowledge information are input into the civil aviation big language model.

[0038] In some exemplary embodiments of the present application, the reply information includes knowledge text and / or business processing buttons;

[0039] The method further comprises:

[0040] S500 , in response to the user clicking the service processing button, generating a corresponding service processing instruction according to the target real-time information, and sending the service processing instruction to the service processing port corresponding to the service processing button.

[0041] In this embodiment, a corresponding message generation method can be set for each type of civil aviation business, so that when the civil aviation language model analyzes that the user has a business processing demand, a corresponding processing button is directly generated, and the relevant business processing port is directly sent when the user clicks the button. Specifically, the message generation method can adopt a preset template and slot method. For example, in a ticket refund scenario, a preset message template can be set, and the message template includes fixed content in the ticket refund instruction message, and the variable content is set in the form of slots. When the civil aviation language model recognizes that the user wants to refund a certain itinerary, the corresponding slot in the message template is filled according to the user information and itinerary information (such as flight number) to obtain the business processing instruction, and the business processing instruction is sent to the corresponding business processing port (API port). In this way, the complexity of the user's business processing can be saved, and there is no need to jump to other interfaces for processing, and multiple needs can be completed only in the same interface.

[0042] In some exemplary embodiments of the present application, the method further includes:

[0043] S310, if the target knowledge class intent does not carry a target identifier, the target knowledge class intent and the information to be replied to are input into the civil aviation knowledge database query port to obtain the target civil aviation knowledge information returned by the civil aviation knowledge database query port.

[0044] Specifically, the target knowledge class intent and the information to be replied can be concatenated to form a semantic vector, and the obtained semantic vector is input into the civil aviation knowledge database query port, so that the civil aviation knowledge database corresponding to the civil aviation knowledge database query port uses a semantic matching method to determine the target civil aviation knowledge information to be returned.

[0045] S320, splicing the target civil aviation knowledge information, the target knowledge class intent and the information to be replied to, and inputting them into the civil aviation large language model to obtain reply information returned by the civil aviation large language model.

[0046] The target civil aviation knowledge information, the target knowledge class intention and the information to be replied are processed by the civil aviation large language model, so that the civil aviation large language model can further accurately judge the user's purpose according to the information to be replied and the target knowledge class intention, and generate reply information according to the acquired target civil aviation knowledge information and the learned prior knowledge, so as to achieve accurate reply. It is worth noting that in this embodiment, the civil aviation large language model can be restricted to generate reply information based on the target civil aviation knowledge information by means of prompt injection, so as to avoid the occurrence of "hallucination" (i.e., the large language model makes up a reply according to the probability space).

[0047] Specifically, in an exemplary embodiment of the present application, at least some of the child nodes in the knowledge intent tree have corresponding necessary information. The necessary information is information for determining whether the corresponding child node is the target intent. Specifically, the child nodes with corresponding necessary information, why the corresponding necessary information is, can be stored in a preset information table for easy access, or can be directly embedded in the data of the child node in the form of information.

[0048] Determining the target knowledge class intent from the preset knowledge class intent tree includes:

[0049] S201, according to the information to be replied, match the target knowledge class intent from the several knowledge class intents contained in the knowledge class intent tree layer by layer. Specifically, in the matching process, the knowledge class intent tree can be searched based on the existing tree search algorithm to complete the matching of the target knowledge class intent. And in the matching process, if the node currently being matched (which can be any node in the implementation class intent tree) meets the preset matching conditions (for example, the matching degree reaches a preset threshold), it will be determined as the currently matched knowledge class intent.

[0050] S202: If the currently matched knowledge class intent is a leaf node, the currently matched knowledge class intent is determined as the target knowledge class intent. Otherwise, the matching is continued in the child node corresponding to the currently matched knowledge class intent.

[0051] S203: If there is no sub-node meeting the matching condition in the sub-node corresponding to the currently matched knowledge type intent, an information acquisition prompt is generated according to the necessary information corresponding to the sub-node corresponding to the currently matched knowledge type intent.

[0052] S204, outputting the information acquisition prompt to acquire the supplementary information input by the user;

[0053] S205, after splicing the information acquisition prompt, the supplementary information and the information to be replied into new information to be replied, determine the target knowledge class intent from a preset knowledge class intent tree according to the new information to be replied.

[0054] Due to the different language expression habits of different users, many users will have the problem of missing necessary information (i.e., information to be answered) when they ask questions for the first time. Although the civil aviation large language model can respond to questions that lack necessary information, it is often difficult for the response to meet the user's expectations, that is, it cannot solve the user's problem. At this time, the user can only judge which necessary information is missing in the question they entered based on the response content of the civil aviation large language model, and re-edit the question. On the one hand, this method is inefficient, and on the other hand, it is sometimes difficult for users to accurately judge which necessary information they need to supplement, which further reduces the user experience.

[0055] In this embodiment, accurate intent matching will be performed before the information to be replied is input into the civil aviation language model, and when it is no longer possible to match the intent of a lower level (i.e., when there is no sub-node that meets the matching conditions in the sub-node corresponding to the currently matched knowledge intent), an information acquisition prompt is generated based on the necessary information corresponding to the sub-node of the current node, so that the user can directly input supplementary information based on the information acquisition prompt to obtain a more accurate reply. Specifically, the information acquisition prompt can also be generated using the civil aviation language model based on the corresponding necessary information.

[0056] Let's take an example to illustrate. If the information to be replied entered by the user is "How much luggage can I carry when I take a plane?", it will be matched to the civil aviation knowledge intent when the intent is matched for the first time. At this time, the knowledge intent tree will be matched layer by layer. If the knowledge intent corresponding to the node in the current knowledge intent tree is "Baggage allowance regulations", the intents corresponding to its corresponding child nodes are "A airline baggage allowance regulations", "B airline baggage allowance regulations" and "C airline baggage allowance regulations". At the same time, the necessary information corresponding to "A airline baggage allowance regulations", "B airline baggage allowance regulations" and "C airline baggage allowance regulations" is the name of the airline. At this time, since the information to be replied entered by the user does not contain the name of the airline, the name of the airline will be determined as necessary information and the corresponding information acquisition prompt will be generated (for example, since different airlines have different regulations on baggage allowances, which airline's regulations do you want to guide specifically?), so that the user can enter the corresponding supplementary information, so that the civil aviation large language model can make an accurate reply.

[0057] In some existing large language models, even if they are trained with civil aviation expertise, when the user inputs information such as "How much luggage can I take on a plane?", most of them can only give a direct reply. Since they need to give a reply, in order to ensure the accuracy of the reply, all the baggage limit regulations of all airlines are often output. In this way, on the one hand, the processing time will be very long due to the long output text. On the other hand, the user's reading pressure is also very high, and the reply content is not targeted, which makes it difficult to meet user needs.

[0058] In the solution provided in this embodiment, a question-and-answer interaction method can be used. When the user's question lacks necessary information, the user is prompted to supplement it, and a targeted and accurate answer is given based on the supplemented content. In this way, on the one hand, the length of the output text can be reduced, and on the other hand, the user's needs can be better met, the overall number of interactions can be reduced, and the user does not need to repeatedly think about how to write the question.

[0059] In some exemplary embodiments of the present application, the method may be applied to an intelligent system, which includes: a civil aviation large language model, a civil aviation business database query port, and several real-time information acquisition ports.

[0060] Specifically, in the embodiment of the present application, the civil aviation large language model can be a complete large language model, or it can be a large language model group formed by a combination of multiple models. Furthermore, if the civil aviation large language model is a large language model group formed by a combination of multiple models, which can also include other artificial intelligence models other than the large language model (such as machine models, neural network models, etc.), it is only necessary to ensure that the large language model group contains at least one large language model that has been targeted with civil aviation knowledge (which may include pre-training, instruction fine-tuning, reinforcement learning, etc.). In this embodiment, the large language model generally refers to a large language model based on the Transformer architecture. Specifically, the large language model may include both an encoder and a decoder, or only a decoder, or only an encoder.

[0061] The civil aviation business database query port has a corresponding civil aviation data storage database, which can be a graph database, a table database, a vector database, etc. In this embodiment, the civil aviation data storage database is a table database, specifically, an SQL database. It can include flight data of all historical, ongoing or unexecuted flights, user itinerary data, user order data, or itinerary data of other means of transportation, etc.

[0062] Several real-time information acquisition ports can be pre-set API ports, such as flight dynamic acquisition port, user itinerary acquisition port, and user portrait acquisition port. Specifically, these real-time information acquisition ports can directly reuse some API ports of other itinerary management software, systems, and platforms.

[0063] In some exemplary embodiments of the present application, between step S100 and step S200, the method further includes:

[0064] S110, according to the information to be replied, determining the target intention corresponding to the information to be replied from the civil aviation intention list. The civil aviation intention list includes civil aviation knowledge intentions, a plurality of first-category civil aviation business intentions, and a plurality of second-category civil aviation business intentions, wherein the task complexity corresponding to the first-category civil aviation business intentions is higher than the task complexity of the second-category civil aviation business intentions.

[0065] Specifically, the first type of civil aviation business intent is not provided with a corresponding real-time information acquisition port, and each second type of civil aviation business intent is provided with at least one real-time information acquisition port. Figure 1Generally refers to the user's intention to handle civil aviation, such as refunding tickets, changing tickets, etc. Or query or count some real-time, quasi-real-time, high-timeliness or high-frequency data that will change, such as querying the current location and current status of a flight. Civil aviation business intent is mainly different from civil aviation knowledge intent. Civil aviation knowledge intent Figure 1 Generally refers to users who want to query some inherent and short-term civil aviation knowledge, such as the customer service phone number of XX airport, the official website of XX airport, the baggage requirements of a certain flight, etc.

[0066] The step S200 further includes:

[0067] S210, if the target intent is any one of several first-category civil aviation business intents, the civil aviation big language model is used to obtain a database query statement according to the information to be answered. Specifically, in this embodiment, the database query statement may be an SQL statement. The text2sql capability of the civil aviation big prediction model may be specifically trained, so that the civil aviation big language model can generate a database query statement according to the information to be answered.

[0068] S211, sending the database query statement to the civil aviation business database query port to obtain a target data set returned by the civil aviation business database query port.

[0069] S212, concatenating the target data set and the information to be replied to, and inputting the concatenated data set into the civil aviation large language model to obtain reply information returned by the civil aviation large language model.

[0070] S220, if the target intent is any one of several second-category civil aviation business intents, obtain a port information list corresponding to the target intent. The port information list includes port information of at least some of the several real-time information acquisition ports; the port information includes a port function introduction of the corresponding real-time information acquisition port. The port function introduction may include information about which fields can be obtained from the real-time information acquisition port, how to call the port, and other information.

[0071] S221, concatenating the port information list and the information to be replied and inputting them into the civil aviation large language model, so that the civil aviation large language model calls at least one of the real-time information acquisition ports corresponding to the port information list to obtain the target real-time information, and outputs reply information according to the target real-time information and the information to be replied.

[0072] That is, in this embodiment, the port information list is input into the civil aviation large language model as a prompt instruction, so that the civil aviation large language model can select the real-time information acquisition port that can be used this time from several real-time information acquisition ports corresponding to the port information list according to the thinking chain technology.

[0073] Specifically, in this embodiment, the port information list corresponding to each second-category civil aviation business intention can be set by the staff according to their work experience. It can also be determined based on the real-time information acquisition ports whose correlation with the current second-category civil aviation business intention is greater than a preset threshold value obtained through statistical calculation based on historical data, and based on these real-time information acquisition ports.

[0074] In an intelligent system based on a civil aviation large language model provided by an embodiment of the present application, several first-category civil aviation business intentions and several second-category civil aviation business intentions are set in the civil aviation intent list. At the same time, the task complexity corresponding to the first-category civil aviation business intention is higher than the task complexity of the second-category civil aviation business intention (it can also be understood that the first-category civil aviation business does not have a corresponding API port). In addition, after obtaining the information to be replied, the target intention will be directly determined. In this way, when the civil aviation large language model processes the information to be replied, it does not need to identify the user's intention by itself, and can quickly decide which strategy to use for processing based on whether the target intention obtained is a first-category civil aviation business intention or a second-category civil aviation business intention. Thereby reducing the difficulty of training the civil aviation large prediction model.

[0075] Furthermore, in the embodiment of the present application, a distinction is made between the first type of civil aviation business intentions and the second type of civil aviation business intentions, so that for processing tasks with higher complexity, there is no need to specifically design corresponding API ports (i.e., real-time information acquisition ports). Instead, the civil aviation large language model automatically obtains database query statements based on the information to be replied, thereby calling the civil aviation business database query port to obtain the target data set returned by the civil aviation business database query port, and using this as basic knowledge to generate reply information. In this way, on the one hand, it avoids setting up a real-time information acquisition port in a targeted manner according to each user's needs. At the same time, only the existing business query API can be set as a real-time information acquisition port, or a small number of targeted real-time information acquisition ports can be set. On the other hand, even if the user has new needs that the staff has not anticipated in advance, the civil aviation large language model can obtain the corresponding information according to the civil aviation business database query port for query, thereby generating effective reply information.

[0076] In this embodiment, an example is used for illustration. If the information to be replied input by the user is "Please help me check when my flight can be checked in", it can be classified as the second type of civil aviation business intention. After the intelligent system obtains the flight information of all flights that the user has not checked in according to the preset user itinerary acquisition port, the large language model can make targeted answers according to the above flight information. This type of intention can be solved by setting a simple API port. However, if the information to be replied input by the user is "Please help me check how many flights A and B airlines have in my historical flights", "How many of my historical flights depart at night, and how many of them are delayed", similar to this type of randomness and complexity of the intention, it is difficult to quickly obtain effective information by setting the API port. In this embodiment, the information to be replied such as "Please help me check how many flights A and B airlines have in my historical flights", "How many of my historical flights depart at night, and how many of them are delayed" will be corresponded to the first type of civil aviation business intention, so that the civil aviation large language model can effectively reply according to the data obtained from the civil aviation business database query port.

[0077] In some exemplary embodiments of the present application, the civil aviation large language model includes a civil aviation-specific model and a general model.

[0078] The step of using the civil aviation language model to obtain a database query statement according to the information to be answered includes:

[0079] The civil aviation-specific model is used to obtain a database query statement according to the information to be answered.

[0080] The step of splicing the target data set with the information to be replied and inputting the spliced ​​information into the civil aviation large language model to obtain reply information returned by the civil aviation large language model includes:

[0081] The target data set and the information to be replied are concatenated and input into the general knowledge model to obtain reply information returned by the general knowledge model.

[0082] The step of splicing the port information list and the information to be replied and inputting them into the civil aviation large language model, so that the civil aviation large language model calls at least one of the real-time information acquisition ports corresponding to the port information list to obtain target real-time information, and outputting reply information according to the target real-time information and the information to be replied, comprises:

[0083] The port information list and the information to be replied are concatenated and input into the civil aviation-specific model, so that the civil aviation-specific model calls at least one of the real-time information acquisition ports corresponding to the port information list to obtain the target real-time information, and outputs reply information according to the target real-time information and the information to be replied.

[0084] The list of civil aviation intentions also includes general knowledge intentions;

[0085] The intelligent system is also used to perform the following steps:

[0086] If the target intent is the general intent, the information to be replied is input into the civil aviation large language model to obtain reply information output by the civil aviation large language model.

[0087] The step of inputting the information to be replied to into the civil aviation large language model to obtain reply information output by the civil aviation large language model includes:

[0088] The information to be replied is input into the general knowledge model to obtain reply information output by the general knowledge model.

[0089] The intelligent system further includes a first intention recognition model;

[0090] The step of determining, from a civil aviation intention list, a target intention corresponding to the information to be replied to according to the information to be replied to includes:

[0091] The information to be replied is input into the first intention recognition model, so that the first intention recognition model determines the target intention corresponding to the information to be replied from the civil aviation intention list.

[0092] The intelligent system further includes a second intention recognition model;

[0093] Determining the target knowledge class intent from the preset knowledge class intent tree includes:

[0094] The information to be replied is input into the second intention recognition model so that the second intention recognition model determines the target knowledge class intention from the preset knowledge class intention tree.

[0095] Specifically, the first intention recognition model and / or the second intention recognition model may be included in the civil aviation large language model, or may exist independently. The first intention recognition model and / or the second intention recognition model may also be constructed using a large language model, which mainly performs classification tasks. In some embodiments, the capabilities of the first intention recognition model and / or the second intention recognition model and the tasks to be processed can also be trained simultaneously (such as instruction fine-tuning) so that it can be integrated with the civil aviation professional model into the same model. In this embodiment, the civil aviation-specific model, the general knowledge model, the first intention recognition model and the second intention recognition model are four independent large language models. This allows each model to be trained in a targeted manner according to its functional and capability requirements, without having to use a large amount of training to enable a model to simultaneously have the processing capabilities of multiple tasks, thereby reducing the computing power cost, time cost and data volume of model training.

[0096] In existing knowledge question-answering systems, the large language models used are mostly general knowledge models, that is, they have no obvious knowledge domain bias and can effectively answer general knowledge questions. However, due to the lack of specialized knowledge in certain fields or the need to ensure general knowledge, they are unable to effectively learn professional questions in professional fields during the learning process.

[0097] If only professional knowledge is used to train an initially untrained large language model, a high-quality large language model cannot be trained due to the small corpus of a single professional knowledge. In some other solutions, the relevant knowledge of a certain profession is further used for targeted training on the trained general knowledge model. However, this solution often causes the model to forget the learned general knowledge, or causes the probability space of the large language model to be affected by the particularity of professional knowledge, resulting in errors in the judgment of general knowledge (for example, the same noun has a large difference in meaning between the general knowledge field and the civil aviation field).

[0098] In response to the above problems, in this embodiment, the civil aviation large language model sets up two independent large language models, namely a civil aviation-specific model and a general knowledge model. The general knowledge model is obtained after training with general knowledge, and the civil aviation-specific model is obtained after further training with professional knowledge in the civil aviation field on the basis of a pre-trained general knowledge model (which may be the same as or different from the aforementioned general knowledge model).

[0099] In actual application, if the target intent corresponding to the information to be replied is the general knowledge intent in the civil aviation intent list, the information to be replied can be directly input into the general knowledge model for reply. If it is the civil aviation field intent (that is, the target intent is the first type of civil aviation business intent, the second type of civil aviation business intent or the civil aviation knowledge intent), the subsequent reply information is mainly generated using the civil aviation professional model.

[0100] In this way, it is possible to avoid the problem of failure to meet user needs due to errors in the judgment of general knowledge by the civil aviation-specific model.

[0101] It is worth noting that in some specific cases, in order to obtain reply information that better meets user needs, even if the target intention is in the civil aviation field, the general model will be used to generate reply information, or the civil aviation-specific model and the general model will work together to generate reply information.

[0102] For example, in an embodiment of the present application, if the target intent is a first-class civil aviation business intent, a civil aviation-specific model will be used to generate a database query statement, and a general model will be used to generate reply information based on the target data set returned by the query and the information to be replied.

[0103] In this embodiment, since the general knowledge model has no knowledge of civil aviation and no training on the ability to generate database query statements, the civil aviation-specific model is used to generate database query statements to ensure the accuracy of the database query statements. At the same time, since the general knowledge model has a stronger general knowledge ability, the general knowledge model responds to the reply information based on the target data set returned by the query, and uses its better data statistical analysis ability and language fluency than the civil aviation-specific model to make the final generated reply information more meet user needs.

[0104] In addition, although the steps of the method in the present disclosure are described in a specific order in the drawings, this does not require or imply that the steps must be performed in this specific order, or that all the steps shown must be performed to achieve the desired results. Additionally or alternatively, some steps may be omitted, multiple steps may be combined into one step, and / or one step may be decomposed into multiple steps, etc.

[0105] Through the description of the above implementation, it is easy for those skilled in the art to understand that the example implementation described here can be implemented by software, or by software combined with necessary hardware. Therefore, the technical solution according to the implementation of the present disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, including several instructions to enable a computing device (which can be a personal computer, a server, a mobile terminal, or a network device, etc.) to execute the method according to the implementation of the present disclosure.

[0106] In an exemplary embodiment of the present disclosure, an electronic device capable of implementing the above method is also provided.

[0107] Those skilled in the art will appreciate that various aspects of the present application may be implemented as a system, method or program product. Therefore, various aspects of the present application may be specifically implemented in the following forms, namely: a complete hardware implementation, a complete software implementation (including firmware, microcode, etc.), or a combination of hardware and software, which may be collectively referred to as "circuit", "module" or "system" herein.

[0108] The electronic device according to this embodiment of the present application is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0109] The electronic device is presented in the form of a general-purpose computing device. The components of the electronic device may include, but are not limited to: the at least one processor mentioned above, the at least one storage device mentioned above, and a bus connecting different system components (including storage devices and processors).

[0110] The storage stores program codes, which can be executed by the processor, so that the processor executes the steps described in the above “exemplary method” section of this specification according to various exemplary embodiments of the present application.

[0111] The memory may include readable media in the form of volatile memory, such as random access memory (RAM) and / or cache memory, and may further include read only memory (ROM).

[0112] The storage may also include a program / utility having a set (at least one) of program modules, such program modules including but not limited to: an operating system, one or more application programs, other program modules and program data, each of which or some combination may include the implementation of a network environment.

[0113] The bus may represent one or more of several types of bus structures including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, a processor, or a local bus using any of a variety of bus architectures.

[0114] The electronic device may also communicate with one or more external devices (e.g., keyboards, pointing devices, Bluetooth devices, etc.), may communicate with one or more devices that enable a user to interact with the electronic device, and / or may communicate with any device (e.g., routers, modems, etc.) that enables the electronic device to communicate with one or more other computing devices. Such communication may be performed through an input / output (I / O) interface. Furthermore, the electronic device may also communicate with one or more networks (e.g., local area networks (LANs), wide area networks (WANs), and / or public networks, such as the Internet) through a network adapter. The network adapter communicates with other modules of the electronic device through a bus. It should be understood that, although not shown in the figure, other hardware and / or software modules may be used in conjunction with the electronic device, including but not limited to: microcode, device drivers, redundant processors, external disk drive arrays, RAID systems, tape drives, and data backup storage systems, etc.

[0115] Through the description of the above implementation, it is easy for those skilled in the art to understand that the example implementation described here can be implemented by software, or by software combined with necessary hardware. Therefore, the technical solution according to the implementation of the present disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, including several instructions to enable a computing device (which can be a personal computer, a server, a terminal device, or a network device, etc.) to execute the method according to the implementation of the present disclosure.

[0116] In an exemplary embodiment of the present disclosure, a computer-readable storage medium is also provided, on which a program product capable of implementing the above method of the present specification is stored. In some possible implementations, various aspects of the present application can also be implemented in the form of a program product, which includes a program code, and when the program product is run on a terminal device, the program code is used to enable the terminal device to execute the steps according to various exemplary implementations of the present application described in the above "Exemplary Method" section of the present specification.

[0117] The program product may use any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples (non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0118] Computer readable signal media may include data signals propagated in baseband or as part of a carrier wave, in which readable program code is carried. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Readable signal media may also be any readable medium other than a readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0119] The program code embodied on the readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wired, optical cable, RF, etc., or any suitable combination of the foregoing.

[0120] Program code for performing the operations of the present application may be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java, C++, etc., and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as a separate software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server. In the case of a remote computing device, the remote computing device may be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computing device (e.g., using an Internet service provider to connect through the Internet).

[0121] In addition, the above-mentioned figures are only schematic illustrations of the processes included in the method according to the exemplary embodiments of the present application, and are not intended to be limiting. It is easy to understand that the processes shown in the above-mentioned figures do not indicate or limit the time sequence of these processes. In addition, it is also easy to understand that these processes can be performed synchronously or asynchronously, for example, in multiple modules.

[0122] It should be noted that, although several modules or units of the device for action execution are mentioned in the above detailed description, this division is not mandatory. In fact, according to the embodiments of the present disclosure, the features and functions of two or more modules or units described above can be embodied in one module or unit. On the contrary, the features and functions of one module or unit described above can be further divided into multiple modules or units to be embodied.

[0123] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed in the present application should be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be based on the protection scope of the claims.

Claims

1. A method for obtaining reply information based on a civil aviation large language model, characterized in that: The following steps are involved: Get the information to be replied to entered by the user; If the target intent corresponding to the information to be replied is a civil aviation knowledge intent, then determining the target knowledge intent from a preset knowledge intent tree; The target knowledge class intention is any leaf node in the knowledge class intention tree; at least some of the leaf nodes in the knowledge class intention tree are provided with a target identifier; The leaf node with the target identifier has a corresponding real-time information acquisition port; if the leaf node is provided with the target identifier, it indicates that the intention corresponding to the information to be replied has a probability of being a civil aviation business intention; If the target knowledge class intent carries a target identifier, then the real-time information acquisition port corresponding to the current target knowledge class intent is called according to the user's account information to obtain the target real-time information corresponding to the user; and inputting the target knowledge class intention and the information to be answered into a preset civil aviation knowledge database query port to obtain the target civil aviation knowledge information returned by the civil aviation knowledge database query port; The information to be replied, the target real-time information and the target civil aviation knowledge information are spliced ​​and input into the civil aviation large language model to obtain reply information output by the civil aviation large language model.

2. The method for obtaining reply information according to claim 1, characterized in that: The civil aviation large language model is configured to give priority to outputting the reply information as a civil aviation business processing task when the input information includes both the target real-time information and the target civil aviation knowledge information.

3. The method for obtaining reply information according to claim 2, characterized in that: The reply information includes knowledge text and / or page jump hyperlink; the page jump hyperlink is used to jump to the corresponding civil aviation business processing page.

4. The method for obtaining reply information according to claim 2, characterized in that: The reply information includes knowledge text and / or business processing buttons; The method further comprises: In response to the user clicking the service processing button, a corresponding service processing instruction is generated according to the target real-time information, and the service processing instruction is sent to the service processing port corresponding to the service processing button.

5. The method for obtaining reply information according to claim 1, characterized in that: The method further comprises: If the target knowledge class intent does not carry a target identifier, the target knowledge class intent and the information to be replied are input into the civil aviation knowledge database query port to obtain the target civil aviation knowledge information returned by the civil aviation knowledge database query port; The target civil aviation knowledge information, the target knowledge class intent and the information to be replied are spliced ​​and input into the civil aviation large language model to obtain the reply information returned by the civil aviation large language model.

6. The method for obtaining reply information according to any one of claims 1 or 5, characterized in that: At least some of the sub-nodes in the knowledge-based intention tree have corresponding necessary information; Determining the target knowledge class intent from the preset knowledge class intent tree includes: According to the information to be replied, a target knowledge class intent is matched layer by layer from a plurality of knowledge class intents included in the knowledge class intent tree; If the currently matched knowledge class intent is a leaf node, the currently matched knowledge class intent is determined as the target knowledge class intent; otherwise, the matching continues in the child node corresponding to the currently matched knowledge class intent; If there is no sub-node that meets the matching condition in the sub-node corresponding to the currently matched knowledge-type intent, an information acquisition prompt is generated according to the necessary information corresponding to the sub-node corresponding to the currently matched knowledge-type intent; Outputting the information acquisition prompt to acquire the supplementary information input by the user; After the information acquisition prompt, the supplementary information and the information to be replied are spliced ​​into new information to be replied, the target knowledge class intent is determined from a preset knowledge class intent tree according to the new information to be replied.

7. An electronic device, characterized in that: including a processor and a memory; The processor is used to execute the steps of the method according to any one of claims 1 to 6 by calling the program or instruction stored in the memory.

8. A non-transitory computer-readable storage medium, characterized in that: The non-transitory computer-readable storage medium stores a program or instruction, and the program or instruction enables a computer to execute the steps of the method according to any one of claims 1 to 6.

Images