Management and control intelligent interactive question answering method, device and system

Abstract

The invention relates to the technical field of network intelligent operation and maintenance, and provides a management and control intelligent interactive question answering method, device and system. Wherein a network communication knowledge base is constructed, and a general large model is trained into a network communication field large model based on the network communication knowledge base; calling a network communication field large model to perform reasoning on the natural language input by the user to output an undetermined answer A1, and performing information extraction on the natural language input by the user based on the cue word to obtain an information extraction result; performing knowledge reasoning based on an information extraction result and a network communication knowledge base, and outputting an undetermined answer A2; and performing controllable constraint on the to-be-determined answer A1 and the to-be-determined answer A2 to generate a final answer A3. According to the method, the knowledge graph is quickly constructed, the mining cost is reduced, the universality is improved, the question and answer accuracy and reliability based on the generative AI technology can be improved, and the effect of managing and controlling the intelligent interactive question and answer system is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of network intelligent operation and maintenance, in particular to a management and control intelligent interactive question and answer method, device and system. BACKGROUND

[0002] Both knowledge graph technology and generative AI technology are used to build intelligent interactive question and answer systems. The industry and other patents have mature methods for building intelligent interactive question and answer systems using the two technologies separately. However, from the actual application, knowledge graph has the problems of high difficulty in constructing knowledge graph, difficulty in accumulating the quantity and quality of data, difficulty in mining scattered documents in the management and control field and high mining cost, and cannot well handle the incomplete and dynamic characteristics of knowledge graph in real application, and cannot effectively model unknown entities and represent new facts. Therefore, the knowledge graph method is usually customized for specific small field applications or tasks (such as Baidu's character relationship, movie relationship, etc.), lacking sufficient universality.

[0003] And the large model will remember the facts and knowledge contained in the training corpus during training. Although it has been successful in many applications such as intent understanding, its accuracy and reliability are still questionable. For example, when asked "When did Einstein discover gravity?", chatgpt may say "Einstein discovered gravity in 1687", which contradicts the theory of gravity established by Isaac Newton, affecting the credibility, especially in numbers and letters. Moreover, as a black box model, large models also lack explainability, implicitly representing knowledge in the form of parameters, making it difficult to explain or verify the acquired knowledge. The specific patterns and functions used for prediction or decision-making are not directly accessible or interpretable to humans.

[0004] In summary, using only large model technology to build a question and answer system cannot meet the requirements of the management and control system in engineering applications in terms of accuracy, reliability and explainability, and using only knowledge graph technology to build a question and answer system will result in the problem of narrow support of the question and answer system, high mining cost and insufficient universality.

[0005] Therefore, overcoming the defects of the prior art is an urgent problem in the technical field. SUMMARY

[0006] The technical problem to be solved by the present application is that using only large model technology to build a question and answer system cannot meet the requirements of the management and control system in engineering applications in terms of accuracy, reliability and explainability, and using only knowledge graph technology to build a question and answer system will result in the problem of narrow support of the question and answer system, high mining cost and insufficient universality.

[0007] The present application adopts the following technical solutions:

[0008] In a first aspect, the present application provides an intelligent interactive question and answer method, comprising:

[0009] A network communication knowledge base is constructed, and a general large model is trained into a network communication field large model based on the network communication knowledge base; the network communication field large model is a large model suitable for information communication networks, including network operation management large models, network operation and maintenance large models, and other large models applied to different scenarios;

[0010] The network communication field large model is called to infer and output a pending answer A1 for the natural language input by the user, and information extraction is performed on the natural language input by the user based on a prompt word (prompt) to obtain an information extraction result; knowledge reasoning is performed based on the information extraction result and the network communication knowledge base to output a pending answer A2;

[0011] The pending answer A1 and the pending answer A2 are subjected to controllable constraint to generate a final answer A3.

[0012] Preferably, the method further comprises:

[0013] After the user obtains the final answer, the user provides a correct answer to the feedback engine through a feedback button of a page, and the feedback engine updates the network communication knowledge base in the knowledge graph engine in combination with the information extraction result;

[0014] After the network communication knowledge base is updated, a question and answer data training set is automatically generated, and the generative AI engine is triggered to use the training set to perform online fine-tuning training on the network communication field large model to improve the generative AI capability.

[0015] Preferably, the construction of the network communication knowledge base specifically comprises,

[0016] A network communication knowledge graph is constructed, and the ontology relationship and schema form of the network communication knowledge graph are defined;

[0017] A general large model is called to perform information extraction on network communication related technical text files according to the ontology relationship and schema form of the network communication knowledge graph, and a network communication knowledge base composed of triples is generated.

[0018] Preferably, the ontology relationship of the network communication knowledge graph is specifically: a compact triple relationship composed of a theme, a modifier, and a feature item; wherein the content recorded in a fault case is split into one or more triple relationships, and there is an overlap of at least one content of the theme, the modifier, and the feature item between adjacent triple relationships.

[0019] Preferably, the training of the general large model into the network communication field large model based on the network communication knowledge base specifically comprises,

[0020] Automatic generation of question and answer data training set based on network communication knowledge base;

[0021] Taking the question and answer data training set as input, the general large model is fine-tuned to a network communication field large model.

[0022] Preferably, the fine-tuning of the general large model to the network communication field large model further comprises:

[0023] Different question methods are used to convert large models to supplement sample data for data enhancement, and three-stage training based on sample data is used for fine-tuning to improve the ability of the large model in the communication field;

[0024] The network communication field large model optimized by fine-tuning is further used to expand the triple knowledge base.

[0025] Preferably, the automatic generation of question and answer data training set specifically comprises:

[0026] In a self-questioning and self-answering manner, two elements in the triple are inserted into the question template, and the third element is taken as the answer to establish the question and answer data training set.

[0027] Preferably, the calling of the network communication field large model to infer the natural language input by the user to output the pending answer A1 specifically comprises:

[0028] Short-term memory, used to directly take the historical dialogue as the history parameter;

[0029] Long truncation, used to truncate the complete historical dialogue in the history parameter when it exceeds the maximum Token length limit, and the complete historical dialogue is truncated from recent to distant according to the dialogue time;

[0030] Long-term memory, used to summarize the content of the truncated part based on the prompt to form long-term memory;

[0031] Intention clarification, used to ensure the accuracy of the reply content, specifically when the user's intention is unclear, the prompt is used to ask the user for intention clarification to obtain accurate user intention; when the user's intention is clear, the question (Query) parameter of the question content in the historical dialogue in the history parameter is rewritten to restate as an independent question with accurate intention.

[0032] Preferably, the calling of the network communication field large model to infer the natural language input by the user to output the pending answer A1 further comprises:

[0033] Semantic retrieval, used to vectorize the knowledge base and build an index when the service starts, and to retrieve the most similar user question by vector when called;

[0034] Answer generation, which is used to pass the question-answer pairs obtained by semantic retrieval as context information to obtain the initial answer;

[0035] Hallucination detection is used to detect hallucinations of the initial answer based on the prompt to prevent it from generating hallucination content, including: when judging the authenticity of a given answer, at least one new evidence needs to be completed to support the authenticity of the answer.

[0036] In a second aspect, the present invention also provides a management and control intelligent interactive question-and-answer system, comprising a network communication field large model generation module, a first pending answer generation module, a second pending answer generation module, and a final answer generation module, wherein the network communication field large model generation module is respectively connected to the data of the first pending answer generation module and the second pending answer generation module, and the output ends of the first pending answer generation module and the second pending answer generation module data are also respectively connected to the final answer generation module data. Specifically:

[0037] The network communication domain large model generation module is used to build a network communication knowledge base and train the general large model into a network communication domain large model based on the network communication knowledge base;

[0038] The first pending answer generation module is used to call the large model in the network communication field to infer the natural language input by the user and output the pending answer A1;

[0039] The second pending answer generation module extracts information from the natural language input by the user based on the prompt word to obtain the information extraction result; performs knowledge reasoning based on the information extraction result and the network communication knowledge base, and outputs the pending answer A2;

[0040] The final answer generation module is used to generate the final answer A3 by applying controllable constraints to the pending answer A1 and the pending answer A2.

[0041] In a third aspect, the present invention provides a method for managing and controlling intelligent interactive question-answering, comprising:

[0042] Based on text documents such as manuals and / or fault case libraries, standard specifications, etc., information is extracted from the text documents according to the ontology relationship and schema form of the pre-defined network communication knowledge graph to generate a triple knowledge base;

[0043] Constructing a question-answer data training set from the triple knowledge base, importing the question-answer data training set into a large model, and publishing the large model to a generative AI engine to provide generative AI capabilities;

[0044] Based on the natural language of user input, the generative AI engine interface is called, and based on the original problem of user input, the large model outputs the pending answer A1 and the prompt based on the extracted information, and the large model outputs the information extraction result B1;

[0045] The information extraction result B1 is called as input to the triple knowledge base in the knowledge graph engine for knowledge reasoning, and the pending answer A2 based on the information extraction result is queried out;

[0046] The pending answer A1 and the pending answer A2 are called as input to the result optimization module to generate an answer A3 as a controllable constraint, which is output to the user as the final answer of the intelligent assistant.

[0047] Preferably, the method further comprises:

[0048] After the user obtains the final answer, the correct answer is provided to the feedback engine through the feedback button of the page, and the triple knowledge base in the knowledge graph engine is updated by the feedback engine combined with the information extraction result;

[0049] After updating the triple knowledge base, the automatic script automatically generates a question and answer data training set, and periodically triggers the call to the generative AI engine to use the training set to fine-tune the large model in the field of network communication to improve the generative AI capability.

[0050] Preferably, after updating the triple knowledge base, the automatic script automatically generates a question and answer data training set, specifically including:

[0051] In a self-questioning and self-answering manner, the two elements in the triple are inserted into the question template, and the third element is established as the answer to build the question and answer data training set.

[0052] Preferably, the method further comprises:

[0053] The generated question and answer data are used to fine-tune the large model, and at the same time, different question methods are used to convert the large model to supplement sample data for data enhancement, and the large model is fine-tuned based on the sample data to improve the capability of the large model in the field of communication; and the large model optimized by fine-tuning is further used to expand the triple knowledge base.

[0054] Preferably, the ontology relationship of the pre-defined network communication knowledge graph is a compact triple relationship composed of a theme, a modifier and a feature item; wherein the content recorded in a fault case is split into one or more triple relationships, and there is an overlap of at least one content of the theme, the modifier and the feature item between adjacent triple relationships.

[0055] Preferably, the large model at least includes a short-term memory module, an ultra-long truncation module, a long-term memory module and an intent clarification module, specifically:

[0056] The short-term memory module is configured to directly take the historical dialogue as a history parameter;

[0057] The ultra-long truncation module is configured to perform truncation processing on the complete historical dialogue from the recent to the distant when the complete historical dialogue contained in the history parameter exceeds the maximum Token length limit;

[0058] The long-term memory module is configured to perform dialogue summarization on the content of the truncated part based on a prompt to form long-term memory;

[0059] The intent clarification module is configured to ensure the accuracy of the reply content, specifically, when the user intent is unclear, the prompt is used to ask the user for intent clarification to obtain an accurate user intent; and when the user intent is clear, the query parameter rewriting is performed on the question content in the historical dialogue in the history parameter so as to restate the independent question with accurate intent.

[0060] Preferably, the large model further comprises a semantic retrieval module, an answer generation module and an illusion detection module, specifically:

[0061] The semantic retrieval module is configured to vectorize the knowledge base and build an index when the service is started, and perform vector retrieval to obtain the most similar user question when called;

[0062] The answer generation module is configured to take the question and answer pair obtained by the semantic retrieval as Context information to obtain an initial answer;

[0063] The illusion detection module is configured to perform illusion detection on the initial answer based on the prompt to prevent the generation of illusion content, including: when judging the authenticity of a given answer, at least one new evidence is needed to support the authenticity of the answer.

[0064] Preferably, the method of confirming the understanding ability of the large model by using multiple calls of the large model in the process of outputting the A1 answer is used to avoid the illusion problem of the large model itself and improve the accuracy.

[0065] In a fourth aspect, the present application provides an intelligent interactive question and answer system, comprising:

[0066] The triple knowledge base generation module is configured to extract information in the text document according to the pre-defined ontology relationship and schema form of the network communication knowledge graph according to the text document including the manual and / or the fault case library, so as to generate a triple knowledge base;

[0067] The training set generation and import module is configured to construct a question and answer data training set from the triple knowledge base, import the question and answer data training set into the large model, and publish the large model to the generative AI engine to provide generative AI capability.

[0068] The first stage answer generation module calls a generative AI engine interface based on the natural language input by the user, calls a large model to output a pending answer A1 based on the original question input by the user and calls a large model to output an information extraction result B1 based on the prompt of the extracted information, respectively;

[0069] The second stage answer generation module queries a pending answer A2 based on the information extraction result by calling a triple knowledge base in the knowledge graph engine for knowledge reasoning with the information extraction result B1 as input;

[0070] The final answer generation module calls a result optimization module to generate an answer A3 as a controllable constraint with the pending answer A1 and the pending answer A2 as input, and outputs the answer A3 as the final answer of the intelligent assistant to the user.

[0071] In a fifth aspect, the present application further provides a controlled intelligent interactive question and answer device for implementing the controlled intelligent interactive question and answer method of the first aspect and the third aspect, and the device comprises:

[0072] At least one processor; and a memory connected in communication with the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the processor to execute the controlled intelligent interactive question and answer method of the first aspect and the third aspect.

[0073] In a sixth aspect, the present application further provides a non-volatile computer storage medium, and the computer storage medium stores computer executable instructions, and the computer executable instructions are executed by one or more processors to complete the controlled intelligent interactive question and answer method of the first aspect.

[0074] The present application can quickly build a knowledge graph, reduce the mining cost, improve the generality, and improve the accuracy and reliability of the question and answer based on the generative AI technology, and improve the effect of the controlled intelligent interactive question and answer system. BRIEF DESCRIPTION OF DRAWINGS

[0075] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following will briefly introduce the drawings needed to be used in the embodiments of the present application. Obviously, the drawings described below are only some embodiments of the present application, and for those skilled in the art, other drawings can also be obtained from these drawings without creative labor.

[0076] Figure 1 is an architecture example diagram of a controlled intelligent interactive question and answer method provided by the embodiments of the present application implemented on a control system;

[0077] Figure 2is a process flow schematic diagram of a management and control intelligent interactive question and answer method provided by an embodiment of the present application;

[0078] Figure 3 is a process flow schematic diagram of a feedback-based management and control intelligent interactive question and answer method provided by an embodiment of the present application;

[0079] Figure 4 is a process flow schematic diagram of a management and control intelligent interactive question and answer method provided by an embodiment of the present application;

[0080] Figure 5 is a schematic diagram of a management and control intelligent interactive question and answer system architecture provided by an embodiment of the present application;

[0081] Figure 6 is a process flow schematic diagram of a management and control intelligent interactive question and answer method provided by an embodiment of the present application;

[0082] Figure 7 is a process flow schematic diagram of a feedback-based management and control intelligent interactive question and answer method provided by an embodiment of the present application;

[0083] Figure 8 is a schematic diagram of a management and control intelligent interactive question and answer method large model internal sub-module structure provided by an embodiment of the present application;

[0084] Figure 9 is a schematic diagram of a management and control intelligent interactive question and answer system architecture provided by an embodiment of the present application;

[0085] Figure 10 is a schematic diagram of a management and control intelligent interactive question and answer method training process provided by an embodiment of the present application;

[0086] Figure 11 is a schematic diagram of a management and control intelligent interactive question and answer large system internal sub-module implementation process provided by an embodiment of the present application;

[0087] Figure 12 is a schematic diagram of a deployment reasoning instance process for steps 305-310 provided by an embodiment of the present application;

[0088] Figure 13 is a schematic diagram of a management and control intelligent interactive question and answer device structure provided by an embodiment of the present application. DETAILED DESCRIPTION

[0089] In order to make the purpose, technical scheme and advantages of the present application clearer, the present application will be further described in detail below in combination with the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and do not limit the present application.

[0090] In the description of the present application, the terms "inner", "outer", "longitudinal", "transverse", "upper", "lower", "top", "bottom" and the like indicate the orientation or positional relationship based on the orientation or positional relationship shown in the drawings, and are only for the convenience of describing the present application and do not require the present application to be constructed and operated in a particular orientation, and therefore should not be understood as a limitation on the present application.

[0091] The terms "first", "second", and the like in the present application are only for descriptive purposes and should not be understood as indicating or implying relative importance or implicitly indicating the number of the technical features indicated. Therefore, the features defined with "first", "second" and the like can explicitly or implicitly include one or more features. In the description of the present application, unless otherwise specified and limited, the term "a plurality of" means two or more.

[0092] In the present application, unless otherwise specified and limited, the term "connection" should be understood broadly, for example, "connection" can be fixed connection, or detachable connection, or integral; can be directly connected, or indirectly connected through intermediate medium. In addition, the term "coupling" can be an electrically connected way to achieve signal transmission.

[0093] In addition, the technical features involved in each embodiment of the present application described below can be combined with each other as long as there is no conflict between them.

[0094] Example 1:

[0095] The present application provides a kind of management and control intelligent interaction question and answer method, as shown in Figure 2 The objects and functional subjects involved in the corresponding method steps can also be understood with reference to an example architecture of the management and control system as shown in Figure 1 The method comprises:

[0096] In step 101, a network communication knowledge base is constructed, and a general large model is trained into a network communication field large model based on the network communication knowledge base.

[0097] The network communication knowledge base is constructed, specifically including constructing a network communication knowledge graph, defining the ontology relationship and schema form of the network communication knowledge graph; calling the general large model to perform information extraction on network communication related technical text files according to the ontology relationship and schema form of the network communication knowledge graph, to generate a network communication knowledge base composed of triples.

[0098] The ontology relationship of the network communication knowledge graph is specifically: a compact triple relationship composed of a subject, an attribute and a feature item; wherein the content recorded in a fault case is split into one or more triple relationships, and there is an overlap of at least one content of the subject, the attribute and the feature item between adjacent triple relationships.

[0099] In step 102, a network communication field large model is called to infer the natural language input by the user to output a pending answer A1.

[0100] The calling network communication field large model infers the natural language input by the user to output a pending answer A1, specifically including: short-term memory, used to directly take the historical dialogue as a history parameter; super-long truncation, used to truncate the complete historical dialogue from near to far when the complete historical dialogue contained in the history parameter exceeds the maximum Token length limit; long-term memory, used to summarize the content of the truncated part based on the prompt to form long-term memory; intent clarification, used to ensure the accuracy of the reply content, specifically when the user's intent is unclear, the user is asked based on the prompt to clarify the intent to obtain an accurate user intent; after the user's intent is clear, the question content in the historical dialogue in the history parameter is rewritten as a Query parameter to restate as an independent question with accurate intent.

[0101] Preferably, the calling network communication field large model infers the natural language input by the user to output a pending answer A1, further including: semantic retrieval, used to vectorize the knowledge base and build an index at the start of the service, and perform vector retrieval to obtain the most similar user question when called; answer generation, used to take the question and answer pair obtained by semantic retrieval as Context information to obtain an initial answer; illusion detection, used to detect illusions based on the prompt for the initial answer to prevent it from generating illusion content, including: when judging the authenticity of a given answer, at least one new evidence is needed to support the authenticity of the answer.

[0102] In step 103, information extraction is performed on the natural language input by the user based on the prompt (prompt), and an information extraction result is obtained; knowledge reasoning is performed based on the information extraction result and the network communication knowledge base, and a pending answer A2 is output.

[0103] In step 104, the pending answer A1 and the pending answer A2 are subjected to controllable constraint to generate a final answer A3.

[0104] The embodiment of the application can quickly construct a knowledge graph, reduce the cost of mining, improve the generality, and improve the accuracy and reliability of the question and answer based on the generative AI technology, and improve the effect of the intelligent interactive question and answer system.

[0105] In combination with the embodiment of the application, as a complete scheme with a feedback mechanism, as shown in Figure 3 The method steps can further include:

[0106] In step 105, after the user obtains the final answer, the correct answer is provided to the feedback engine through the feedback button of the page, and the network communication knowledge base in the knowledge graph engine is updated by the feedback engine in combination with the information extraction result.

[0107] In step 106, after updating the network communication knowledge base, a question and answer data training set is automatically generated, and the generative AI engine is triggered to use the training set to fine-tune the network communication field large model online to improve the generative AI capability.

[0108] In step 101 of the embodiment of the application, the general large model is trained into a network communication field large model based on the network communication knowledge base, and a specific implementation method process is provided in the specific embodiment of the application, as shown in the following table. Figure 4 As shown in the table, it includes:

[0109] In step 1011, a question and answer data training set is automatically generated based on the network communication knowledge base.

[0110] In step 1012, the question and answer data training set is used as input, and the general large model is fine-tuned into a network communication field large model. Wherein, the fine-tuning of the general large model into the network communication field large model further includes:

[0111] The different question methods are converted by using the large model to supplement the sample data for data enhancement, and the three-stage training is fine-tuned based on the sample data to improve the capability of the large model in the communication field; and the network communication field large model optimized by fine-tuning is further used to expand the triple knowledge base.

[0112] Wherein, the automatically generated question and answer data training set specifically includes: in a self-questioning and self-answering manner, inserting two elements in the triple into a question template, and establishing a question and answer data training set with the third element as the answer.

[0113] Embodiment 2:

[0114] The application also provides a management and control intelligent interactive question and answer system, as shown in the following table. Figure 5 The network communication field large model generation module, the first pending answer generation module, the second pending answer generation module and the final answer generation module are connected in data, and the output ends of the first pending answer generation module and the second pending answer generation module are further connected in data with the final answer generation module, and the network communication field large model generation module is used for constructing a network communication knowledge base and training a general large model into a network communication field large model based on the network communication knowledge base.

[0115] As shown in the table, it includes:

[0116] The network communication field large model generation module is used for constructing a network communication knowledge base and training a general large model into a network communication field large model based on the network communication knowledge base.

[0117] The first pending answer generation module is configured to call a network communication field large model to infer the natural language input by the user to output a pending answer A1.

[0118] The second pending answer generation module is configured to perform information extraction on the natural language input by the user based on a prompt word to obtain an information extraction result, and perform knowledge reasoning based on the information extraction result and a network communication knowledge base to output a pending answer A2.

[0119] The final answer generation module is configured to generate a final answer A3 by performing controllable constraint on the pending answer A1 and the pending answer A2.

[0120] Embodiment 1 is an embodiment scheme of the common inventive concept of the system side of embodiment 1, therefore, the related method content and the extended implementation process in embodiment 1 are also applicable to the present embodiment, and will not be described in detail here.

[0121] Embodiment 3:

[0122] Embodiment 3 of the present application provides a method for controlling intelligent interactive question and answer, which is a method process performance of embodiment 1 in a certain instance scenario, and can also be understood as a lower implementation scheme process of embodiment 1, as shown in the following table: Figure 6 The method comprises the following steps:

[0123] In step 201, according to the text documents including manuals and / or fault case libraries, the information in the text documents is extracted according to the ontology relationship and schema form of the pre-defined network communication knowledge graph, so as to generate a triple knowledge base.

[0124] The ontology relationship of the pre-defined network communication knowledge graph is specifically: a compact triple relationship composed of a subject, an attribute and a feature item; wherein the content recorded in a fault case is split into one or more triple relationships, and there is an overlap of at least one content of the subject, the attribute and the feature item between adjacent triple relationships.

[0125] The schema form is a data format, which is expressed in the following data format table:

[0126] 690|Single disk has|LFADA02

[0127] 690|Single disk has|LFADA04

[0128] 690|Single disk has|LFADA08

[0129] 690|Single disk has|LFADA09

[0130] LFADA02|Port rate|60G

[0131] LFADA04 | Port Rate | 70G

[0132] LFADA08 | Port Rate | 90G

[0133] LFADA09 | Port Rate | 80G

[0134] The original data content matched in the manual is: "SPN device 690 series single disc force has LFADA02, LFADA04, LFADA08, LFADA09, LFADA02 port rate 60G, LFADA04 port rate 70G, LFADA08 port rate 90G, LFADA09 port rate 80G".

[0135] In step 202, a triple knowledge base is constructed for question and answer data training set, and the question and answer data training set is imported into a large model, and the large model is published to a generative AI engine to provide generative AI capability.

[0136] Among them, generative AI (Generative AI) is an artificial intelligence technology that uses machine learning models and deep learning techniques to generate new content by studying the patterns of historical data, which can be text, images, audio or video. Generative AI does not generate output according to given rules or data, but generates completely new content autonomously, similar to human creativity. For example, the recently popular chat robot ChatGPT uses the core model GPT-3 to generate high-quality natural language text, which can be used in chat, writing, automated customer service, etc.

[0137] In step 203, based on the natural language input by the user, the generative AI engine interface is called, and the large model is called to output the pending answer A1 based on the original question input by the user and the information extraction result B1 based on the prompt of the extracted information.

[0138] In step 204, the information extraction result B1 is input to the triple knowledge base in the knowledge graph engine for knowledge reasoning, and the pending answer A2 based on the information extraction result is queried out.

[0139] In step 205, the pending answer A1 and the pending answer A2 are input to the result optimization module to generate an answer A3 as an intelligent assistant final answer output to the user.

[0140] The embodiment of the application can quickly construct a knowledge graph, reduce the cost of mining, improve the generality, and improve the accuracy and reliability of the question and answer based on the generative AI technology, and improve the effect of the intelligent interactive question and answer system.

[0141] In combination with the embodiments of the present application, there is also a preferred implementation scheme, as shown in the figure, after the above-mentioned step 205 is executed, in order to better maintain the running accuracy of the large model, the method further comprises: Figure 7

[0142] In step 206, after the user obtains the final answer, the correct answer is provided to the feedback engine through the feedback button of the page, and the triple knowledge base in the knowledge graph engine is updated by the feedback engine in combination with the information extraction result.

[0143] In step 207, after updating the triple knowledge base, the automatic script automatically generates a question and answer data training set, and periodically triggers the call to generate the AI engine to use the training set to fine-tune the large model in the field of network communication to improve the generative AI capability.

[0144] In the above-mentioned step 207, after updating the triple knowledge base, the automatic script automatically generates a question and answer data training set, and specifically includes: in a self-questioning and self-answering manner, inserting two elements in the triple into the question template, and establishing a question and answer data training set with the third element as the answer.

[0145] In addition, in addition to the above-optimized way of establishing a question and answer data training set, in the implementation process of the embodiments of the present application, the generated question and answer data can be used to fine-tune the large model, and at the same time, the large model can be used to convert different question methods to supplement sample data for data enhancement, and based on the sample data, fine-tune the three-stage training to improve the ability of the large model in the field of communication; and the large model optimized by fine-tuning is further used to expand the triple knowledge base.

[0146] In the specific implementation process of the embodiments of the present application, the implementation of the large model can also be expanded to the following details for content subdivision expression, as shown in the figure, the large model at least includes a short-term memory module, an ultra-long truncation module, a long-term memory module and an intent clarification module, in order to Figure 8 Figure 1 For reference, the bearing object of each module included in the above-mentioned large model is Figure 1 the large model management and large model reasoning, for example, the short-term memory module, the ultra-long truncation module, the long-term memory module can be attributed to Figure 1 the large model management, and the intent clarification module can be attributed to Figure 1 the large model reasoning, specifically:

[0147] The short-term memory module is used to directly take the historical dialogue as a history parameter;

[0148] ​​An ultra-long truncation module is configured to truncate a complete history dialogue included in a history parameter when the complete history dialogue exceeds a maximum token length limit, and the truncation is performed from recent to remote based on a dialogue time;

[0149] A long-term memory module is configured to summarize the content of the truncated part based on a prompt to generate a dialogue summary to form a long-term memory. For example, a concise summary is created based on the following dialogue, focusing on key information to be remembered and potentially important information. The summary is concise and controlled within 400 words.

[0150] An intent clarification module is configured to ensure the accuracy of the reply content. Specifically, when the user's intent is unclear, the user is asked to clarify the intent based on a prompt to obtain an accurate user intent. When the user's intent is clear, the question content in the history dialogue in the history parameter is rewritten as a Query parameter to restate it as an independent question with accurate intent. For example, based on the following dialogue and subsequent questions, the subsequent questions are restated as independent questions with accurate intent. If the user's true intent is not understood or uncertain, the user is asked to obtain their true intent, so that the problem is solved step by step to ensure that the correct answer is obtained.

[0151] Further, the large model further includes a semantic retrieval module, an answer generation module, and an illusion detection module. The newly added semantic retrieval module, answer generation module, and illusion detection module can all be classified as Figure 1 large model inference in the large model; and here, the answer generation module is classified as a large model, rather than being separately provided outside the large model inference as in Figure 1 It can also be understood that the answer generation module in the large model here is the output of the pending answer A1, and Figure 1 the answer generation module in the large model is used to generate the final answer A3 by controllably constraining the pending answer A1 and the pending answer A2. Specifically:

[0152] The semantic retrieval module is configured to vectorize the knowledge base and build an index when the service is started, and perform vector retrieval to obtain the most similar user question when called.

[0153] The answer generation module is configured to input the question and answer pair obtained by semantic retrieval as Context information to obtain an initial answer.

[0154] The illusion detection module is configured to detect illusions based on a prompt to prevent the generation of illusion content. The answer includes: when judging the authenticity, at least one new evidence needs to be completed to support the authenticity of the answer.

[0155] Wherein, the method of using multiple calls to the large model to confirm the understanding ability of the large model in the process of outputting the A1 answer is used to avoid the hallucination problem of the large model itself and improve the accuracy.

[0156] Embodiment 4:

[0157] Based on the method of embodiment 3, the embodiment of the application further proposes an intelligent interactive question and answer system, as shown in the figure, which comprises: Figure 9

[0158] A triple knowledge base generation module is configured to extract information in a text document according to a pre-defined ontology relationship and schema form of a network communication knowledge graph, and generate a triple knowledge base according to the text document including a manual and / or a fault case library.

[0159] A training set generation and import module is configured to construct a question and answer data training set from the triple knowledge base, import the question and answer data training set into a large model, and publish the large model to a generative AI engine to provide generative AI capabilities.

[0160] A first stage answer generation module is configured to call a generative AI engine interface based on a natural language input by a user, call a large model to output a pending answer A1 based on a prompt input by the user, and call the large model to output an information extraction result B1 based on extracted information.

[0161] A second stage answer generation module is configured to input the information extraction result B1 into a triple knowledge base in the knowledge graph engine to perform knowledge reasoning, and query a pending answer A2 based on the information extraction result.

[0162] A final answer generation module is configured to input the pending answer A1 and the pending answer A2 into a result optimization module to generate an answer A3 as an intelligent assistant final answer output to the user.

[0163] The embodiment of the application can quickly construct a knowledge graph, reduce the cost of mining, improve the generality, and improve the accuracy and reliability of the question and answer based on the generative AI technology, and improve the effect of the intelligent interactive question and answer system.

[0164] It should be noted that the present application is based on the system side of the common inventive concept of embodiment 3, therefore, the related extension embodiments and detail contents in embodiment 3 are also applicable to the embodiment of the present application, and will not be repeated here.

[0165] Embodiment 5:

[0166] ​The embodiment of the application is based on the method of embodiment 1 and the system of embodiment 2, and can also be understood as based on the method of embodiment 3 and the system of embodiment 4. From a possible example scenario, an example scheme has been initially described in a relatively complete flow process, such as Figure 10 As shown in the training state relationship diagram of the following steps 301-304, Figure 12 is a deployment reasoning example process diagram of steps 305-310. The method of the embodiment of the application comprises the following steps:

[0167] In step 301, a network communication knowledge graph is constructed, and the ontology and schema form are defined.

[0168] In step 302, based on manual, fault case library and other text documents, a large model is used to extract information according to the ontology relationship and schema form of the network communication knowledge graph defined in step 301, to generate a triple knowledge base.

[0169] For example: According to the manual description, “SPN device 690 series single disk has LFADA02, LFADA04, LFADA08, LFADA09, LFADA02 port rate 60G, LFADA04 port rate 70G, LFADA08 port rate 90G, LFADA09 port rate 80G”.

[0170] The large model can be directly used to extract information to generate a triple knowledge base:

[0171] 690|single disk has|LFADA02

[0172] 690|single disk has|LFADA04

[0173] 690|single disk has|LFADA08

[0174] 690|single disk has|LFADA09

[0175] LFADA02|port rate|60G

[0176] LFADA04|port rate|70G

[0177] LFADA08|port rate|90G

[0178] LFADA09|port rate|80G

[0179] In step 303, a triple knowledge base in step 302 is used to quickly construct a question and answer data training set, and a large model is fine-tuned and trained into a network communication field large model;

[0180] For example:

[0181] LFADA02 | Port Rate | 60G;

[0182] Q: What is the port rate of LFADA02?

[0183] A: 60G.

[0184] Q: What is the single disk rate of 60G in 690?

[0185] A: LFADA02.

[0186] Q: What parameter is 60G in 690?

[0187] A: Port Rate.

[0188] In step 304, steps 302 and 303 are repeated to fine-tune the large model with new question and answer data pairs, and to use the large model to convert different question forms to supplement sample data for data augmentation. Based on the sample data, fine-tune the three-stage training to improve the large model's ability in the field of communication. Further expand the triple knowledge base using the fine-tuned and optimized large model, and repeat these two processes until the knowledge base and large model reach the ideal range.

[0189] In step 305, the triple knowledge base optimized in step 304 is imported into the control system knowledge graph engine to provide knowledge graph capabilities, and the optimized large model in the field of network communication is published to the generative AI engine to provide generative AI capabilities.

[0190] In step 306, when the intelligent assistant module of the control system provides services, based on the user's input natural language, the generative AI engine interface in step 305 is called, and based on the user's input original question, the large model outputs the pending answer A1. In the process of outputting the A1 answer, it is not simply calling the large model, but using multiple calls to the large model to confirm the understanding ability of the large model to avoid the illusion problem of the large model itself and improve the accuracy. The sub-module steps in the large model are as shown in Figure 11

[0191] Unlike general patents that only generate answers based on large models, the 7 processes of the sub-module not only ensure that the large model truly understands the question query to avoid nonsense, but also maximize the use of context history to accurately understand user intent.

[0192] Short-term memory: In the short-term memory module, the historical dialogue is directly used as the history parameter;

[0193] Long truncation: When the user's complete multi-round historical dialogue exceeds the maximum Token length limit, the truncation process is performed from near to far.

[0194] ​Long-term memory: Call the large model for the content of the truncated part, and create a concise summary based on the prompt (e.g. create a concise summary according to the following dialogue, focusing on the key information to be remembered and potential important information. The summary remains concise, controlled within 400 words.) dialogue summary, constitute long-term memory.

[0195] Intention clarification: In order to ensure the accuracy of the reply content, when the user's intention is not clear, the user needs to be asked based on the prompt (e.g. based on the following dialogue and subsequent questions, restate the subsequent questions as independent questions with accurate intentions. If you do not understand or are not sure about the user's true intention, please ask the user to get their true intention. Let's solve this problem step by step to ensure that we get the correct answer) Ask the user to clarify the intention to get the accurate user intention. When the user's intention is clear, the user's question needs to be rewritten as an independent question with an accurate intention.

[0196] Semantic retrieval: When the service starts, vectorize the knowledge base and build an index, and perform vector retrieval to get the top K most similar user questions when called.

[0197] Answer generation: Pass the question and answer pair obtained by semantic retrieval as Context information into the large model to get the initial answer.

[0198] Hallucination detection: Call the large model based on the prompt (e.g. as a reliable artificial intelligence assistant. When judging the authenticity of the given answer, you can refer to the provided reasoning if necessary. But need to be vigilant about the possible errors in the provided reasoning. When something does not conform to the facts, multiple evidence must be provided to support the decision) to perform hallucination detection to prevent it from generating hallucination content.

[0199] At the same time as the large model outputs the final result, it also calls the large model based on the prompt for information extraction to output information extraction results B1.

[0200] For example:

[0201] User question: What single disk supports "supports undervoltage detection, anti-momentary interruption and momentary interruption detection" and has a port rate greater than or equal to 50G in SPN?

[0202] Call the generative AI engine interface in step 305, call the large model based on the original question of the user's question to output the pending answer A1: 690 LFADA02, LFADA04, LFADA94, LFADA08.

[0203] Call the large model based on the prompt for information extraction to output information extraction results B1:

[0204] Network type: SPN;

[0205] Device type: single disk;

[0206] Detection attribute: over / under voltage detection, anti-momentary, momentary detection;

[0207] Port rate: greater than or equal to 50G;

[0208] In step 307, the information extraction result B1 output in step 306 is called as input to query the pending answer A2 based on the information extraction result by performing knowledge reasoning on the triple knowledge base in the knowledge graph engine in step 305.

[0209] For example: B1: network type: SPN;

[0210] Device type: single disk;

[0211] Detection attribute: over / under voltage detection, anti-momentary, momentary detection;

[0212] Port rate: greater than or equal to 50G;

[0213] B1 is called as input to query the pending answer A2: LFADA02, LFADA04, LFADA08 by performing knowledge reasoning on the triple knowledge base in the knowledge graph engine in step 305.

[0214] In step 308, the pending answer A1 in step 306 and the pending answer A2 in step 307 are called as input to generate answer A3 by the result optimization module to output the final answer of the intelligent assistant to the user.

[0215] For example:

[0216] A1: LFADA02, LFADA04, LFADA94, LFADA08 of 690;

[0217] A2: LFADA02, LFADA04, LFADA08;

[0218] The result optimization module is called to generate answer A3 to output to the user: LFADA02, LFADA04, LFADA08 of 690.

[0219] In step 309, after the user obtains the final answer in step 308, the user can provide the correct answer to the feedback engine through the feedback button of the page, and the feedback engine updates the triple knowledge base in the knowledge graph engine combined with the information extraction result in step 306.

[0220] For example: after the user confirms that the answer is correct, the correct answer is provided to the feedback engine, and the information extraction result in step 306 is updated to the triple knowledge base.

[0221] For example:

[0222] Step 306 information extraction result:

[0223] Network type: SPN;

[0224] Device type: single disk;

[0225] Detection attribute: overvoltage detection, anti-momentary interruption, momentary interruption detection;

[0226] Port rate: greater than or equal to 50G;

[0227] Supplement update triple knowledge base: detection attribute: overvoltage detection, anti-momentary interruption, momentary interruption detection;

[0228] In step 310, after updating the triple knowledge base in step 309, the automatic script automatically generates a question and answer data training set, and the system triggers the call regularly to generate the AI engine using the training set to fine-tune the large model in the network communication field online to improve the generative AI capability and continue to provide services in step 306.

[0229] For example: supplement update triple knowledge base: detection attribute: overvoltage detection, anti-momentary interruption, momentary interruption detection.

[0230] Automatically generate question and answer data set:

[0231] Question: What are the detection attributes of LFADA02, LFADA04, and LFADA08 of 690?

[0232] Answer: Overvoltage detection, anti-momentary interruption, and momentary interruption detection.

[0233] Regularly fine-tune and optimize the deployed large model online.

[0234] Embodiment 6:

[0235] As shown in Figure 13 , it is an architecture schematic diagram of the intelligent interactive question and answer device for management and control according to the embodiment of the application. The intelligent interactive question and answer device for management and control according to the embodiment of the application comprises one or more processors 21 and a memory 22. Among them, Figure 13 The processor 21 is taken as an example.

[0236] The processor 21 and the memory 22 can be connected through a bus or other ways, Figure 13 The connection through the bus is taken as an example.

[0237] Memory 22, as a non-volatile computer-readable storage medium, can be used to store non-volatile software programs and non-volatile computer executable programs, such as the intelligent interactive question-and-answer method for management and control in Examples 1 and 3. Processor 21 executes the intelligent interactive question-and-answer method for management and control by running the non-volatile software programs and instructions stored in memory 22.

[0238] The memory 22 may include high-speed random access memory and non-volatile memory, such as at least one disk storage device, flash memory device, or other non-volatile solid-state memory device. In some embodiments, the memory 22 may optionally include a memory remotely located relative to the processor 21, and such remote memory may be connected to the processor 21 via a network. Examples of such networks include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0239] The program instructions / modules are stored in the memory 22, and when executed by the one or more processors 21, the intelligent interactive question-answering method for management and control in the above embodiment 1 is executed, for example, the above described Figures 2-4 , Figure 6 and Figure 7 The steps shown.

[0240] It is worth noting that the information interaction, execution process, etc. between the modules and units within the above-mentioned devices and systems are based on the same concept as the processing method embodiment of the present invention. The specific content can be found in the description of the method embodiment of the present invention and will not be repeated here.

[0241] Those skilled in the art will understand that all or part of the steps in the various methods of the embodiments can be completed by instructing related hardware through a program, and the program can be stored in a computer-readable storage medium, which may include: read-only memory (ROM), random access memory (RAM), a disk or an optical disk, etc.

[0242] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A management and control intelligent interactive question-answering method, characterized in that: include: Build a network communication knowledge base and train the general large model into a large model in the network communication field based on the network communication knowledge base; The network communication domain large model is used to infer the natural language input by the user and output the pending answer A1. The information of the natural language input by the user is extracted based on the prompt word to obtain the information extraction result. The knowledge inference is performed based on the information extraction result and the network communication knowledge base to output the pending answer A2. The pending answers A1 and A2 are subjected to controllable constraints to generate the final answer A3.

2. The intelligent interactive question-answering method for management and control according to claim 1, characterized in that: The method also includes: After obtaining the final answer, the user provides the correct answer to the feedback engine through the feedback button on the page. The feedback engine combines the information extraction results to update the network communication knowledge base in the knowledge graph engine; After the network communication knowledge base is updated, a question-and-answer data training set is automatically generated, and the generative AI engine is regularly triggered to use the training set to perform online fine-tuning training on the large model in the network communication field to enhance the generative AI capabilities.

3. The intelligent interactive question-answering method for management and control according to claim 1, characterized in that: The construction of the network communication knowledge base specifically includes: Construct a network communication knowledge graph and define its ontology and schema. The general large model is called to extract information from network communication related technical text files according to the ontology relationship and schema form of the network communication knowledge graph to generate a network communication knowledge base composed of triples.

4. The intelligent interactive question-answering method for management and control according to any one of claims 1 to 3, characterized in that: The ontological relationship of the network communication knowledge graph is specifically: a compact triple relationship consisting of a subject, an attributive, and a characteristic item; wherein the content recorded in a fault case is split into one or more triple relationships, and there is an overlap of at least one content among the subject, the attributive, and the characteristic item between adjacent triple relationships.

5. The intelligent interactive question-answering method for management and control according to claim 1, characterized in that: The method of training a general large model into a large model in the field of network communication based on the network communication knowledge base specifically includes: Automatically generate question-answering data training sets based on network communication knowledge base; The question-answer data training set is used as input, and the general large model is fine-tuned and trained into a large model in the network communication field.

6. The intelligent interactive question-answering method for management and control according to claim 5, characterized in that: The fine-tuning and training of the general large model into a large model in the network communication field also includes: We use the large model to transform different questions and supplement sample data for data enhancement. We then perform three-stage fine-tuning training based on the sample data to improve the model's capabilities in the communications field. The fine-tuned and optimized large model in the network communication field is then used to further expand the triple knowledge base.

7. The intelligent interactive question-answering method for management and control according to claim 2 or 5, characterized in that: The automatic generation of the question-answering data training set specifically includes: In a self-questioning and self-answering manner, two elements in the triplet are inserted into the question template, and the third element is used as the answer to establish a question-answering data training set.

8. The intelligent interactive question-answering method for management and control according to any one of claims 1 to 4, characterized in that: The calling of the network communication domain large model to infer the natural language input by the user and output the pending answer A1 specifically includes: Short-term memory, used to directly use historical conversations as historical parameters; Overlength truncation: When the length of a complete historical conversation contained in the history parameter exceeds the maximum length specified by the token, the complete historical conversation is truncated from the most recent to the most recent time. Long-term memory, used to summarize the truncated content based on prompts to form long-term memory; Intent clarification is used to clarify the user's intention based on prompts when the user's intention is unclear, in order to obtain the accurate user intention. When the user's intention is clear, the question parameters in the historical conversation in the history parameters are rewritten to restate the question into an independent question with accurate intention.

9. The intelligent interactive question-answering method for management and control according to claim 8, characterized in that: The method of calling a large network communication domain model to infer the natural language input by the user and outputting a pending answer A1 also includes: Semantic retrieval is used to vectorize the knowledge base and build an index when the service starts, and then perform vector retrieval to obtain the most similar user questions when the service is called; Answer generation, which uses the question-answer pairs obtained by semantic retrieval as contextual information to obtain the initial answer; Hallucination detection is used to perform hallucination detection on the initial answer based on the prompt to prevent the initial answer from containing hallucination content. When judging the authenticity of a given answer, at least one new evidence is required to support the authenticity of the answer.

10. A management and control intelligent interactive question-answering device, characterized in that: The device comprises: At least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the processor to execute the intelligent interactive question-and-answer method for management and control described in any one of claims 1-9.

11. A management and control intelligent interactive question-answering system, characterized in that: It includes a network communication field large model generation module, a first pending answer generation module, a second pending answer generation module and a final answer generation module. Among them, the network communication field large model generation module is respectively connected to the data of the first pending answer generation module and the second pending answer generation module, and the output ends of the first pending answer generation module and the second pending answer generation module data are also respectively connected to the final answer generation module data. Specifically: The network communication domain large model generation module is used to build a network communication knowledge base and train the general large model into a network communication domain large model based on the network communication knowledge base; The first pending answer generation module is used to call the large model in the network communication field to infer the natural language input by the user and output the pending answer A1; The second pending answer generation module extracts information from the natural language input by the user based on the prompt word to obtain the information extraction result; performs knowledge reasoning based on the information extraction result and the network communication knowledge base, and outputs the pending answer A2; The final answer generation module is used to generate the final answer A3 by applying controllable constraints to the pending answer A1 and the pending answer A2.

Images