Operation and maintenance management method and device based on big language model questions and answers and electronic equipment

Abstract

The invention provides an operation and maintenance management method and device based on big language model questions and answers and electronic equipment, and relates to the technical field of computers.The method comprises the steps that under the condition that an operation and maintenance fault occurs in the operation and maintenance management process, an operation and maintenance work order is generated based on the operation and maintenance fault; determining an operation and maintenance fault problem based on the operation and maintenance work order; inputting the operation and maintenance fault problem into the large language model to obtain a prediction answer; matching the predicted answer with a target reference answer corresponding to a target reference question in an operation and maintenance management question and answer knowledge base; and when the matching is successful, outputting a prediction answer for indicating to repair the operation and maintenance fault. When the operation and maintenance fault occurs, the prediction answer corresponding to the operation and maintenance fault question is output through the large language model, the prediction answer is matched with the target reference answer in the operation and maintenance management question and answer knowledge base, and the prediction answer is output only when matching succeeds, so that the accuracy of knowledge questions and answers is improved; and the accuracy of repairing the operation and maintenance fault based on the prediction answer is further improved.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to an operation and maintenance management method, device and electronic equipment based on large language model question and answer. Background Art

[0002] With the rapid advancement of computer technology, information networks have become a key pillar of social development. In the new era of cloud-network integration, the competitiveness of operators increasingly depends on the flexibility and agility of network services. Using large models to promote the development of intelligent applications has become an irreversible trend. Faced with the huge scale of current intelligent network operation and maintenance, it is difficult to effectively cope with the increasingly complex operation and maintenance challenges by relying solely on manual experience and automated operation and maintenance management methods that rely on work order systems. Therefore, agent-based question-answering systems are gradually emerging and are widely used in areas such as work order management, customer after-sales service, enterprise information technology (IT) support, and call centers to efficiently create, manage, and resolve various transaction requests from users, customers, partners, and internal employees of the enterprise. In the future, the standardization, unification, and clarification of transaction processing will be undertaken more by agents. This transformation is not only an inevitable result of technological progress, but also a key path to improve service efficiency and quality.

[0003] In the related art, a user's question is usually input into a large language model, and the large language model directly outputs the answer corresponding to the question input by the user.

[0004] However, in the above-mentioned related technologies, it is easy for the answer output by the large language model to be different from the answer the user wants, thereby reducing the accuracy of the knowledge question and answer. Summary of the invention

[0005] The present invention provides an operation and maintenance management method, device and electronic device based on large language model question and answer, which are used to solve the defect of reducing the accuracy of knowledge question and answer in the prior art.

[0006] The present invention provides an operation and maintenance management method based on large language model question answering, comprising: In the event of an operation and maintenance failure during the operation and maintenance management process, an operation and maintenance work order is generated based on the operation and maintenance failure; Determine operation and maintenance fault issues based on operation and maintenance work orders; Inputting the operation and maintenance fault problem into a large language model to obtain a predicted answer output by the large language model; In the case where there is a target reference question matching the operation and maintenance fault question in the operation and maintenance management question and answer knowledge base, matching the predicted answer with the target reference answer corresponding to the target reference question in the operation and maintenance management question and answer knowledge base to obtain a matching result; In the case where the matching result indicates a successful match, the predicted answer is output, and the predicted answer is used to instruct to repair the operation and maintenance fault.

[0007] According to an operation and maintenance management method based on large language model question answering provided by the present invention, the method further includes: When there is no target reference question matching the operation and maintenance fault question in the operation and maintenance management question and answer knowledge base, outputting the predicted answer; receiving feedback information from the user regarding the predicted answer; When the feedback information indicates that the predicted answer is correct, the correspondence between the predicted answer and the operation and maintenance fault problem is updated in the operation and maintenance management question and answer knowledge base.

[0008] According to an operation and maintenance management method based on large language model question answering provided by the present invention, the method further includes: When the matching result indicates a matching failure, outputting the predicted answer and the target reference answer; receiving selection information of the user with respect to the predicted answer and the target reference answer; In a case where the selection information includes the predicted answer, the predicted answer is associated with the target reference question in the operation and maintenance management question and answer knowledge base.

[0009] According to an operation and maintenance management method based on large language model question answering provided by the present invention, the large language model is trained based on the following method: constructing an initial student model; Inputting a first sample operation and maintenance fault problem into the initial student model and the teacher model to obtain a first sample answer output by the initial student model and a second sample answer output by the teacher model; Constructing first loss information based on the first sample answer and the second sample answer; Constructing second loss information based on the first sample answer and the answer label of the first sample operation and maintenance failure question; Based on the first loss information and the second loss information, adjusting the model parameters of the initial student model to obtain the student model; The student model is determined as the large language model.

[0010] According to an operation and maintenance management method based on large language model question answering provided by the present invention, the teacher model is trained based on the following method: Constructing an initial teacher model; Inputting the second sample operation and maintenance fault problem into the initial teacher model to obtain a third sample answer output by the initial teacher model; Constructing third loss information based on the third sample answer and the answer label of the second sample operation and maintenance failure question; Based on the third loss information, the model parameters of the initial teacher model are adjusted to obtain the teacher model.

[0011] According to an operation and maintenance management method based on large language model question answering provided by the present invention, the method further includes: Acquire real-time operation data during operation and maintenance management based on preset cycles; Determine first operation data whose change amount is less than a preset value and second operation data whose change amount is greater than or equal to the preset value in the real-time operation data; Collect the first operation data based on a preset interval to obtain third operation data; determining the second operation data and the third operation data as target operation data; Extracting key data from the target operation data; Generate reference questions and reference answers corresponding to the reference questions based on the key data; The operation and maintenance management question and answer knowledge base is constructed based on the reference questions and the reference answers corresponding to the reference questions.

[0012] According to an operation and maintenance management method based on large language model question and answer provided by the present invention, the step of generating reference questions and reference answers corresponding to the reference questions based on the key data includes: Inputting the key data into a text error correction model to obtain corrected key data output by the text error correction model; The reference question and a reference answer corresponding to the reference question are generated based on the corrected key data.

[0013] The present invention also provides an operation and maintenance management device based on large language model question answering, comprising: A first generating unit is used to generate an operation and maintenance work order based on the operation and maintenance failure when an operation and maintenance failure occurs during the operation and maintenance management process; A first determination unit, used to determine the operation and maintenance fault problem based on the operation and maintenance work order; A prediction unit, used for inputting the operation and maintenance fault problem into a large language model to obtain a prediction answer output by the large language model; a matching unit, configured to match the predicted answer with a target reference answer corresponding to the target reference question in the operation and maintenance management question and answer knowledge base to obtain a matching result when there is a target reference question matching the operation and maintenance fault question in the operation and maintenance management question and answer knowledge base; The first output unit is used to output the predicted answer when the matching result indicates a successful match, and the predicted answer is used to indicate that the operation and maintenance fault is to be repaired.

[0014] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, an operation and maintenance management method based on large language model question and answer as described above is implemented.

[0015] The present invention also provides a non-transitory computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, it implements the operation and maintenance management method based on large language model question and answer as described in any of the above.

[0016] The present invention also provides a computer program product, including a computer program, which, when executed by a processor, implements any of the above-mentioned operation and maintenance management methods based on large language model question and answer.

[0017] The operation and maintenance management method, device and electronic device based on large language model question and answer provided by the present invention, in the case of an operation and maintenance failure during the operation and maintenance management process, determine the operation and maintenance failure problem based on the operation and maintenance failure, input the operation and maintenance failure problem into the large language model, obtain the predicted answer output by the large language model, and in the case that there is a target reference question matching the operation and maintenance failure problem in the operation and maintenance management question and answer knowledge base, match the predicted answer with the target reference answer corresponding to the target reference question in the operation and maintenance management question and answer knowledge base, and when the matching result indicates a successful match, output the predicted answer again, and repair the operation and maintenance failure through the predicted answer. It can be seen that the present invention can output the predicted answer corresponding to the operation and maintenance failure problem through the large language model when an operation and maintenance failure occurs during the operation and maintenance management process, and match the predicted answer with the target reference answer corresponding to the target reference question matching the operation and maintenance failure problem in the operation and maintenance management question and answer knowledge base, and output the predicted answer only when the match is successful, thereby improving the accuracy of the knowledge question and answer of the operation and maintenance management, and further improving the accuracy of repairing the operation and maintenance failure based on the predicted answer. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0019] Figure 1 This is one of the flow charts of the operation and maintenance management method based on large language model question and answer provided by an embodiment of the present invention.

[0020] Figure 2 This is the second flow chart of the operation and maintenance management method based on large language model question and answer provided in an embodiment of the present invention.

[0021] Figure 3 This is the third flow chart of the operation and maintenance management method based on large language model question and answer provided in an embodiment of the present invention.

[0022] Figure 4 It is a schematic diagram of the knowledge distillation compression framework provided by an embodiment of the present invention.

[0023] Figure 5 This is the fourth flow chart of the operation and maintenance management method based on large language model question and answer provided in an embodiment of the present invention.

[0024] Figure 6 Schematic diagram of the structure of the DE-CO model provided by the embodiment of the present invention.

[0025] Figure 7 It is a structural diagram of an operation and maintenance management system based on large language model question and answer provided by an embodiment of the present invention.

[0026] Figure 8 It is a structural diagram of an operation and maintenance management device based on large language model question and answer provided by an embodiment of the present invention.

[0027] Fig. 9 It is a schematic diagram of the physical structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0028] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0029] Combine the following Figure 1-Figure 7 The operation and maintenance management method based on large language model question and answer of the present invention is described. The execution subject of the operation and maintenance management method based on large language model question and answer can be an electronic device such as a terminal, a computer or a server, or it can be an operation and maintenance management device based on large language model question and answer set in the electronic device. The operation and maintenance management device based on large language model question and answer can be implemented by software, hardware or a combination of both.

[0030] Figure 1 is one of the flow charts of the operation and maintenance management method based on large language model question answering provided by an embodiment of the present invention, such as Figure 1As shown, the operation and maintenance management method based on large language model question answering includes the following steps: Step 101: When an operation and maintenance failure occurs during the operation and maintenance management process, an operation and maintenance work order is generated based on the operation and maintenance failure.

[0031] For example, when an operation and maintenance failure occurs during the operation and maintenance management process, the user can describe the operation and maintenance failure, obtain operation and maintenance failure description information, generate an operation and maintenance work order based on the operation and maintenance failure description information, and upload the operation and maintenance work order to an electronic device loaded with the operation and maintenance management method, so that the electronic device receives the operation and maintenance work order.

[0032] Step 102: Determine the operation and maintenance fault problem based on the operation and maintenance work order.

[0033] For example, when an operation and maintenance work order is received, the operation and maintenance work order is input into the problem generation model, and the content of the operation and maintenance work order is summarized by the problem generation model to obtain the operation and maintenance fault problem. For example, if it is found that the processor keeps alarming during the operation and maintenance management process, the processor alarming is regarded as an operation and maintenance fault, and the operation and maintenance fault problem generated based on the operation and maintenance fault can be "Why does the processor keep alarming?"

[0034] It should be noted that the question generation model can also generate operation and maintenance fault questions based on historical question-answer pairs and description information of operation and maintenance faults to improve the accuracy of operation and maintenance fault question generation, and the present invention is not limited to this.

[0035] Step 103: input the operation and maintenance fault problem into the large language model to obtain a predicted answer output by the large language model.

[0036] For example, an operation and maintenance fault problem is input into the large language model, the large language model searches for information related to the operation and maintenance fault problem, and determines the searched related information as a predicted answer to the operation and maintenance fault problem.

[0037] Step 104: When there is a target reference question matching the operation and maintenance fault problem in the operation and maintenance management question and answer knowledge base, the predicted answer is matched with the target reference answer corresponding to the target reference question in the operation and maintenance management question and answer knowledge base to obtain a matching result.

[0038] For example, all reference questions and reference answers in the operation and maintenance management process are pre-organized, and an operation and maintenance management question and answer knowledge base is pre-built based on the correspondence between the reference questions and the reference answers. When the predicted answer output by the large language model is obtained, the operation and maintenance fault problem is matched with the reference question in the operation and maintenance management question and answer knowledge base. When it is determined that there is a target reference question matching the operation and maintenance fault problem in the operation and maintenance management question and answer knowledge base, the predicted answer is matched with the target reference answer corresponding to the target reference question to obtain a matching result.

[0039] It should be noted that one reference answer may correspond to multiple semantically similar reference questions. Of course, a reference question may also correspond to multiple semantically similar reference answers, and the present invention does not limit this.

[0040] Step 105: When the matching result indicates a successful match, output the predicted answer, where the predicted answer is used to instruct to repair the operation and maintenance fault.

[0041] For example, when the matching result represents that the predicted answer matches the target reference answer successfully, it means that the predicted answer output by the large language model is the correct answer, and the predicted answer is output to facilitate the user to repair the operation and maintenance fault based on the predicted answer. The operation and maintenance fault problem is "Why does the processor keep alarming?" and the corresponding predicted answer is "The processor keeps alarming, which may be caused by excessive temperature. Please try to increase the fan power to cool the processor." When the matching result represents that the predicted answer fails to match the target reference answer, it means that the predicted answer output by the large language model is not in the operation and maintenance management question and answer knowledge base. At this time, a prompt message can be output, and the prompt message is used to prompt the user to re-enter the operation and maintenance fault problem.

[0042] The operation and maintenance management method based on large language model question and answer provided by the present invention, in the case of an operation and maintenance failure in the operation and maintenance management process, determines the operation and maintenance failure problem based on the operation and maintenance failure, inputs the operation and maintenance failure problem into the large language model, obtains the predicted answer output by the large language model, and in the case of a target reference question matching the operation and maintenance failure problem in the operation and maintenance management question and answer knowledge base, matches the predicted answer with the target reference answer corresponding to the target reference question in the operation and maintenance management question and answer knowledge base, and outputs the predicted answer when the matching result indicates a successful match, and repairs the operation and maintenance failure through the predicted answer. It can be seen that the present invention can output the predicted answer corresponding to the operation and maintenance failure problem through the large language model when an operation and maintenance failure occurs in the operation and maintenance management process, and matches the predicted answer with the target reference answer corresponding to the target reference question matching the operation and maintenance failure problem in the operation and maintenance management question and answer knowledge base, and outputs the predicted answer only when the match is successful, thereby improving the accuracy of the knowledge question and answer of the operation and maintenance management, and further improving the accuracy of repairing the operation and maintenance failure based on the predicted answer.

[0043] In one embodiment, Figure 2 This is a second flow chart of the operation and maintenance management method based on large language model question answering provided by an embodiment of the present invention, such as Figure 2 As shown, after the above step 103, the operation and maintenance management method based on large language model question answering further includes the following steps: Step 106: If there is no target reference question matching the operation and maintenance fault question in the operation and maintenance management question and answer knowledge base, output the predicted answer.

[0044] For example, when it is determined that there is no target reference question matching the operation and maintenance fault problem in the operation and maintenance management question and answer knowledge base, it means that the input operation and maintenance fault problem is a new problem. At this time, the predicted answer is directly output and the feedback request information is output. For example, the feedback request information can be "Are you satisfied with the output predicted answer?"

[0045] Step 107: Receive feedback information from the user regarding the predicted answer.

[0046] For example, when the user receives feedback request information, if the user is satisfied with the output predicted answer, the input feedback information may be information indicating satisfaction; if the user is dissatisfied with the output predicted answer, the input feedback information may be information indicating dissatisfaction.

[0047] Step 108: When the feedback information indicates that the predicted answer is correct, the corresponding relationship between the predicted answer and the operation and maintenance fault problem is updated in the operation and maintenance management question and answer knowledge base.

[0048] For example, when it is determined that the feedback information input by the user is information representing satisfaction, it means that the predicted answer output by the large language model is correct. At this time, the correspondence between the predicted answer and the operation and maintenance fault problem is updated to the operation and maintenance management question and answer knowledge base to enrich the question and answer pairs in the operation and maintenance management question and answer knowledge base.

[0049] It should be noted that when the feedback information represents that the predicted answer is correct, the operation and maintenance fault problem can also be synonymously transformed to obtain multiple semantically similar operation and maintenance fault problems, and then the correspondence between the multiple semantically similar operation and maintenance fault problems and the predicted answers are stored in the operation and maintenance management question and answer knowledge base, so that when relevant operation and maintenance fault problems are input later, the predicted answers output by the large language model can be verified based on the reference answers corresponding to the operation and maintenance fault problems in the operation and maintenance management question and answer knowledge base, thereby further improving the accuracy of knowledge questions and answers.

[0050] In this embodiment, when there is no target reference question matching the operation and maintenance fault problem in the operation and maintenance management question and answer knowledge base, a predicted answer is output. When the feedback information input by the user for the predicted answer indicates that the predicted answer is correct, the correspondence between the predicted answer and the operation and maintenance fault problem is updated to the operation and maintenance management question and answer knowledge base, thereby realizing automatic updating of the operation and maintenance management question and answer knowledge base, enriching the question and answer pairs in the operation and maintenance management question and answer knowledge base, and facilitating the subsequent input of related operation and maintenance fault problems. The predicted answer output by the large language model can be verified based on the reference answer corresponding to the operation and maintenance fault problem in the operation and maintenance management question and answer knowledge base, thereby further improving the accuracy of the knowledge questions and answers of operation and maintenance management.

[0051] In one embodiment, Figure 3This is a flowchart of the operation and maintenance management method based on large language model question answering provided by an embodiment of the present invention. Figure 3 As shown, after the above step 104, the operation and maintenance management method based on large language model question answering further includes the following steps: Step 109: When the matching result indicates a matching failure, output the predicted answer and the target reference answer.

[0052] For example, when the matching result indicates that the predicted answer fails to match the target reference answer, it means that the predicted answer output by the large language model does not exist in the operation and maintenance management question and answer knowledge base. At this time, the predicted answer and the target reference answer are output at the same time.

[0053] Step 110: Receive the user's selection information regarding the predicted answer and the target reference answer.

[0054] For example, when the predicted answer and the target reference answer are output simultaneously, a selection request for the predicted answer and the target reference answer is also output, and the selection request is used to request the user to select a satisfactory answer from the predicted answer and the target reference answer.

[0055] Step 111: When the selection information includes the predicted answer, the predicted answer is associated with the target reference question in the operation and maintenance management question and answer knowledge base.

[0056] For example, when it is determined that the received selection information includes a predicted answer, it is determined that the user believes that the predicted answer is the correct answer. At this time, since the operation and maintenance management question and answer knowledge base includes target reference questions that match the operation and maintenance fault problems, the predicted answer can be directly associated with the target reference questions in the operation and maintenance management question and answer knowledge base, and there is no need to store the operation and maintenance fault problems in the operation and maintenance management question and answer knowledge base.

[0057] In this embodiment, when the matching result indicates a matching failure, the predicted answer and the target reference answer are output, and the user's selection information for the predicted answer and the target reference answer is received. When the selection information includes the predicted answer, the predicted answer and the target reference question are associated in the operation and maintenance management question and answer knowledge base, thereby enriching the answer information corresponding to the target reference question in the operation and maintenance management question and answer knowledge base, so that when the subsequent user inputs an operation and maintenance fault problem related to the target reference question, the predicted answer output by the large language model can be verified based on all the answers corresponding to the target reference question in the operation and maintenance management question and answer knowledge base, thereby further improving the accuracy of the knowledge question and answer.

[0058] In one embodiment, the large language model is trained based on the following method: Construct an initial student model; input a first sample operation and maintenance fault problem into the initial student model and the teacher model to obtain a first sample answer output by the initial student model and a second sample answer output by the teacher model; construct first loss information based on the first sample answer and the second sample answer; construct second loss information based on the first sample answer and the answer label of the first sample operation and maintenance fault problem; adjust model parameters of the initial student model based on the first loss information and the second loss information to obtain the student model; determine the student model as the large language model.

[0059] For example, Figure 4 is a schematic diagram of a knowledge distillation compression framework provided by an embodiment of the present invention, such as Figure 4 As shown in Figure 1, the knowledge distillation compression framework includes a teacher model and a student model. The training of the student model has two objectives: one is the original objective function, also called the hard target, which is the cross entropy between the category probability output of the student model and the true label; the other is the soft target, which is the cross entropy between the category probability output of the student model and the category probability output of the teacher model. The soft target is obtained based on the output of the original softmax function, and a temperature constant T is added to control the smoothness of the predicted probability. The adjusted softmax function is expressed by the following formula (1): Among them, z is the logical value logits, is the corresponding value of the i-th category in logits, is the first The corresponding values ​​of the categories, represents an exponential function, and the loss function L usually uses the relative entropy (Kullback-Leibler, KL) divergence to calculate the difference. T usually takes an integer value greater than 1, at which time the difference in the predicted values ​​of the target class and the non-target class is reduced, and logits are "softened". On the contrary, when T is less than 1, the numerical difference between the target class and the non-target class will be further widened, and logits tend to be one-hot encoded vectors. The classification prediction result refers to the output of the last fully connected layer of the classifier (called logits).

[0060] During the operation and maintenance management process, an initial student model is first constructed, and multiple first sample operation and maintenance fault problems in the operation and maintenance management process are obtained, and the multiple first sample operation and maintenance fault problems are input into the initial student model and the trained teacher model to obtain first sample answers corresponding to each first sample operation and maintenance fault problem output by the initial student model and second sample answers to each first sample operation and maintenance fault problem output by the teacher model, and first loss information is constructed based on each first sample answer and each corresponding second sample answer; second loss information is constructed based on each first sample answer and the answer label of each first sample operation and maintenance fault problem; the first loss information and the second loss information are fused to obtain total loss information, and the model parameters of the initial student model are adjusted based on the total loss information until the convergence condition is reached, and finally a trained student model is obtained, and the finally trained student model is determined as a large language model.

[0061] In this embodiment, since the teacher model is a large and complex model that has been trained and has excellent performance, the output of the teacher model is regarded as the accurate answer, and the student model is a smaller model with fewer parameters. The student model is trained by the teacher model, and the goal is to improve the accuracy of the student model output by imitating the output of the teacher model. Therefore, when the student model is used as a large language model, it can improve the efficiency of the large language model in outputting answers, and can also improve the accuracy of the large language model's output answers.

[0062] In one embodiment, the teacher model is trained based on the following method: Construct an initial teacher model; input a second sample operation and maintenance fault problem into the initial teacher model to obtain a third sample answer output by the initial teacher model; construct third loss information based on the third sample answer and the answer label of the second sample operation and maintenance fault problem; based on the third loss information, adjust the model parameters of the initial teacher model to obtain the teacher model.

[0063] For example, first, an initial teacher model is constructed, and multiple second sample operation and maintenance failure problems in the operation and maintenance management process are obtained. The second sample operation and maintenance failure problems may be the same as or different from the first sample operation and maintenance failure problems. The multiple second sample operation and maintenance failure problems are input into the initial teacher model to obtain third sample answers corresponding to each second sample operation and maintenance failure problem output by the initial teacher model. Third loss information is constructed based on each third sample answer and the answer label of each second sample operation and maintenance failure problem. The model parameters of the initial teacher model are adjusted based on the third loss information until the convergence conditions are met, and finally a trained teacher model is obtained.

[0064] In this embodiment, a teacher model is pre-trained based on multiple second sample operation and maintenance fault problems in the operation and maintenance management process. Since the teacher model is a large and complex model with excellent performance, directly using the teacher model as a large language model will lead to a decrease in the efficiency of knowledge question answering. Therefore, a student model with a smaller scale and fewer parameters is subsequently trained based on the trained teacher model, and the answer output by the student model is made close to the correct answer output by the teacher model, thereby improving the efficiency and accuracy of knowledge question answering.

[0065] In one embodiment, Figure 5 This is a fourth flow chart of the operation and maintenance management method based on large language model question answering provided by an embodiment of the present invention, such as Figure 5 As shown, before the above step 101, the operation and maintenance management method based on large language model question answering also includes the following steps: Step 501: Acquire real-time operation data in the operation and maintenance management process based on a preset period.

[0066] For example, real-time operation data in the operation and maintenance management process is obtained at preset intervals, and all the obtained real-time operation data is stored. The real-time operation data in the operation and maintenance management process may include equipment main indicator data, log data, configuration information, third-party related data, business related data, equipment status data and network security data, etc. Among them, the equipment main indicator data can be obtained through commands, log data, calling service interfaces, etc., and various performance indicators of the equipment are collected, such as port historical traffic, disk occupancy, instantaneous traffic, central processing unit (CPU) occupancy, graphics processing unit (GPU) occupancy, memory occupancy and number of connections, etc. Log data includes log data of various services of the big data platform, such as alarm duration, alarm level, etc. These log data record the operation status and error information of the system in operation and maintenance management; configuration information involves real-time update and monitoring of configuration information such as system, service, host, etc.; third-party related data includes monitoring system, security system, etc.; business related data, for example, in the e-commerce system of operation and maintenance management, business related data may include key business indicators such as order volume, payment amount, and user visit volume. Taking EMU operation and maintenance as an example, equipment status data includes vehicle speed, positioning information, traction, voltage and current, etc.; network security data includes network performance indicators such as network bandwidth, latency, packet loss rate, as well as security-related data such as security incidents and attack attempts.

[0067] Step 502: Determine first operating data whose variation is less than a preset value and second operating data whose variation is greater than or equal to the preset value in the real-time operating data.

[0068] The preset value may be set based on demand, and the present invention does not limit this.

[0069] For example, when real-time operation data is obtained, the change in the real-time operation data is analyzed. If the change in a part of the real-time operation data within a period of time is less than a preset value, it means that the fluctuation of this part of the data is small, and this part of the data is used as the first operation data with small fluctuation. If the change in another part of the real-time operation data within a period of time is greater than or equal to the preset value, it means that the fluctuation of this other part of the data is large, and this other part of the data is used as the second operation data with large fluctuation.

[0070] Step 503: Collect the first operation data based on a preset interval to obtain third operation data.

[0071] For example, since the fluctuation of the first operation data is small, in order to reduce the size of the data, the quadratic interpolation method can be used to make a difference between every three adjacent points, that is, the first operation data is collected based on a preset interval, so that the data volume of the third operation data obtained by the interval collection is smaller than the data volume of the first operation data. The length of the preset interval can be N, and the specific value of N can be set based on demand. The advantage of doing quadratic interpolation is that the interval is uniform, which is more compatible with the subsequent Transformer model time series processing, and can also restore the missing data of the specific scene more realistically. The specific quadratic interpolation can be expressed by the following formula (2): in, =The current value of the classification object. The classification object refers to each type of real-time running data. = 3 adjacent points of the classification object, =The sequence number of the classification object.

[0072] Step 504: Determine the second operating data and the third operating data as target operating data.

[0073] For example, when the second operating data and the third operating data collected at intervals are obtained, the second operating data and the third operating data are fused to obtain the target operating data.

[0074] Step 505: extract key data from the target operation data.

[0075] For example, the target operation data is encoded so that the Transformer model can process it. The target operation data can be converted into an input representation acceptable to the Transformer model using techniques such as word embedding and position encoding. Select a Transformer model suitable for operation and maintenance data, such as Bidirectional Encoder Representations from Transformers (BERT) and Generative Pre-trained Transformer (GPT). Use a labeled data set to train the model so that it can learn the ability to extract key information from input data. Use the trained Transformer model to process the target operation data and extract key data. These key data can include system performance indicators (such as CPU usage, memory occupancy, etc.), error logs, alarm data, false alarm data, and data with large fluctuations. The Transformer model is a neural network model based on the self-attention mechanism, which is used to process sequence data. Compared with the traditional recurrent neural network model, the Transformer model has better parallel performance and shorter training time, so it has been widely used in the field of natural language processing.

[0076] Step 506: Generate reference questions and reference answers corresponding to the reference questions based on the key data.

[0077] For example, when key data is obtained, possible reference questions are generated in combination with the experience and knowledge of operation and maintenance management. These reference questions involve multiple aspects such as system performance, error troubleshooting, and user behavior analysis. For each reference question, the generation capability of the Transformer model or other knowledge bases are used to generate corresponding reference answers. For example, the target operation data is that the CPU usage of an e-commerce platform suddenly increases to more than 90% and lasts for more than 10 minutes. The key data extracted is that the CPU usage rate increases abnormally, which may be caused by reasons such as excessive system load, insufficient memory, or malicious attacks. The generated reference questions can be: Why does the CPU usage rate of the e-commerce platform suddenly increase? How to solve the problem of excessive CPU usage of the e-commerce platform? The reference answer can be: The sudden increase in the CPU usage rate of the e-commerce platform may be caused by reasons such as excessive system load, insufficient memory, or malicious attacks. It is recommended to check the system resource usage, optimize the code and configuration, or strengthen security protection measures. The following measures can be taken to solve the problem of excessive CPU usage of the e-commerce platform: optimize the code and database query to reduce unnecessary resource consumption; increase server resources such as CPU and memory; strengthen security protection measures to prevent malicious attacks and malicious access.

[0078] Step 507: construct the operation and maintenance management question and answer knowledge base based on the reference questions and the reference answers corresponding to the reference questions.

[0079] For example, when each reference question and a reference answer corresponding to each reference question are obtained, each reference question and a reference answer corresponding to each reference question are stored in a database to obtain an operation and maintenance management question and answer knowledge base.

[0080] In this embodiment, it is possible to determine target operating data based on real-time operating data in the operation and maintenance management process, extract key data from the target operating data, and generate reference questions and reference answers corresponding to the reference questions based on the key data. Then, an operation and maintenance management question and answer knowledge base is constructed based on the reference questions and the reference answers corresponding to the reference questions, thereby realizing the automatic construction of an operation and maintenance management question and answer knowledge base.

[0081] In one embodiment, the above step 506 generates reference questions and reference answers corresponding to the reference questions based on the key data, which can be specifically implemented in the following manner: The key data is input into a text error correction model to obtain the corrected key data output by the text error correction model; and the reference question and the reference answer corresponding to the reference question are generated based on the corrected key data.

[0082] The text error correction model may be a spelling correction model (Detector-Corrector, DE-CO), which corrects misplaced information caused by potentially hidden abnormal instructions.

[0083] For example, Figure 6 is a schematic diagram of the structure of the DE-CO model provided in an embodiment of the present invention, such as Figure 6 As shown in the figure, the DE-CO model consists of a misspelled word identifier and a misspelled word corrector. The function of the misspelled word identifier is to identify misspelled words in adversarial samples and provide these misspelled words to the misspelled word corrector for correction, while ensuring that the correctly spelled words in the adversarial samples are not modified. The function of the misspelled word corrector is to correct the misspelled words based on the spelling information and context information of the misspelled words. When the DE-CO model is applied to the error correction of key data, the key data is input into the DE-CO model to obtain the corrected key data output by the DE-CO model, and then the reference questions and the reference answers corresponding to the reference questions are generated based on the corrected key data.

[0084] In this embodiment, key data is input into a text error correction model for error correction, and reference questions and reference answers corresponding to the reference questions are generated based on the corrected key data, so that the generated reference questions and reference answers are more accurate, thereby improving the accuracy of the operation and maintenance management question and answer knowledge base, and further improving the accuracy of the operation and maintenance management knowledge question and answer.

[0085] Figure 7 is a schematic diagram of the structure of an operation and maintenance management system based on large language model question answering provided by an embodiment of the present invention, such as Figure 7 As shown, the operation and maintenance management system based on large language model question and answer includes a question collection device, a question and answer module including an operation and maintenance management question and answer knowledge base, a large language model and a question answering device. Among them, the question collection device is used to receive operation and maintenance fault problems, and input the operation and maintenance fault problems into the large language model, obtain the predicted answers output by the large language model, extract keywords from the operation and maintenance fault problems, match the extracted keywords with the reference keywords of the reference questions in the operation and maintenance management question and answer knowledge base, and when a reference keyword matching the keyword is found in the operation and maintenance management question and answer knowledge base, it indicates that there is a target reference question matching the operation and maintenance fault problem in the operation and maintenance management question and answer knowledge base, then the target reference answer of the reference question to which the matched reference keyword belongs is matched with the predicted answer to obtain a matching result, and when the matching result indicates a successful match, the predicted answer is sent to the question answering device, and the predicted answer is displayed by the question answering device, and the predicted answer is used to indicate that the operation and maintenance fault is to be repaired.

[0086] The large language model in the present invention is obtained by training the initial student model based on the trained teacher model. The knowledge distillation compression algorithm is combined with the operation and maintenance fault problems input at the question and answer request layer in the large model architecture, and semantics are accurately extracted and predicted at the large model analysis layer to obtain the predicted answer. The predicted answer is further verified based on the operation and maintenance management question and answer knowledge base, so that the answer output by the operation and maintenance management system based on the large language model question and answer is more accurate.

[0087] The operation and maintenance management device based on large language model question and answer provided by the present invention is described below. The operation and maintenance management device based on large language model question and answer described below and the operation and maintenance management method based on large language model question and answer described above can be referenced to each other.

[0088] Figure 8 is a structural diagram of an operation and maintenance management device based on large language model question answering provided by an embodiment of the present invention, such as Figure 8 As shown, the operation and maintenance management device 800 based on large language model question answering includes a first generation unit 801, a first determination unit 802, a prediction unit 803, a matching unit 804 and a first output unit 805; wherein: The first generating unit 801 is used to generate an operation and maintenance work order based on the operation and maintenance failure when an operation and maintenance failure occurs during the operation and maintenance management process; The first determining unit 802 is used to determine the operation and maintenance fault problem based on the operation and maintenance work order; A prediction unit 803, configured to input the operation and maintenance fault problem into a large language model to obtain a prediction answer output by the large language model; A matching unit 804 is used to match the predicted answer with the target reference answer corresponding to the target reference question in the operation and maintenance management question and answer knowledge base to obtain a matching result when there is a target reference question matching the operation and maintenance fault question in the operation and maintenance management question and answer knowledge base; The first output unit 805 is used to output the predicted answer when the matching result indicates a successful match.

[0089] The operation and maintenance management device based on large language model question and answer provided by the present invention, in the case of an operation and maintenance failure during the operation and maintenance management process, determines the operation and maintenance failure problem based on the operation and maintenance failure, inputs the operation and maintenance failure problem into the large language model, obtains the predicted answer output by the large language model, and in the case of a target reference question matching the operation and maintenance failure problem in the operation and maintenance management question and answer knowledge base, matches the predicted answer with the target reference answer corresponding to the target reference question in the operation and maintenance management question and answer knowledge base, and outputs the predicted answer when the matching result indicates a successful match, and repairs the operation and maintenance failure through the predicted answer. It can be seen that the present invention can output the predicted answer corresponding to the operation and maintenance failure problem through the large language model when an operation and maintenance failure occurs during the operation and maintenance management process, and matches the predicted answer with the target reference answer corresponding to the target reference question matching the operation and maintenance failure problem in the operation and maintenance management question and answer knowledge base, and outputs the predicted answer only when the match is successful, thereby improving the accuracy of the knowledge question and answer of the operation and maintenance management, and further improving the accuracy of repairing the operation and maintenance failure based on the predicted answer.

[0090] Based on any of the above embodiments, the operation and maintenance management device 800 based on large language model question and answer further includes: A second output unit, configured to output the predicted answer when there is no target reference question matching the operation and maintenance fault question in the operation and maintenance management question and answer knowledge base; A first receiving unit, configured to receive feedback information from the user regarding the predicted answer; An updating unit is used to update the correspondence between the predicted answer and the operation and maintenance fault problem into the operation and maintenance management question and answer knowledge base when the feedback information indicates that the predicted answer is correct.

[0091] Based on any of the above embodiments, the operation and maintenance management device 800 based on large language model question and answer further includes: A third output unit, configured to output the predicted answer and the target reference answer when the matching result indicates a matching failure; A second receiving unit, configured to receive selection information of the user with respect to the predicted answer and the target reference answer; An associating unit is used to associate the predicted answer with the target reference question in the operation and maintenance management question and answer knowledge base when the selection information includes the predicted answer.

[0092] Based on any of the above embodiments, the large language model is trained in the following manner: constructing an initial student model; Inputting a first sample operation and maintenance fault problem into the initial student model and the teacher model to obtain a first sample answer output by the initial student model and a second sample answer output by the teacher model; Constructing first loss information based on the first sample answer and the second sample answer; Constructing second loss information based on the first sample answer and the answer label of the first sample operation and maintenance failure question; Based on the first loss information and the second loss information, adjusting the model parameters of the initial student model to obtain the student model; The student model is determined as the large language model.

[0093] Based on any of the above embodiments, the teacher model is trained in the following manner: Constructing an initial teacher model; Inputting the second sample operation and maintenance fault problem into the initial teacher model to obtain a third sample answer output by the initial teacher model; Constructing third loss information based on the third sample answer and the answer label of the second sample operation and maintenance failure question; Based on the third loss information, the model parameters of the initial teacher model are adjusted to obtain the teacher model.

[0094] Based on any of the above embodiments, the operation and maintenance management device 800 based on large language model question and answer further includes: An acquisition unit, used to acquire real-time operation data in the operation and maintenance management process based on a preset period; A second determining unit, configured to determine, in the real-time operating data, first operating data whose variation is less than a preset value and second operating data whose variation is greater than or equal to a preset value; a collecting unit, configured to collect the first operation data based on a preset interval to obtain third operation data; a third determining unit, configured to determine the second operating data and the third operating data as target operating data; An extraction unit, used to extract key data from the target operation data; A second generating unit, configured to generate reference questions and reference answers corresponding to the reference questions based on the key data; A construction unit is used to construct the operation and maintenance management question and answer knowledge base based on the reference questions and the reference answers corresponding to the reference questions.

[0095] Based on any of the above embodiments, the generating unit is specifically used for: Inputting the key data into a text error correction model to obtain corrected key data output by the text error correction model; The reference question and a reference answer corresponding to the reference question are generated based on the corrected key data.

[0096] Fig. 9 is a schematic diagram of the physical structure of an electronic device provided by an embodiment of the present invention, such as Fig. 9 As shown, the electronic device may include: a processor 910, a communication interface 920, a memory 930 and a communication bus 940, wherein the processor 910, the communication interface 920 and the memory 930 communicate with each other through the communication bus 940. The processor 910 may call the logic instructions in the memory 930 to execute the operation and maintenance management method based on large language model question answering, the method comprising: in the event of an operation and maintenance failure during the operation and maintenance management process, generating an operation and maintenance work order based on the operation and maintenance failure; Determine operation and maintenance fault issues based on operation and maintenance work orders; Inputting the operation and maintenance fault problem into a large language model to obtain a predicted answer output by the large language model; In the case where there is a target reference question matching the operation and maintenance fault question in the operation and maintenance management question and answer knowledge base, matching the predicted answer with the target reference answer corresponding to the target reference question in the operation and maintenance management question and answer knowledge base to obtain a matching result; In the case where the matching result indicates a successful match, the predicted answer is output, and the predicted answer is used to instruct to repair the operation and maintenance fault.

[0097] In addition, the logic instructions in the above-mentioned memory 930 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on this understanding, the technical solution of the present invention can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0098] On the other hand, the present invention further provides a computer program product, the computer program product includes a computer program, the computer program can be stored on a non-transitory computer-readable storage medium, when the computer program is executed by a processor, the computer can execute the operation and maintenance management method based on large language model question answering provided by the above methods, the method comprising: in the event of an operation and maintenance failure occurring during the operation and maintenance management process, generating an operation and maintenance work order based on the operation and maintenance failure; Determine operation and maintenance fault issues based on operation and maintenance work orders; Inputting the operation and maintenance fault problem into a large language model to obtain a predicted answer output by the large language model; In the case where there is a target reference question matching the operation and maintenance fault question in the operation and maintenance management question and answer knowledge base, matching the predicted answer with the target reference answer corresponding to the target reference question in the operation and maintenance management question and answer knowledge base to obtain a matching result; In the case where the matching result indicates a successful match, the predicted answer is output, and the predicted answer is used to instruct to repair the operation and maintenance fault.

[0099] In another aspect, the present invention further provides a non-transitory computer-readable storage medium having a computer program stored thereon, which is implemented when the computer program is executed by a processor to execute the operation and maintenance management method based on large language model question answering provided by the above methods, the method comprising: in the event of an operation and maintenance failure occurring during the operation and maintenance management process, generating an operation and maintenance work order based on the operation and maintenance failure; Determine operation and maintenance fault issues based on operation and maintenance work orders; Inputting the operation and maintenance fault problem into a large language model to obtain a predicted answer output by the large language model; In the case where there is a target reference question matching the operation and maintenance fault question in the operation and maintenance management question and answer knowledge base, matching the predicted answer with the target reference answer corresponding to the target reference question in the operation and maintenance management question and answer knowledge base to obtain a matching result; In the case where the matching result indicates a successful match, the predicted answer is output, and the predicted answer is used to instruct to repair the operation and maintenance fault.

[0100] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0101] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0102] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. An operation and maintenance management method based on large language model question answering, characterized in that: include: In the event of an operation and maintenance failure during the operation and maintenance management process, an operation and maintenance work order is generated based on the operation and maintenance failure; Determine operation and maintenance fault issues based on operation and maintenance work orders; Inputting the operation and maintenance fault problem into a large language model to obtain a predicted answer output by the large language model; In the case where there is a target reference question matching the operation and maintenance fault question in the operation and maintenance management question and answer knowledge base, matching the predicted answer with the target reference answer corresponding to the target reference question in the operation and maintenance management question and answer knowledge base to obtain a matching result; In the case where the matching result indicates a successful match, the predicted answer is output, and the predicted answer is used to instruct to repair the operation and maintenance fault.

2. The operation and maintenance management method based on large language model question answering according to claim 1 is characterized in that: The method further comprises: When there is no target reference question matching the operation and maintenance fault question in the operation and maintenance management question and answer knowledge base, outputting the predicted answer; receiving feedback information from a user regarding the predicted answer; When the feedback information indicates that the predicted answer is correct, the correspondence between the predicted answer and the operation and maintenance fault problem is updated in the operation and maintenance management question and answer knowledge base.

3. The operation and maintenance management method based on large language model question answering according to claim 1 is characterized in that: The method further comprises: When the matching result indicates a matching failure, outputting the predicted answer and the target reference answer; Receiving selection information of the user with respect to the predicted answer and the target reference answer; In a case where the selection information includes the predicted answer, the predicted answer is associated with the target reference question in the operation and maintenance management question and answer knowledge base.

4. The operation and maintenance management method based on large language model question answering according to claim 1 is characterized in that: The large language model is trained based on the following method: constructing an initial student model; Inputting a first sample operation and maintenance fault problem into the initial student model and the teacher model to obtain a first sample answer output by the initial student model and a second sample answer output by the teacher model; Constructing first loss information based on the first sample answer and the second sample answer; Constructing second loss information based on the first sample answer and the answer label of the first sample operation and maintenance failure question; Based on the first loss information and the second loss information, adjusting the model parameters of the initial student model to obtain the student model; The student model is determined as the large language model.

5. The operation and maintenance management method based on large language model question answering according to claim 4 is characterized in that: The teacher model is trained based on the following method: Constructing an initial teacher model; Inputting the second sample operation and maintenance fault problem into the initial teacher model to obtain a third sample answer output by the initial teacher model; Constructing third loss information based on the third sample answer and the answer label of the second sample operation and maintenance failure question; Based on the third loss information, the model parameters of the initial teacher model are adjusted to obtain the teacher model.

6. The operation and maintenance management method based on large language model question answering according to any one of claims 1 to 5, characterized in that: The method further comprises: Acquire real-time operation data during operation and maintenance management based on preset cycles; Determine first operation data whose change amount is less than a preset value and second operation data whose change amount is greater than or equal to the preset value in the real-time operation data; Collect the first operation data based on a preset interval to obtain third operation data; determining the second operation data and the third operation data as target operation data; Extracting key data from the target operation data; Generate reference questions and reference answers corresponding to the reference questions based on the key data; The operation and maintenance management question and answer knowledge base is constructed based on the reference questions and the reference answers corresponding to the reference questions.

7. The operation and maintenance management method based on large language model question answering according to claim 6 is characterized in that: The generating of reference questions and reference answers corresponding to the reference questions based on the key data includes: Inputting the key data into a text error correction model to obtain corrected key data output by the text error correction model; The reference question and the reference answer corresponding to the reference question are generated based on the corrected key data.

8. An operation and maintenance management device based on large language model question answering, characterized in that: include: A first generating unit is used to generate an operation and maintenance work order based on the operation and maintenance failure when an operation and maintenance failure occurs during the operation and maintenance management process; A first determination unit, used to determine the operation and maintenance fault problem based on the operation and maintenance work order; A prediction unit, used for inputting the operation and maintenance fault problem into a large language model to obtain a prediction answer output by the large language model; a matching unit, configured to match the predicted answer with a target reference answer corresponding to the target reference question in the operation and maintenance management question and answer knowledge base to obtain a matching result when there is a target reference question matching the operation and maintenance fault question in the operation and maintenance management question and answer knowledge base; The first output unit is used to output the predicted answer when the matching result indicates a successful match, and the predicted answer is used to indicate that the operation and maintenance fault is to be repaired.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the operation and maintenance management method based on large language model question and answer as described in any one of claims 1 to 7 is implemented.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the operation and maintenance management method based on large language model question and answering is implemented as described in any one of claims 1 to 7.

Images