Large model fault prediction method
Through big data and machine learning methods, pre-training models are used to fine-tune and train the historical alarms and fault information of server nodes, which realizes automatic fault prediction and early warning of operation and maintenance systems, and solves the problem that traditional methods cannot meet the accuracy of fault prediction of complex systems and Real-time issues significantly reduce operation and maintenance costs and business risks.
Patent Information
- Application Number
- CN202410986900.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-23
- Publication Date
- 2024-12-03
AI Technical Summary
Traditional operation and maintenance fault prediction methods rely on manual experience and rules, and are difficult to meet the requirements of complex systems for accuracy and real-time failure prediction, and cannot effectively prevent and warning system failures.
Using big data and machine learning methods, by collecting historical alarm information and fault information of server nodes, cleaning, integrating and standardizing processing, forming a data set, fine-tuning training is used for pre-training models, and performing fault prediction and inference tasks regularly to provide fault warning result.
It realizes automated fault prediction and early warning, reduces the burden on operation and maintenance personnel, avoids damage to the system and downtime caused by failures, and reduces operation and maintenance costs and business risks.
Smart Images

Figure CN119067248A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of large model operation and maintenance technology, and in particular to a large model fault prediction method. Background Art
[0002] With the rapid development of information technology, operation and maintenance systems are becoming increasingly complex, and the prediction and troubleshooting of system failures have become an important part of operation and maintenance work. Theoretically, any unsafe accident can be prevented, and behind every serious accident, there must be a large number of minor accidents, precursors to attempted accidents, and potential accidents. Traditional operation and maintenance fault prediction methods mainly rely on manual experience and rules, which are difficult to meet the requirements of complex systems for fault prediction accuracy and real-time performance. Therefore, developing a large model fault prediction method based on big data and machine learning is of great significance to improving operation and maintenance efficiency.
[0003] Based on the above situation, the present invention proposes a large model fault prediction method. Summary of the invention
[0004] In order to make up for the defects of the prior art, the present invention provides a simple and efficient large model fault prediction method.
[0005] The present invention is achieved through the following technical solutions:
[0006] A large model fault prediction method comprises the following steps:
[0007] Step S1, collecting historical alarm information and fault information of each server node in the area;
[0008] Step S2, cleaning, integrating and standardizing the collected alarm information and fault information to form a data set required for fine-tuning the large model;
[0009] Step S3, selecting a pre-trained model according to GPU resources, fine-tuning the pre-trained model using the data set, and outputting the fine-tuned model;
[0010] Step S4, using the fine-tuned model to perform fault prediction reasoning tasks on a daily basis;
[0011] Step S5: Send the model inference results to the operation and maintenance personnel in the corresponding area and prepare for the next fault prediction.
[0012] In step S1, the historical alarm information includes but is not limited to the node IP, alarm name and alarm time, and the fault information includes but is not limited to the time when the fault occurred.
[0013] The implementation process of step S2 is as follows:
[0014] Step S2.1: In the acquired historical alarm information, if any of the three fields, namely, node IP, alarm name or alarm time, contain null values, the corresponding records are deleted;
[0015] Step S2.2, sort the acquired historical alarm information from recent to early according to the time of fault occurrence. If there are multiple records with fault occurrence time on the same day, only the record of the earliest fault occurrence on that day is retained, and the rest of the records are deleted;
[0016] Step S2.3, create a hash table for storing alarm information, the key of the hash table is a string type, specifically a literal value of a date information of a certain day, the format is: 20xx-xx-xx, and all subsequent dates are strings of this format; the value corresponding to the key in the hash table is a list type, which is used to store one or more alarm information generated on the corresponding date;
[0017] Obtain and traverse the alarm information of the latest three years stored in the database, and put the alarm information into the hash table; if the list corresponding to the key of the date in the hash table exists, directly put the corresponding alarm information into the list, otherwise initialize it to an empty list first, and then put the corresponding alarm information into the list;
[0018] Step S2.4, create a hash table for storing fault information, the key of the hash table is a string type, specifically a literal value of a certain date information, the format is: 20xx-xx-xx, all subsequent dates are strings in this format; the key of the hash table is the date of a certain day, the value corresponding to the key is an integer type, 0 represents no fault on that date, 1 represents a fault on that date;
[0019] Obtain and traverse the fault information of the last three years stored in the database, put the fault information into the hash table, and set the value corresponding to the date of the fault to 1;
[0020] Step S2.5, create a list for storing question-answer pairs, and then traverse the date set of the last three years in chronological order; for each date, check whether there is a corresponding value in the corresponding hash table storing alarm information:
[0021] If it does not exist, skip this traversal;
[0022] Otherwise, create a question-answer pair and put it into the list corresponding to the hash table;
[0023] In step S2.6, the created question-answer pair list is serialized into json format and saved into a file as a dataset for fine-tuning the pre-trained model.
[0024] In step S2.5, the specific steps of creating a question-answer pair include:
[0025] Step S2.5.1, construct the question in the question-answer pair;
[0026] Use each date traversed in step S2.5 as a key, query the corresponding value in the hash table that stores the alarm information, that is, the alarm information list, serialize it into a json string, and then concatenate the following content: "You are a professional operation and maintenance engineer. The following json array is a list of alarm information for each server node in a certain area today. Please predict whether there will be a failure in this area tomorrow based on the json array. The json array is: {json_array}"; after the concatenation is completed, use the concatenated content as the question in the question-answer pair, where {json_array} is the alarm information list after being serialized into a json string;
[0027] Step S2.5.2, constructing the answer in the question-answer pair;
[0028] Use the date value of the day after each date traversed in step S2.5 as the key to query whether there is a corresponding value in the hash table that stores the fault information:
[0029] If it exists, set the answer to the question and answer pair to 1;
[0030] Otherwise, set the correct answer to 0.
[0031] In step S2.6, the json format is: [{"prompt":"Question 1","chosen":"Answer 1","rejected":""},{"prompt":"Question 2","chosen":"Answer 2","rejected":""}], where the value of prompt is the question in the question-answer pair, the value of chosen is the answer corresponding to the question, and the value of rejected is set to an empty string.
[0032] In step S3, if the GPU resource is a single A800 graphics card, the Baichuan2-13B-Chat open source large model is selected;
[0033] Then, select the Deepspeed-Chat open source tool, set the training step parameter to 1, set the parameter of the distributed training model to single card, and change the model path to the path location of the Baichuan2-13B-Chat open source large model, and change the dataset path to the json file location saved in step S2.6. The remaining parameters are default.
[0034] Finally, start the Deepspeed-Chat model training script, perform supervised fine-tuning training, and wait for the training to complete and output the log "exits successfully."
[0035] The implementation process of step S4 is as follows:
[0036] Step S4.1, use the trained model path as the path of the inference task model, load the model and initialize the inference task;
[0037] Step S4.2, load the json file saved in step S2.6 as a question-answer pair array, traverse the question-answer pair array, and take out a question-answer pair with an answer of 0 and an answer of 1 as the conversation context for each question to the model;
[0038] Step S4.3, query the alarm information of each server node in the area yesterday at 0:00 every day:
[0039] If there is no alarm information, the subsequent steps are skipped and the result is directly returned: "Since no alarm information was generated in this area yesterday, the big model predicts that no failure will occur today";
[0040] If there is alarm information, save it in a list, then serialize it into a json string, and then construct the question in the following format: "You are a professional operation and maintenance engineer. The following json array is a list of alarm information of each server node in a certain area today. Please predict whether there will be a failure in this area tomorrow based on the json array. The json array is: {json_array}", where {json_array} is the alarm information list after serialization into a json string;
[0041] Step S4.4, input the question context constructed in step S4.2 and the question constructed in step S4.3 to the big model to ask questions, and wait for the big model to complete reasoning and return the answer.
[0042] The implementation process of step S5 is as follows:
[0043] Step S5.1, map the answer returned by the big model into specific results and send them to the relevant operation and maintenance personnel in the corresponding area:
[0044] If the answer is 0, the result returned is: "The large model predicts that no failure will occur today";
[0045] If the answer is 1, the result returned is: "The big model predicts that a failure will occur today";
[0046] Step S5.2, clear the large model conversation context to ensure the independence of the prediction results when the fault prediction task is executed next time.
[0047] A large model fault prediction device, characterized in that it includes a memory and a processor; the memory is used to store a computer program, and the processor is used to implement the above method steps when executing the computer program.
[0048] A readable storage medium, characterized in that: a computer program is stored on the readable storage medium, and the computer program implements the above method steps when executed by a processor.
[0049] The beneficial effects of the present invention are: the large-model fault prediction method realizes automated fault prediction and early warning, can reduce the workload of operation and maintenance personnel, avoid damage and downtime caused by faults to the system, reduce business interruptions and losses caused by faults, thereby significantly reducing operation and maintenance costs and business risks. BRIEF DESCRIPTION OF THE DRAWINGS
[0050] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0051] Attached Figure 1 It is a schematic diagram of the large model fault prediction method of the present invention. DETAILED DESCRIPTION
[0052] In order to enable those skilled in the art to better understand the technical solutions in the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.
[0053] The large model fault prediction method comprises the following steps:
[0054] Step S1, collecting historical alarm information and fault information of each server node in the area;
[0055] Step S2, cleaning, integrating and standardizing the collected alarm information and fault information to form a data set required for fine-tuning the large model;
[0056] Step S3, selecting a pre-trained model according to GPU resources, fine-tuning the pre-trained model using the data set, and outputting the fine-tuned model;
[0057] Step S4, using the fine-tuned model to perform fault prediction reasoning tasks on a daily basis;
[0058] Step S5: Send the model inference results to the operation and maintenance personnel in the corresponding area and prepare for the next fault prediction.
[0059] In step S1, the historical alarm information includes but is not limited to the node IP, alarm name and alarm time, and the fault information includes but is not limited to the time when the fault occurred.
[0060] The implementation process of step S2 is as follows:
[0061] Step S2.1: In the acquired historical alarm information, if any of the three fields, namely, node IP, alarm name or alarm time, contain null values, the corresponding records are deleted;
[0062] Step S2.2, sort the acquired historical alarm information from recent to early according to the time of fault occurrence. If there are multiple records with fault occurrence time on the same day, only the record of the earliest fault occurrence on that day is retained, and the rest of the records are deleted;
[0063] Step S2.3, create a hash table for storing alarm information, the key of the hash table is a string type, specifically a literal value of a date information of a certain day, the format is: 20xx-xx-xx, and all subsequent dates are strings of this format; the value corresponding to the key in the hash table is a list type, which is used to store one or more alarm information generated on the corresponding date;
[0064] Obtain and traverse the alarm information of the latest three years stored in the database, and put the alarm information into the hash table; if the list corresponding to the key of the date in the hash table exists, directly put the corresponding alarm information into the list, otherwise initialize it to an empty list first, and then put the corresponding alarm information into the list;
[0065] Step S2.4, create a hash table for storing fault information, the key of the hash table is a string type, specifically a literal value of a certain date information, the format is: 20xx-xx-xx, all subsequent dates are strings in this format; the key of the hash table is the date of a certain day, the value corresponding to the key is an integer type, 0 represents no fault on that date, 1 represents a fault on that date;
[0066] Obtain and traverse the fault information of the last three years stored in the database, put the fault information into the hash table, and set the value corresponding to the date of the fault to 1;
[0067] Step S2.5, create a list for storing question-answer pairs, and then traverse the date set of the last three years in chronological order; for each date, check whether there is a corresponding value in the corresponding hash table storing alarm information:
[0068] If it does not exist, skip this traversal;
[0069] Otherwise, create a question-answer pair and put it into the list corresponding to the hash table;
[0070] In step S2.6, the created question-answer pair list is serialized into json format and saved into a file as a dataset for fine-tuning the pre-trained model.
[0071] In step S2.5, the specific steps of creating a question-answer pair include:
[0072] Step S2.5.1, construct the question in the question-answer pair;
[0073] Use each date traversed in step S2.5 as a key, query the corresponding value in the hash table that stores the alarm information, that is, the alarm information list, serialize it into a json string, and then concatenate the following content: "You are a professional operation and maintenance engineer. The following json array is a list of alarm information for each server node in a certain area today. Please predict whether there will be a failure in this area tomorrow based on the json array. The json array is: {json_array}"; after the concatenation is completed, use the concatenated content as the question in the question-answer pair, where {json_array} is the alarm information list after being serialized into a json string;
[0074] Step S2.5.2, constructing the answer in the question-answer pair;
[0075] Use the date value of the day after each date traversed in step S2.5 as the key to query whether there is a corresponding value in the hash table that stores the fault information:
[0076] If it exists, set the answer to the question and answer pair to 1;
[0077] Otherwise, set the correct answer to 0.
[0078] In step S2.6, the json format is: [{"prompt":"Question 1","chosen":"Answer 1","rejected":""},{"prompt":"Question 2","chosen":"Answer 2","rejected":""}], where the value of prompt is the question in the question-answer pair, the value of chosen is the answer corresponding to the question, and the value of rejected is set to an empty string.
[0079] In step S3, if the GPU resource is a single A800 graphics card, the Baichuan2-13B-Chat open source large model is selected;
[0080] Then, select the Deepspeed-Chat open source tool, set the training step parameter to 1, set the parameter of the distributed training model to single card, and change the model path to the path location of the Baichuan2-13B-Chat open source large model, and change the dataset path to the json file location saved in step S2.6. The remaining parameters are default.
[0081] Finally, start the Deepspeed-Chat model training script, perform supervised fine-tuning training, and wait for the training to complete and output the log "exits successfully."
[0082] The implementation process of step S4 is as follows:
[0083] Step S4.1, use the trained model path as the path of the inference task model, load the model and initialize the inference task;
[0084] Step S4.2, load the json file saved in step S2.6 as a question-answer pair array, traverse the question-answer pair array, and take out a question-answer pair with an answer of 0 and an answer of 1 as the conversation context for each question to the model;
[0085] Step S4.3, query the alarm information of each server node in the area yesterday at 0:00 every day:
[0086] If there is no alarm information, the subsequent steps are skipped and the result is directly returned: "Since no alarm information was generated in this area yesterday, the big model predicts that no failure will occur today";
[0087] If there is alarm information, save it in a list, then serialize it into a json string, and then construct the question in the following format: "You are a professional operation and maintenance engineer. The following json array is a list of alarm information of each server node in a certain area today. Please predict whether there will be a failure in this area tomorrow based on the json array. The json array is: {json_array}", where {json_array} is the alarm information list after serialization into a json string;
[0088] Step S4.4, input the question context constructed in step S4.2 and the question constructed in step S4.3 to the big model to ask questions, and wait for the big model to complete reasoning and return the answer.
[0089] The implementation process of step S5 is as follows:
[0090] Step S5.1, map the answer returned by the big model into specific results and send them to the relevant operation and maintenance personnel in the corresponding area:
[0091] If the answer is 0, the result returned is: "The large model predicts that no failure will occur today";
[0092] If the answer is 1, the result returned is: "The big model predicts that a failure will occur today";
[0093] Step S5.2, clear the large model conversation context to ensure the independence of the prediction results when the fault prediction task is executed next time.
[0094] The large model fault prediction device comprises a memory and a processor; the memory is used to store a computer program, and the processor is used to implement the above method steps when executing the computer program.
[0095] The readable storage medium stores a computer program, and when the computer program is executed by a processor, the above method steps are implemented.
[0096] This large-model fault prediction method can avoid damage and downtime caused by faults to the system by predicting and handling faults in advance, reduce business interruptions and losses caused by faults, and thus significantly reduce operation and maintenance costs and business risks. At the same time, it provides an automated fault prediction and early warning system that can reduce the workload of operation and maintenance personnel, allowing them to focus on solving more complex problems, thereby improving overall operation and maintenance efficiency and work quality.
[0097] The embodiment described above is only one specific implementation of the present invention. Common changes and substitutions made by those skilled in the art within the scope of the technical solution of the present invention should be included in the protection scope of the present invention.
Claims
1. A large model fault prediction method, characterized by: The following steps are involved: Step S1, collecting historical alarm information and fault information of each server node in the area; Step S2, cleaning, integrating and standardizing the collected alarm information and fault information to form a data set required for fine-tuning the large model; Step S3, selecting a pre-trained model according to GPU resources, fine-tuning the pre-trained model using the data set, and outputting the fine-tuned model; Step S4, using the fine-tuned model to perform fault prediction reasoning tasks on a daily basis; Step S5: Send the model inference results to the operation and maintenance personnel in the corresponding area and prepare for the next fault prediction.
2. The large model fault prediction method according to claim 1, characterized in that: In step S1, the historical alarm information includes but is not limited to the node IP, alarm name and alarm time, and the fault information includes but is not limited to the time when the fault occurred.
3. The large model fault prediction method according to claim 1, characterized in that: The implementation process of step S2 is as follows: Step S2.1: In the acquired historical alarm information, if any of the three fields, namely, node IP, alarm name or alarm time, contain null values, the corresponding records are deleted; Step S2.2, sort the acquired historical alarm information from recent to early according to the time of fault occurrence. If there are multiple records with fault occurrence time on the same day, only the record of the earliest fault occurrence on that day is retained, and the rest of the records are deleted; Step S2.3, create a hash table for storing alarm information, the key of the hash table is a string type, specifically a literal value of a date information of a certain day, the format is: 20xx-xx-xx, and all subsequent dates are strings of this format; the value corresponding to the key in the hash table is a list type, which is used to store one or more alarm information generated on the corresponding date; Obtain and traverse the alarm information of the latest three years stored in the database, and put the alarm information into the hash table; if the list corresponding to the key of the date in the hash table exists, directly put the corresponding alarm information into the list, otherwise initialize it to an empty list first, and then put the corresponding alarm information into the list; Step S2.4, create a hash table for storing fault information, the key of the hash table is a string type, specifically a literal value of a certain date information, the format is: 20xx-xx-xx, all subsequent dates are strings in this format; the key of the hash table is the date of a certain day, the value corresponding to the key is an integer type, 0 represents no fault on that date, 1 represents a fault on that date; Obtain and traverse the fault information of the last three years stored in the database, put the fault information into the hash table, and set the value corresponding to the date of the fault to 1; Step S2.5, create a list for storing question-answer pairs, and then traverse the date set of the last three years in chronological order; for each date, check whether there is a corresponding value in the corresponding hash table storing alarm information: If it does not exist, skip this traversal; Otherwise, create a question-answer pair and put it into the list corresponding to the hash table; In step S2.6, the created question-answer pair list is serialized into json format and saved into a file as a dataset for fine-tuning the pre-trained model.
4. The large model fault prediction method according to claim 3 is characterized in that: In step S2.5, the specific steps of creating a question-answer pair include: Step S2.5.1, construct the question in the question-answer pair; Use each date traversed in step S2.5 as a key, query the corresponding value in the hash table that stores the alarm information, that is, the alarm information list, serialize it into a json string, and then concatenate the following content: "You are a professional operation and maintenance engineer. The following json array is a list of alarm information for each server node in a certain area today. Please predict whether there will be a failure in this area tomorrow based on the json array. The json array is: {json_array}"; after the concatenation is completed, use the concatenated content as the question in the question-answer pair, where {json_array} is the alarm information list after being serialized into a json string; Step S2.5.2, constructing the answer in the question-answer pair; Use the date value of the day after each date traversed in step S2.5 as the key to query whether there is a corresponding value in the hash table that stores the fault information: If it exists, set the answer to the question and answer pair to 1; Otherwise, set the correct answer to 0.
5. The large model fault prediction method according to claim 3, characterized in that: In step S2.6, the json format is: [{"prompt":"Question 1","chosen":"Answer 1","rejected":""},{"prompt":"Question 2","chosen":"Answer 2","rejected":""}], where the value of prompt is the question in the question-answer pair, the value of chosen is the answer corresponding to the question, and the value of rejected is set to an empty string.
6. The large model fault prediction method according to claim 1, characterized in that: In step S3, if the GPU resource is a single A800 graphics card, the Baichuan2-13B-Chat open source large model is selected; Then, select the Deepspeed-Chat open source tool, set the training step parameter to 1, set the parameter of the distributed training model to single card, and change the model path to the path location of the Baichuan2-13B-Chat open source large model, and change the dataset path to the json file location saved in step S2.
6. The remaining parameters are default. Finally, start the Deepspeed-Chat model training script, perform supervised fine-tuning training, and wait for the training to complete and output the "exits successfully." log.
7. The large model fault prediction method according to claim 4, characterized in that: The implementation process of step S4 is as follows: Step S4.1, use the trained model path as the path of the inference task model, load the model and initialize the inference task; Step S4.2, load the json file saved in step S2.6 as a question-answer pair array, traverse the question-answer pair array, and take out a question-answer pair with an answer of 0 and an answer of 1 as the conversation context for each question to the model; Step S4.3, query the alarm information of each server node in the area yesterday at 0:00 every day: If there is no alarm information, the subsequent steps are skipped and the result is returned directly: "Since no alarm information was generated in this area yesterday, the big model predicts that no failure will occur today"; If there is alarm information, save it in a list, then serialize it into a json string, and then construct the question in the following format: "You are a professional operation and maintenance engineer. The following json array is a list of alarm information for each server node in a certain area today. Please predict whether there will be a failure in the area tomorrow based on the json array. The json array is: {json_array}", where {json_array} is the list of alarm information serialized into a json string; Step S4.4, input the question context constructed in step S4.2 and the question constructed in step S4.3 to the big model to ask questions, and wait for the big model to complete reasoning and return the answer.
8. The large model fault prediction method according to claim 1, characterized in that: The implementation process of step S5 is as follows: Step S5.1, map the answer returned by the big model into specific results and send them to the relevant operation and maintenance personnel in the corresponding area: If the answer is 0, the result returned is: "The large model predicts that no failure will occur today"; If the answer is 1, the result returned is: "The large model predicts that a failure will occur today"; Step S5.2, clear the large model conversation context to ensure the independence of the prediction results when the fault prediction task is executed next time.
9. A large model fault prediction device, characterized in that: The method comprises a memory and a processor; the memory is used to store a computer program, and the processor is used to implement the method according to any one of claims 1 to 8 when executing the computer program.
10. A readable storage medium, characterized in that: The readable storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 8 is implemented.
