A safety-critical software failure mode identification method based on a large language model

Abstract

The application discloses a safety-critical software failure mode identification method based on a large language model, comprising the following steps: constructing a safety-critical field knowledge base, injecting the safety-critical field knowledge base into an initial large language model through a P-Tuning v2 technology to obtain a large language model after first-time fine-tuning; constructing an instruction optimization data set, and performing multi-round iterative training on the large language model after first-time fine-tuning by using the P-Tuning v2 technology to obtain a large language model after second-time fine-tuning; using the large language model after second-time fine-tuning to predict failure modes of industrial failure text data, and based on the failure text data of the prediction failure, constructing a thinking chain data set, and using the thinking chain data set to continue training the large language model after second-time fine-tuning to obtain a safety-critical field specific large language model with ideal final effect; inputting failure text data of a failure mode to be identified into the safety-critical field specific large language model to obtain a failure mode identification result.

Description

Technical Field

[0001] This invention belongs to the field of safety analysis technology for safety-critical software, specifically relating to a method for identifying failure modes of safety-critical software based on a large language model. Background Technology

[0002] Safety-critical software refers to software residing in a safety-critical system whose operation could cause the system to enter a dangerous state, leading to catastrophic consequences such as significant property damage, personal injury, or large-scale environmental damage. Safety-critical software has extremely high requirements for correctness and safety. Through safety analysis, potential hazards and risks can be identified, the software's safety performance can be assessed, its design and operation can be verified to meet relevant safety standards and requirements, comprehensive safety requirements can be generated, and corresponding measures can be taken to improve the reliability and safety of the safety-critical software.

[0003] During the security analysis phase, security engineers need to process the failure text data and extract the corresponding failure modes as input for security analysis. However, as safety-critical software becomes increasingly complex, the applicability of this technique in industry faces significant challenges. Industrial projects are large-scale, containing vast amounts of information and complex patterns, and failure text data is often unstructured. Human processing struggles to handle the ever-increasing volume of text data, failing to meet the demands of large-scale text processing. Furthermore, manually processing massive amounts of text requires extensive division of labor, making it difficult to guarantee consistency and accuracy between the results of different engineers. Therefore, manually processing failure text data is challenging, hindering accurate and efficient information extraction and analysis. Summary of the Invention

[0004] Purpose of the invention: To address the problems of low processing efficiency and low accuracy of existing methods when processing failure text data in safety-critical fields, this invention proposes a failure mode recognition method for safety-critical software based on a large language model. This method can significantly improve the processing efficiency and consistency of processing results for failure text data in safety-critical fields, while reducing labor costs.

[0005] Technical solution: A method for identifying failure modes of safety-critical software based on a large language model, comprising the following steps:

[0006] Step 1: Construct a safety-critical domain knowledge base, which consists of relevant standard documents and common knowledge in the safety-critical domain combined with corresponding manually written prompts; inject the safety-critical domain knowledge base into the large language model through P-Tuning v2 technology to obtain the large language model after the first fine-tuning.

[0007] Step 2: Construct an instruction tuning dataset and use P-Tuning v2 technology to perform multiple rounds of iterative training on the large language model after the first fine-tuning to obtain the large language model after the second fine-tuning; the instruction tuning dataset consists of instruction-response pairs; wherein, invalid text data is used as instructions and invalid patterns are used as responses;

[0008] Step 3: Use the finely tuned large language model to predict failure modes of industrial failure text data, and construct the Mind Chain dataset based on the failure text data that failed to predict. Use the Mind Chain dataset to train the finely tuned large language model to obtain a safety-critical domain-specific large language model with ideal final results.

[0009] Step 4: Input the failure text data of the failure mode to be identified into the proprietary large language model for safety-critical domains to obtain the failure mode identification results.

[0010] Furthermore, in step 1, the construction of the safety-critical domain knowledge base includes the following steps:

[0011] Manually select relevant standard documents and common knowledge in safety-critical areas;

[0012] Match manually selected safety-critical areas with relevant standard documents and common knowledge with manually written prompts;

[0013] Convert the paired content into a JSON string format, as follows:

[0014] {"prompt":"XXXXXXX","knowledge":"XXXXXXXXXXXXXXXXXX"}

[0015] Among them, "prompt" corresponds to the prompt, and "knowledge" corresponds to the standard documents and common knowledge related to safety-critical areas.

[0016] This allows us to construct a knowledge base for safety-critical areas.

[0017] Furthermore, in step 2, the construction of the instruction tuning dataset includes the following steps:

[0018] Obtain failure text data annotated by safety engineers;

[0019] The failure text data annotated by safety engineers is processed into unstructured text data;

[0020] By using unstructured text data as instructions in instruction-response pairs and corresponding failure modes as responses in these pairs, an instruction tuning dataset is constructed.

[0021] Furthermore, the instruction response pair is represented as:

[0022] {"content":"******************","summary":"****************"}

[0023] Here, "content" corresponds to unstructured text data, and "summary" corresponds to the failure mode.

[0024] Furthermore, by adding conjunctions before and after the structured failed text data annotated by safety engineers, unstructured text data is obtained.

[0025] Furthermore, in step 3, the failed text data predicted by the large language model based on secondary fine-tuning are used to construct the thought chain dataset. Specific operations include:

[0026] An example of constructing a thought chain is presented based on the failure to predict invalid text data using a large language model with secondary fine-tuning.

[0027] Using the Mind Chain example, construct a Mind Chain dataset;

[0028] The thought chain example is constructed by analyzing invalid text data and providing a multi-step intermediate reasoning process.

[0029] The intermediate reasoning steps include: identifying the failed object, determining the failure impact, determining the failure cause, and determining the failure mode.

[0030] Beneficial effects: Compared with existing technologies that manually process failed text data and extract safety-critical software failure modes from it, the present invention has the following advantages:

[0031] (1) Automation: Manual methods require a lot of manpower and time costs, while the large language model of this invention can automatically analyze and process a large amount of text data, realize automated failure mode extraction, and greatly improve efficiency;

[0032] (2) Consistency: Manual processing involves a large number of security engineers with varying levels of expertise and different extraction standards, resulting in highly subjective extraction results. In contrast, the large language model of this invention has a consistent standard for processing invalid text data.

[0033] (3) Real-time performance: Manual methods are limited by working time and human resources and cannot achieve real-time failure mode extraction. However, the large language model of this invention can process a large amount of text data in real time and can be continuously updated and optimized over time. Attached Figure Description

[0034] Figure 1This is an overall flowchart of a method for identifying failure modes of safety-critical software based on a large language model, according to the present invention.

[0035] Figure 2 A schematic diagram of virtual tag embedding;

[0036] Figure 3 A schematic diagram of the P-Tuning and P-Tuning v2 technology framework;

[0037] Figure 4 This is a diagram illustrating the optimization of the thought process chain. Detailed Implementation

[0038] The method of the present invention will be further described below with reference to the accompanying drawings and embodiments.

[0039] like Figure 1 As shown, this embodiment discloses a failure mode identification method for safety-critical software based on a large language model. The large language model used in this method is the ChatGLM2-6B pre-trained large language model, which has 6.2 billion parameters and can be deployed locally on consumer-grade graphics cards. As a large language model, ChatGLM2-6B has been optimized for Chinese question-and-answer and dialogue, and has undergone pre-training with 1.4T of Chinese and English identifiers and training aligned with human preferences. With the support of supervised fine-tuning, feedback self-help, and human feedback reinforcement learning techniques, ChatGLM2-6B can generate answers that are quite consistent with human preferences.

[0040] This embodiment discloses a method for identifying failure modes of safety-critical software based on a large language model, which mainly includes the following steps:

[0041] Step 1: Build a safety-critical domain knowledge base and inject it into the large language model.

[0042] The steps for constructing a knowledge base dataset for safety-critical areas include:

[0043] The relevant standard documents and common knowledge in the safety-critical areas are manually selected. These standard documents and common knowledge serve as relevant standards for the design and development of safety-critical software, as well as guidelines for safety analysis methods.

[0044] The selected safety-critical domain-related standard documents and common knowledge are combined with corresponding manually written prompts and encapsulated to construct a safety-critical domain knowledge base. That is, the safety-critical domain-related standard documents and common knowledge are not injected separately, but are combined with corresponding manually written prompts.

[0045] By fine-tuning the knowledge base of safety-critical domains, the large language model is injected, thereby stimulating its reasoning ability when processing text related to prompt words.

[0046] Step 1 will now be explained in more detail.

[0047] To improve the versatility of the large language in safety-critical fields, relevant standards and common knowledge in these fields were manually selected to construct a safety-critical field knowledge base. Referenced materials include: *Military Software Security Design Guidelines GJB / Z 102A-2012*, *Military Software Security Analysis Guidelines GJB / Z 142-2004*, *Failure Mode, Effects and Criticality Analysis Guidelines GJB 1391-2006*, and *General System Security Outline GJB 900-90*. These are the technical standards and related requirements for military safety-critical software, which align with the goal of this method to extract failure modes in safety-critical software. In Excel, prompts were paired with corresponding knowledge items. A partial example of the safety-critical field knowledge base is shown in Table 1.

[0048] Table 1. Examples of the Knowledge Base for Safety-Critical Areas

[0049]

[0050] After constructing the table, the content of the table file is converted into a JSON string format that is easy for machine learning to process, resulting in a JSON format dataset file;

[0051] A single JSON string is shown below:

[0052] {"prompt":"Methods for obtaining failure modes","knowledge":"When conducting FMEA, product failure modes can generally be obtained through statistical, experimental, analytical, and predictive methods."}

[0053] The JSON-formatted dataset file is injected into the large language model in a fine-tuning manner, changing the weight parameters of the original pre-trained model and generating new weights.

[0054] This embodiment uses P-Tuning v2 for fine-tuning. P-Tuning is a relatively new model fine-tuning technique that reduces the number of parameters to 0.1% of the original number through parameter pruning. P-Tuning utilizes a small number of continuous free parameters as input cues to the pre-trained language model and uses gradient descent as an alternative to discrete cue search to optimize continuous cues. P-Tuning v2 is an iterative upgrade of the P-Tuning method. One of its most important improvements is applying continuous cues to every layer of the pre-trained model, not just the input layer. By increasing the capacity of continuous cues, P-Tuning v2's performance is comparable to traditional full-parameter fine-tuning.

[0055] The P-Tuning v2 method will now be further explained:

[0056] The P-Tuning method was proposed primarily to address the problem that the way large models construct their prompts significantly impacts the performance of downstream tasks. For example, GPT-3 uses manually constructed templates for context learning, but these templates are highly sensitive to changes; adding or removing a word, or even changing its position, can cause substantial variations. Furthermore, recent automated template search methods are costly, and the results from previously discrete token searches may not be optimal, leading to performance instability. Therefore, P-Tuning designs a continuously differentiable virtual token, embedded into the tokenized text input. During training, only the parameters of the virtual token are updated, while the parameters of other parts remain fixed. Its structure is as follows: Figure 2 As shown.

[0057] In this context, the word embeddings of the pre-trained language model have become highly discrete. Randomly initializing virtual tokens can easily lead to local optima, even though these virtual tokens should theoretically be interconnected. Therefore, using a prompt encoder for encoding results in faster convergence and better performance, and then inputting the virtual tokens into the larger language model.

[0058] The P-Tuning method only adds virtual tokens to the input layer, while P-Tuning v2, as an iterative upgrade of P-Tuning, adds virtual tokens to every layer of the model. This upgrade has two advantages: first, more learnable parameters (increasing from 0.01% in P-Tuning to 0.1%-3%); and second, virtual tokens added to deeper layers have a more direct impact on model predictions. The characteristics of P-Tuning v2 are as follows... Figure 3 As shown.

[0059] Step 2: Construct an instruction tuning dataset for the failure mode identification task of safety-critical software. Use efficient fine-tuning techniques to iteratively train the model multiple times, and finally test the model's prediction results. Specifically, the instruction tuning dataset is randomly divided into training and test sets proportionally. An initial large language model is used to predict the test set, obtaining the initial large language model prediction results before fine-tuning. Then, based on the training set, the large language model is trained using efficient fine-tuning methods, freezing most model weights and adjusting only some parameters. During efficient fine-tuning, a small number of continuous free parameters are used as input prompts for the pre-trained language model, and gradient descent is used as an alternative to discrete prompt search to optimize continuous prompts. Continuous prompts are applied to every layer of the pre-trained model, not just the input layer. The fine-tuned large model generates new weights. By calling these new weights to predict the test set, the test results after model fine-tuning are obtained, and the improvement in model prediction performance before and after fine-tuning is calculated.

[0060] The steps in this section for constructing the instruction tuning dataset for the safety-critical software failure mode identification task include:

[0061] Collect and organize failed text data from the industry;

[0062] The failure text data from the industry and the corresponding failure modes manually extracted by engineers are processed into "instruction-response" pairs to construct an instruction tuning dataset, where the instruction is the failure text data from the industry and the response is the expected output failure mode.

[0063] Step 2 will now be explained in more detail.

[0064] The first step involves constructing the instruction tuning dataset. During this process, industry failure text data is processed into instruction-response pairs. Table 2 below shows some examples of failure text data annotated by safety engineers from the industry:

[0065] Table 2 Examples of invalid text data

[0066]

[0067] The failure text data annotated by safety engineers is structured, providing a standard answer for the failure mode extraction task and facilitating subsequent supervised fine-tuning. However, to simulate the task scenario of "extracting safety-critical software failure modes from unstructured failure text data," the structured text needs to be processed into unstructured text data for subsequent training and testing. This process is implemented by writing the corresponding code by hand and having it automatically executed by the machine.

[0068] The code logic is as follows: Read the table content row by row, add connecting words before and after the content of the corresponding column in the table to form a complete unstructured failure text description. This failure text description will be used as the instruction in the "instruction-response" pair, while the content corresponding to the "software failure mode" column will be used as the response in the "instruction-response" pair.

[0069] After processing into "instruction-response" pairs, an example is shown below:

[0070] {"content":"During interrupt initialization, a DSP interrupt register malfunction caused the inertial measurement module's 20ms data reception function to fail, resulting in a performance degradation of the programmable controller. The control measure taken was to report the abnormal data reception to the aircraft.","summary":"Interrupt initialization failed"}

[0071] {"content":"During the system's main frequency initialization process, a CPU failure in the DSP caused system timing disorder and timer self-test anomalies. The control measure taken was to simultaneously use software and hardware watchdogs to monitor the software.","summary":"System main frequency initialization failed, timer self-test anomalies"}

[0072] {"content":"During watchdog initialization, an anomaly occurred during DSP watchdog initialization, causing the software watchdog function to fail. The control measure taken was to monitor the software using both software and hardware watchdogs simultaneously.","summary":"Watchdog initialization failed"}

[0073] Then, the constructed instruction fine-tuning dataset was randomly divided into training and test sets in a 7:3 ratio.

[0074] The P-Tuning v2 method was used to optimize the instructions of a large language model for the failure mode identification task in safety-critical software. The hyperparameter settings of the large language model are shown in Table 3 below.

[0075] Table 3 Hyperparameter settings for major language models

[0076]

[0077] The main hyperparameter settings are explained below:

[0078] `PRE_SEQ_LEN` specifies that the maximum length of the natural language instruction is 256, meaning that it can process at most the first 256 tokens in the input sequence.

[0079] `LR` specifies an initial learning rate of 2e-2.

[0080] `NUM_GPUS` specifies the number of GPUs used during training. This experiment uses a single-card deployment, employing only one GPU.

[0081] `--standalone` indicates that distributed training is started in standalone mode (i.e., running on the same machine).

[0082] The `--nnodes` parameter specifies the number of nodes. In this example, there is only one node, so it is set to 1.

[0083] `--nproc-per-node` specifies the number of processes per node. In this example, only one GPU is used, so it is set to 1.

[0084] The `--max_source_length` option specifies that the maximum length of the input text is 256 tokens.

[0085] The `--max_target_length` option specifies the maximum length of the output text as 128 tokens.

[0086] The `--per_device_train_batch_size` option specifies that the batch size for each training session on each GPU is 1.

[0087] The `--per_device_eval_batch_size` option specifies a batch size of 1 for each verification run on each GPU.

[0088] `--gradient_accumulation_steps` specifies that backpropagation should be performed after accumulating 16 gradients.

[0089] `--predict_with_generate` indicates that the generation mode is used when making predictions.

[0090] `max_steps` specifies a maximum training step count of 3000 steps.

[0091] `--logging_steps` specifies that logging should be done every 10 steps.

[0092] `--save_steps` specifies that the model should be saved every 1000 steps.

[0093] The `--quantization_bit` option specifies a quantization bit depth of 4.

[0094] After instruction tuning, the fine-tuned large model was used to make predictions on the test set and compared with the failure modes labeled by safety engineers. The results showed that the large language model has good predictive performance.

[0095] Step 3: Analyze the prediction results obtained in Step 2. For the failed texts, construct the CoT dataset for optimization. Use the CoT dataset to continue training on the existing large language model, refine the inference results, and output the final prediction results.

[0096] CoT (Cooperation of Reasoning) refinement refers to the process of manually constructing a small number of CoT examples to fine-tune unsatisfactory prediction results in instruction fine-tuning tasks. Through continuous multi-step intermediate prompts, the CoT guides the large language model to derive inference results that align with the intentions of safety engineers, visualizes the inference process, and optimizes the failure mode recognition task.

[0097] Step 3 will now be explained in more detail.

[0098] The large language model exhibited biases in failure mode identification and extraction tasks on a small subset of invalid texts. The goal of mind chain optimization is to better align the large language model's predictions with the security engineer's intent. The specific process is as follows: Figure 4 As shown. In this stage, the prediction results from the previous step are first analyzed, and a list of the failed texts that the large language model did not predict is compiled, serving as the basis for optimizing the thought process. For example, the following example:

[0099] {"labels":"Internal watchdog feeding error","predict":"Internal watchdog chip feeding not performed"}

[0100] The "labels" section contains standard failure modes manually extracted by safety engineers, while the "predict" section contains prediction results from a large language model based on the failure text data.

[0101] We selected some failed prediction cases and wrote corresponding thought chain examples to construct a dataset for thought chain optimization. Compared to instruction-based optimization, which only constructs input-output pairs, thought chain optimization uses manually constructed intermediate prompt templates to provide multiple intermediate reasoning steps, guiding the large language model to simulate human thinking, reasoning step by step to reach a conclusion, and visualizing the reasoning process. After thought chain optimization, we used the large language model to re-predict on an existing test set and calculated the improvement in prediction accuracy.

[0102] An example of a single thought chain is as follows:

[0103] Table 4 Examples of Mind Chains

[0104]

[0105] The thought chain was injected into the large model in a fine-tuned manner, and the failure modes of other prediction failure cases were re-identified and extracted. Guided by multiple intermediate prompts, the large language model was able to learn the task paradigm corresponding to the dataset more accurately. The following is a comparison of the prediction performance before and after the thought chain was optimized.

[0106] Table 5 Comparison of the effects before and after mind chain optimization

[0107]

[0108] Through optimization and learning of the thought process chain, the large language model has achieved a significant improvement in performance on the task of identifying failure modes in safety-critical software, demonstrating good transferability. By saving the model file and packaging it into a tool, it can be applied to processing large-scale failure text data.

Claims

1. A method for identifying failure modes of safety-critical software based on a large language model, characterized in that: Includes the following steps: Step 1: Construct a safety-critical domain knowledge base, which consists of relevant standard documents and common knowledge in the safety-critical domain combined with corresponding manually written prompts; inject the safety-critical domain knowledge base into the large language model through P-Tuning v2 technology to obtain the large language model after the first fine-tuning. Step 2: Construct an instruction tuning dataset and use P-Tuning v2 technology to perform multiple rounds of iterative training on the large language model after the first fine-tuning to obtain the large language model after the second fine-tuning; the instruction tuning dataset consists of instruction-response pairs; wherein, invalid text data is used as instructions and invalid patterns are used as responses; Step 3: Use the finely tuned large language model to predict failure modes of industrial failure text data, and construct the thinking chain dataset based on the industrial failure text data that failed to predict. Use the thinking chain dataset to continue training the finely tuned large language model to obtain a safety-critical domain-specific large language model with ideal final results. Step 4: Input the failure text data of the failure mode to be identified into the proprietary large language model for safety-critical domains to obtain the failure mode identification results; In step 3, the thought chain dataset is constructed based on the failed predictions of the large language model after secondary fine-tuning of the industrial failure text data. Specific operations include: A thought chain example is constructed based on the failure prediction of a large language model with secondary fine-tuning in industrial failure text data. Using the Mind Chain example, construct a Mind Chain dataset; The thought chain example is constructed by analyzing failure text data from the industry and providing a multi-step intermediate reasoning process. The intermediate reasoning steps include: identifying the failed object, determining the failure impact, determining the failure cause, and determining the failure mode.

2. The method for identifying failure modes of safety-critical software based on a large language model according to claim 1, characterized in that: Step 1, the construction of the safety-critical domain knowledge base includes the following steps: Manually select relevant standard documents and common knowledge in safety-critical areas; Match manually selected safety-critical areas with relevant standard documents and common knowledge with manually written prompts; Convert the paired content into JSON string format; This allows us to construct a knowledge base for safety-critical areas.

3. The method for identifying failure modes of safety-critical software based on a large language model according to claim 1, characterized in that: Step 2, the construction of the instruction tuning dataset includes the following steps: Obtain structured failure text data annotated by safety engineers; The structured failure text data annotated by safety engineers is processed into unstructured text data; By using unstructured text data as instructions in instruction-response pairs and corresponding failure modes as responses in these pairs, an instruction tuning dataset is constructed.

4. The method for identifying failure modes of safety-critical software based on a large language model according to claim 3, characterized in that: By adding conjunctions before and after invalid text data annotated by safety engineers, unstructured text data is obtained.

Images