AADL model automatic generation method based on large language model

Abstract

The invention discloses an AADL model automatic generation method based on a large language model. The method comprises the steps that a safety key field AADL knowledge base is constructed; injecting the constructed safety key field AADL knowledge base into a local large language model through a fine tuning technology to obtain a fine-tuned large language model; plug-in the safety-critical field AADL knowledge base to the fine-tuned large language model to obtain a special large language model for the safety-critical software; inputting cue words into the special large language model of the safety-critical software to obtain an AADL model generation result; meanwhile, the invention provides a prompt framework based on a natural language demand limiting template, and the performance of a large language model in an AADL model automatic generation task is remarkably improved. The measures not only optimize the understanding of the model on AADL specifications and safety requirements, but also enhance the automatic generation capability of the model in complex system architecture description, thereby improving the generation accuracy and practicability of the model while ensuring the system safety and reliability.

Description

Technical Field

[0001] The present invention belongs to the technical field of safety-critical fields, and specifically is a method for automatically generating an AADL model based on a large language model. Background Art

[0002] Safety-critical systems generally refer to a type of system that may lead to catastrophic consequences if critical functions fail or are lost, and are widely used in key fields such as aviation, aerospace, and nuclear energy. Safety-critical software refers to a type of software that is used in safety-critical systems and whose operation may cause the system to be in a dangerous state, thereby causing property loss, environmental damage, or personal injury. That is, safety-critical software is generally part of a safety-critical system, which may cause or contribute to unsafe conditions. Such software is considered safety-critical. In recent years, accidents caused by software system problems are common. For example, Toyota recalls in the past decade have caused billions of dollars in losses due to various control software problems, such as the failure of the car electric window lift control system in 2012, the speed control failure in 2013, and the Takata airbag failure problem from 2015 to 2018; in 2016, the Japanese Aerospace Agency (JAXA) announced that the 31 billion yen X-ray astronomical detection satellite Hitomi rolled abnormally due to attitude control software failure, and eventually lost control completely. It can be seen that the impact of software systems on the security of embedded systems has gradually become dominant, and the failure and security issues of software control systems have become an important cause of accidents. Therefore, ensuring the security of embedded software has become a very important topic in the current software engineering research field.

[0003] Software security analysis is a widely accepted method to ensure software security. At present, software security analysis work is mainly concentrated in the software requirements specification and software design stages. Lack of foresight in software design and specification errors are the main reasons that affect security. At present, software requirements are mainly described in natural language, which is ambiguous, vague and incomplete, and difficult to analyze and process automatically, making software design mainly rely on designers' understanding of natural language requirements. This is also the fundamental reason why model-driven development (MDD) generally starts with software analysis models or design models rather than requirements models. It is precisely because of the inevitable ambiguity and fuzziness of natural language requirements that the software design model will be deviated due to the inconsistent understanding of requirements by designers, which will lead to software security issues. Summary of the invention

[0004] Purpose of the invention: In order to solve the problem of software security being affected by the ambiguity and vagueness of natural language requirements, the present invention proposes an automatic generation method of AADL models based on a large language model. The present invention aims to explore the use of large language models to assist in AADL modeling of safety-critical software, so as to improve the development efficiency and quality of safety-critical software.

[0005] Technical solution: A method for automatically generating an AADL model based on a large language model, comprising the following steps:

[0006] Build AADL knowledge base in safety-critical areas;

[0007] Through fine-tuning technology, the constructed safety-critical AADL knowledge base is injected into the local large language model to obtain a fine-tuned large language model;

[0008] The safety-critical domain AADL knowledge base is plugged into the fine-tuned large language model to obtain a safety-critical software-specific large language model;

[0009] Input the prompt words into the safety-critical software proprietary large language model to obtain the AADL model generation result.

[0010] Furthermore, the construction of the safety-critical domain AADL knowledge base specifically includes the following operations:

[0011] Determine the scope of the safety-critical domain of the safety-critical domain AADL knowledge base to be constructed;

[0012] Obtain AADL modeling examples within the scope of this safety-critical domain;

[0013] Extracting AADL grammar knowledge from AADL modeling cases; the AADL grammar knowledge includes the definition method, connection relationship, data flow transmission and other key design patterns of AADL components;

[0014] According to the AADL grammar knowledge, manually write prompt content; each prompt content is paired with the corresponding AADL grammar knowledge to form a pairing table;

[0015] The pairing contents in the pairing table are converted into a data format suitable for training the local large language model to obtain the AADL knowledge base in the safety-critical field.

[0016] Furthermore, the conversion of the pairing contents in the pairing table into a data format suitable for training the local large language model specifically includes:

[0017] Convert the pairing content in the pairing table into a json string format, expressed as:

[0018] {"prompt":"xxxxxx","knowledge":"xxxxxxxx"}

[0019] Among them, the content corresponding to prompt is the prompt content, and the content corresponding to knowledge is the corresponding AADL grammar knowledge;

[0020] Convert the json string format to a data format suitable for large language model training, expressed as:

[0021] {"conversations":[{"role":"user","content":"xxxxxx"},{"role":"assistant","content":"xxxxxxxx"}]}

[0022] Among them, conversations represents multiple rounds of interactions between users and the large language model, role represents the role played by the user or the large language model, user represents the user, content represents the text input or output by the corresponding role, xxxxxx represents the actual input content of the user, that is, the prompt word, assistant represents the large language model, and xxxxxxxx represents the response generated by the large language model to the user input.

[0023] Furthermore, through the LoRA fine-tuning technology, the constructed safety-critical AADL knowledge base is injected into the local large language model to obtain the fine-tuned large language model.

[0024] Furthermore, the LoRA fine-tuning technology is used to inject the constructed safety-critical AADL knowledge base into the local large language model to obtain a fine-tuned large language model, which specifically includes:

[0025] The weight matrix W of the local large language model is decomposed into two low-rank matrices A and B, expressed as: W = W_0 + BA, where W_0 is the weight matrix of the local large language model, A and B are both low-rank matrices, and rank(B) < <rank(W);

[0026] During fine-tuning, only two low-rank matrices A and B are trained, keeping W_0 unchanged;

[0027] The two trained low-rank matrices A and B are injected into each layer of the Transformer architecture of the local large language model to obtain the fine-tuned large language model.

[0028] Furthermore, RAG technology is used to externalize the AADL knowledge base in the safety-critical domain to the fine-tuned large language model to obtain a large language model proprietary to safety-critical software.

[0029] Furthermore, the RAG technology is used to externalize the safety-critical domain AADL knowledge base to the fine-tuned large language model to obtain a safety-critical software-specific large language model. The specific operations include:

[0030] According to the query questions input by the user, relevant information is captured from the AADL knowledge base in the safety-critical field;

[0031] Convert the captured relevant information into knowledge vectors in high-dimensional space;

[0032] Based on the relevance to the query question input by the user, the knowledge vectors in the high-dimensional space are sorted, and the knowledge vectors with the highest scores are added to the prompt content.

[0033] Further, the prompt words input into the safety-critical software proprietary large language model are prompt words that conform to the prompt framework;

[0034] The prompt framework is composed of a template based on a defined natural language requirement, a naming format specification and design constraints;

[0035] The limited natural language requirement template includes: component name, component type, input and output, composition, functional requirements, connection analysis, interface requirements and performance requirements.

[0036] Beneficial effects: Compared with the prior art, the present invention has the following advantages:

[0037] (1) Automation: The large language model of the present invention can quickly generate AADL code based on user descriptions, reducing the time for manual writing and being able to handle a large number of modeling tasks, thereby improving production efficiency;

[0038] (2) Accuracy: The large language model of the present invention can generate code according to standard syntax and specifications, reduce human errors and omissions, maintain the consistency of code output, ensure compliance with AADL best practices, and thus improve the reliability and maintainability of system modeling;

[0039] (3) Intelligent assistance: Based on the large language model of the present invention, users can use natural language to describe their needs, and the model understands and converts them into AADL code, which significantly reduces the usage threshold;

[0040] (4) The method of the present invention is used to assist in AADL modeling of safety-critical software, which can greatly improve the development efficiency and quality of safety-critical software and reduce labor costs;

[0041] (5) Aiming at the gap between embedded software requirements and design, the present invention proposes a prompt framework based on a limited natural language requirement template, which eliminates the shortcomings of ambiguity and vagueness of natural language requirements. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 It is a schematic diagram of the overall process of an automatic generation method of AADL model based on a large language model proposed by the present invention;

[0043] Figure 2 This is a schematic diagram of the LoRA fine-tuning technology framework;

[0044] Figure 3 It is a schematic diagram of RAG technology;

[0045] Figure 4 Schematic diagram of a prompt framework based on a limited natural language requirement template. DETAILED DESCRIPTION

[0046] The technical solution of the present invention is now further described in conjunction with the accompanying drawings and embodiments.

[0047] This embodiment proposes an automatic generation method of AADL model based on a large language model. The large language model used in this embodiment is the ChatGLM3-6B pre-trained large language model, which has 6 billion parameters and is convenient for local deployment. ChatGLM3-6B retains many excellent features of the previous two generations of models, such as smooth conversation and low deployment threshold, and introduces a more powerful basic model, more complete functional support and a more comprehensive open source sequence.

[0048] like Figure 1 As shown, the method of this embodiment mainly includes the following steps:

[0049] Step 1: Build an AADL knowledge base for safety-critical areas. The knowledge base consists of manually written model natural language requirements and the corresponding AADL model code. Inject it into the local large language model through LoRA fine-tuning technology to obtain a fine-tuned large language model. After the initial fine-tuning, test the fine-tuned large language model's ability to understand and generate AADL-related tasks, and further fine-tune the model.

[0050] The AADL knowledge base for safety-critical areas mentioned in this step is constructed according to the following sub-steps:

[0051] Sub-step 1: Define the scope of the safety-critical areas to build the knowledge base: First, the safety-critical areas involved need to be identified. Safety-critical areas usually refer to the parts of the system that are critical to safety and whose failure may cause serious consequences. Such areas include but are not limited to aerospace, automotive electronics, nuclear power generation, etc. The critical systems in these areas have extremely high requirements for reliability, safety and functionality.

[0052] Sub-step 2: Obtain AADL modeling cases in safety-critical areas: Collect AADL modeling cases related to safety-critical areas through open source platforms such as Github. The data in AADL modeling cases can help you gain a deeper understanding of the definition and design of the system architecture. In addition, by analyzing these AADL modeling cases, you can extract key AADL model elements, especially core grammatical structures such as component definition and connection definition.

[0053] Sub-step 3: Summarize AADL grammar rules: Based on the extracted AADL modeling cases, summarize the AADL grammar knowledge, especially the parts closely related to the design of safety-critical systems. Focus on extracting the definition method, connection relationship, data flow transmission and other key design patterns of AADL components to ensure the completeness and accuracy of the knowledge base.

[0054] Sub-step 4: Write prompts and pair them with knowledge: Write targeted prompts based on AADL grammar knowledge and case summaries. These prompts will be used to guide the large language model to understand the core concepts and design principles of the AADL architecture. Each prompt should be paired with the corresponding knowledge point and form a structured knowledge base to support fine-tuning training of the large model.

[0055] Table 1. Some examples of AADL knowledge base in safety-critical areas

[0056]

[0057]

[0058] Pair the constructed component definition, connection definition and other knowledge content with the manually written prompts to obtain a paired Excel table, and convert the paired content into a json string format, expressed as:

[0059] {"prompt":"xxxxxx","knowledge":"xxxxxxxx"}

[0060] Among them, the content corresponding to prompt is the prompt, and the content corresponding to knowledge is the corresponding AADL knowledge.

[0061] This format is further converted into a data format suitable for large language model training, expressed as:

[0062] {"conversations":[{"role":"user","content":"xxxxxx"},{"role":"assistant","content":"xxxxxxxx"}]}.

[0063] Among them, conversations represents multiple rounds of interactions between users and the large language model, role represents the role played by the user or the large language model, user represents the user, content represents the text input or output by the corresponding role, xxxxxx represents the actual input content of the user, that is, the prompt word, assistant represents the large language model, and xxxxxxxx represents the response generated by the large language model to the user input.

[0064] For example, the format of a single data set is as follows:

[0065] {"conversations":[{"role":"user","content":"Define a device component"},{"role":"assistant","content":"device MyDevice\nend MyDevice"}]}

[0066] In this way, a knowledge base of safety-critical areas is constructed.

[0067] Afterwards, the constructed safety-critical domain AADL knowledge base is randomly divided into training set and test set in a ratio of 7:3.

[0068] The processed safety-critical AADL knowledge base is first injected into the large language model through fine-tuning. After the first fine-tuning, if the large language model performs poorly on the specific task of AADL generation, such as misunderstanding of the definition of some components, further fine-tuning is required. By adding or adjusting the safety-critical AADL knowledge base, optimizing hyperparameters, adjusting the learning rate, etc., the large language model can be helped to improve its performance in the task of generating AADL models.

[0069] The fine-tuning method of this embodiment uses the LoRA fine-tuning technology, the core idea of ​​which is to inject the trainable low-rank decomposition matrix into each layer of the Transformer architecture after freezing the pre-trained model weights, thereby greatly reducing the amount of trainable parameters on downstream tasks. The LoRA fine-tuning technology specifically includes:

[0070] Low-rank decomposition: Decompose the weight matrix W into two low-rank matrices A and B, that is, W = W_0 + BA, where W_0 is the pre-trained weight matrix, A and B are low-rank matrices, and rank(B) < <rank(W)。

[0071] Training low-rank matrices: During fine-tuning, only the low-rank matrices A and B are trained while keeping W_0 unchanged, which greatly reduces the number of parameters that need to be trained and thus improves the efficiency of fine-tuning.

[0072] Efficient adaptation: By adjusting the low-rank matrices A and B, LoRA can effectively adapt to new tasks or domain requirements while maintaining the overall structure and performance of the model. The core idea of ​​LoRA is as follows Figure 2 shown.

[0073] LoRA fine-tuning reduces training overhead by introducing low-rank matrices while retaining the original knowledge of the large language model, making it efficient to adapt to AADL knowledge in safety-critical areas.

[0074] Step 2: Use RAG technology to combine the organized safety-critical domain AADL knowledge base in Excel format with the fine-tuned large language model to provide an external knowledge source for the fine-tuned large language model, that is, use the above-constructed safety-critical domain AADL knowledge base as an external knowledge source to reduce the model hallucination problem.

[0075] In this embodiment, the Langchain-Chatchat framework is used to integrate various large language models, embedding models and knowledge bases. LangChain-Chatchat is a local knowledge base question-answering application based on large language models such as Langchain and ChatGLM. This framework is used to embed the AADL knowledge base in the safety-critical field into the large language model to enhance its knowledge representation ability in specific fields. The large language model uses ChatGLM3-6B, the word segmenter selects ChineseRecursiveTextSplitter, the Embedding model is bge-large-zh, and the vector database uses the FAISS vector library.

[0076] Figure 3 The RAG method and its implementation principle are shown.

[0077] The RAG method does not change any parameters of the model. It obtains relevant information from an external database and then combines the obtained relevant information with the question as a prompt, which will be provided to the large language model to enhance text generation. The specific operation is as follows:

[0078] Input query: The RAG process starts with a query question entered by the user, such as "Define a device component using AADL." This query can involve various requirements generated by the AADL model.

[0079] Document Retrieval: The retrieval model will fetch relevant information from the uploaded AADL knowledge base in the safety-critical domain.

[0080] Vectorization: The retrieved information is converted into knowledge vectors in a high-dimensional space. These knowledge vectors are stored in a vector database for fast access and processing.

[0081] Relevance Ranking: The vector model ranks the retrieved information based on its relevance to the input query. The documents or paragraphs with the highest scores are selected for further processing.

[0082] Contextual enhancement: Generative models (such as ChatGLM3-6B) enhance the context of user input with retrieved relevant information. During model generation, selected document snippets are added to the prompt to enhance the accuracy and domain applicability of the generated text.

[0083] Through the RAG strategy, the generative model can effectively integrate external knowledge, improve the quality of generated content, and avoid hallucinations without changing the internal parameters of the model.

[0084] In order to reduce the ambiguity and ambiguity of prompt words, a method based on limited natural language requirement templates is proposed according to the software components, hardware components, composite and general components of the AADL model. The requirement template is combined with tasks, naming format specifications and design constraints to form a prompt framework. Figure 4 shown.

[0085] Now combined Figure 4 The steps of constructing the prompt framework based on the limited natural language requirement template are further explained:

[0086] (1) Define the task description for generating the AADL model: Since the large language model needs to clearly define the type of AADL model generated from the provided requirement text, component requirements, and other information, the task description needs to clearly outline the instructions of the large language model. The task description needs to include the following statement: Based on the requirements of the component, use AADL to output the device model code. Please follow the naming format specification.

[0087] (2) Define limited natural language requirement templates: Although large language models perform well in processing natural language, they still have shortcomings such as insufficient ambiguity processing and limited detail processing capabilities when understanding complex requirements. Limited natural language requirement templates are a standardized description of requirements. Based on a comprehensive analysis of RUCM, EARS and other requirement expression methods, limited natural language requirement templates can minimize the change in engineers' requirement writing habits and reduce the ambiguity and vagueness of natural language requirements, thereby reducing errors caused by unclear natural language expression when large language models process AADL generation tasks, and improving the accuracy and reliability of model generation.

[0088] Table 2: Natural language requirement template

[0089]

[0090]

[0091] The following is a detailed introduction to each element in Table 2.

[0092] The component name is limited to a noun or a combination of multiple nouns, which should express the function of the component as intuitively as possible.

[0093] Component type is the AADL component type, such as device, bus, etc.

[0094] Input and output: Embedded system components need to continuously obtain data from other components, calculate and output to other components. Therefore, input and output are used to describe the input data / events obtained by the component and the output to other components of the system after calculation or processing. The input and output data / events of a component are enumerated one by one, and omitted if there is none.

[0095] Composition: Mainly describes the hierarchical division of the system or component. A system can be composed of multiple components, and a component can call other subroutine functions.

[0096] Functional requirements: describes the functional requirements of the component, such as the thread needs to call subroutine XXX to execute specific signal reception and processing logic.

[0097] Connection analysis: describes the data interaction between this component and other components, such as sending and receiving data.

[0098] Interface requirements: describe the interaction methods and requirements between a system or component and the external environment or other components, such as interface data transmission protocols, communication methods between components, etc.

[0099] Performance requirements: Describe the performance requirements of the system, such as response time, processing speed, throughput, etc.

[0100] (3) Define naming format specifications: Provide standard naming rules to ensure that the generated AADL model is consistent and readable.

[0101] (4) Define design constraints: Specify the design rules and restrictions that the large language model must follow when generating the AADL model to ensure that the generated model meets specific design standards or system requirements.

[0102] To generate comprehensive prompts, natural language requirement templates, naming format specifications, and design constraints are incorporated into the task description. This complete prompt is critical to guiding the behavior of large language models during the generation process.

[0103] Based on the above prompt framework and the requirement of limiting the natural language template, a simple practical component is now given as an example of the prompt framework, as shown in Table 3.

[0104] Table 3 Prompt frame example

[0105]

[0106]

[0107] The prompt word is applied to the proprietary large language model of safety-critical software to obtain the AADL model generation result. The model generation result is shown below and meets the component requirements.

[0108] Process Controller

[0109] features

[0110] radar_signal:in data port;

[0111] target_distance:out data port;

[0112] properties

[0113] Execution_Time=>1ms..2ms; -- The calculation execution time of the component is 1-2ms

[0114] Deadline=>1ms;--The deadline of the component is 1ms

[0115] Period=>1ms; --The component period is 1ms

[0116] end Controller;

[0117] In the safety-critical field, by organizing the AADL knowledge item dataset and injecting it into the large language model using fine-tuning technology, and adopting the retrieval enhancement generation RAG strategy to provide a plug-in knowledge base for the fine-tuned language model, combined with the prompt framework method based on limited natural language templates, the performance of the large language model in the AADL model automatic generation task is significantly improved. These measures not only optimize the model's understanding of AADL specifications and safety requirements, but also enhance its automatic generation capabilities in complex system architecture descriptions, thereby improving the generation accuracy and practicality of the model while ensuring system safety and reliability.

Claims

1. A method for automatically generating an AADL model based on a large language model, characterized by: The following steps are involved: Build AADL knowledge base in safety-critical areas; Through fine-tuning technology, the constructed safety-critical AADL knowledge base is injected into the local large language model to obtain a fine-tuned large language model; The safety-critical domain AADL knowledge base is plugged into the fine-tuned large language model to obtain a safety-critical software-specific large language model; Input the prompt words into the safety-critical software proprietary large language model to obtain the AADL model generation result.

2. The method for automatically generating an AADL model based on a large language model according to claim 1, characterized in that: The specific operations of constructing the AADL knowledge base in the safety-critical field include: Determine the scope of the safety-critical domain of the safety-critical domain AADL knowledge base to be constructed; Obtain AADL modeling examples within the scope of this safety-critical domain; Extracting AADL grammar knowledge from AADL modeling cases; the AADL grammar knowledge includes the definition method, connection relationship, data flow transmission and other key design patterns of AADL components; According to the AADL grammar knowledge, manually write prompt content; each prompt content is paired with the corresponding AADL grammar knowledge to form a pairing table; The pairing contents in the pairing table are converted into a data format suitable for training the local large language model to obtain the AADL knowledge base in the safety-critical field.

3. The method for automatically generating an AADL model based on a large language model according to claim 2, characterized in that: The step of converting the pairing contents in the pairing table into a data format suitable for training the local large language model specifically includes: Convert the pairing content in the pairing table into a json string format, expressed as: {"prompt":"xxxxxx","knowledge":"xxxxxxxx"} Among them, the content corresponding to prompt is the prompt content, and the content corresponding to knowledge is the corresponding AADL grammar knowledge; Convert the json string format to a data format suitable for large language model training, expressed as: {"conversations":[{"role":"user","content":"xxxxxx"},{"role":"assistant","content":"xxxxxxxx"}]} Among them, conversations represents multiple rounds of interactions between users and the large language model, role represents the role played by the user or the large language model, user represents the user, content represents the text input or output by the corresponding role, xxxxxx represents the actual input content of the user, that is, the prompt word, assistant represents the large language model, and xxxxxxxx represents the response generated by the large language model to the user input.

4. The method for automatically generating an AADL model based on a large language model according to claim 1, characterized in that: Through the LoRA fine-tuning technology, the constructed safety-critical AADL knowledge base is injected into the local large language model to obtain the fine-tuned large language model.

5. The method for automatically generating an AADL model based on a large language model according to claim 4, characterized in that: The LoRA fine-tuning technology is used to inject the constructed safety-critical AADL knowledge base into the local large language model to obtain a fine-tuned large language model, which specifically includes: The weight matrix W of the local large language model is decomposed into two low-rank matrices A and B, expressed as: W = W_0 + BA, where W_0 is the weight matrix of the local large language model, A and B are both low-rank matrices, and rank(B) < <rank(W); During fine-tuning, only two low-rank matrices A and B are trained, keeping W_0 unchanged; The two trained low-rank matrices A and B are injected into each layer of the Transformer architecture of the local large language model to obtain the fine-tuned large language model.

6. The method for automatically generating an AADL model based on a large language model according to claim 1, characterized in that: The RAG technology is used to plug the AADL knowledge base in the safety-critical domain into the fine-tuned large language model to obtain a large language model proprietary to safety-critical software.

7. The method for automatically generating an AADL model based on a large language model according to claim 6, characterized in that: The RAG technology is used to plug the safety-critical domain AADL knowledge base into the fine-tuned large language model to obtain a safety-critical software-specific large language model. The specific operations include: According to the query questions input by the user, relevant information is captured from the AADL knowledge base in the safety-critical field; Convert the captured relevant information into knowledge vectors in high-dimensional space; Based on the relevance to the query question input by the user, the knowledge vectors in the high-dimensional space are sorted, and the knowledge vectors with the highest scores are added to the prompt content.

8. The method for automatically generating an AADL model based on a large language model according to claim 1, characterized in that: The prompt words input into the safety-critical software proprietary large language model are prompt words that conform to the prompt framework; The prompt framework is composed of a template based on a defined natural language requirement, a naming format specification and design constraints; The limited natural language requirement template includes: component name, component type, input and output, composition, functional requirements, connection analysis, interface requirements and performance requirements.

Images