Intelligent completion method for aircraft manufacturing process instructions based on lightweight large language model

Abstract

This invention provides an intelligent completion method for aircraft manufacturing process instructions based on a lightweight large language model. By constructing a domain-specific corpus, training a lightweight model that supports intermediate filling, implementing localized deployment optimization, and building a real-time interactive completion process, it achieves efficient, intelligent, and secure completion of process instructions in a low-computing-power CPU environment. Through dedicated training and local deployment of the lightweight large language model, it effectively improves the efficiency and quality of process instruction compilation, reduces manual workload, and promotes the standardization and intelligent development of process flows.

Description

Technical Field

[0001] This invention relates to the field of intelligent manufacturing, and in particular to an intelligent completion method for aircraft manufacturing process instructions based on a lightweight large language model. Background Technology

[0002] In high-precision and high-reliability technology fields such as aerospace and equipment manufacturing, the compilation of process instructions is a core link connecting product design and actual production. Its standardization, accuracy, and compilation efficiency directly affect the quality, cost, and production cycle of the final product. With the development of intelligent manufacturing, the traditional model relying on process engineers manually consulting standards and compiling instructions based on experience is no longer suitable for the demands of modern production, which involves multiple varieties, small batches, and rapid iterations. Therefore, improving the intelligence level of process design has become one of the key directions for technological upgrading in the industry.

[0003] In recent years, artificial intelligence technology, especially large language models, has made significant progress in natural language processing and text generation. The industry has begun exploring its application in the automatic generation and assisted compilation of process instructions, aiming to reduce repetitive work, minimize human error, and promote the standardization and accumulation of process knowledge through technological means. In practice, cloud-based interface services based on large language models are typically used. By inputting information such as processing characteristics and materials, the model outputs a preliminary description of the process steps.

[0004] However, existing technological solutions still face a series of prominent bottlenecks in real-world industrial scenarios, hindering their large-scale application and deep empowerment: First, the reliability and accuracy of the generated results are insufficient. Process instructions are highly specialized, rigorous, and logically demanding, and must strictly adhere to technical standards, equipment parameters, and safety regulations. General-purpose language models lack sufficient domain-specific knowledge, and when generating specific process parameters, processing steps, and testing requirements, they are prone to producing "illusionary" content or vague descriptions that do not match reality, posing technical risks. Therefore, the model output still requires extensive manual verification and revision by experienced process engineers, failing to substantially reduce workload and sometimes even incurring additional time costs due to error correction.

[0005] Secondly, existing solutions struggle to meet the demands for real-time, convenient, and interactive assistance. Process planning is a dynamic, interactive, and creative process that requires engineers to think and write simultaneously. Current cloud-based solutions, due to the massive number of model parameters, reliance on high-performance GPU computing power, and complex service deployments, suffer from significant response latency. This prevents them from providing smooth, intelligent auto-completion, parameter suggestions, or specification checks in an "instant input" manner during engineer input, resulting in a fragmented interactive experience that cannot be seamlessly integrated into existing workflows.

[0006] Furthermore, data security and deployment models present significant constraints. Process data in fields such as aerospace and equipment manufacturing contains core intellectual property and sensitive information, demanding extremely stringent data confidentiality. Uploading process data to cloud servers for processing, even with private deployment, still carries potential risks of leakage during network transmission and service maintenance. Many organizations handling state secrets or core corporate secrets explicitly require a completely local offline environment. The massive size and high computing power requirements of general-purpose large-scale models make it difficult to run locally on ordinary industrial control computers or internal secure terminals lacking high-performance graphics cards.

[0007] In summary, existing process instruction generation technologies based on general-purpose large language models cannot yet meet the stringent requirements of high-end manufacturing for process design support tools in terms of accuracy, real-time performance, security, and ease of deployment. Therefore, the industry urgently needs an innovative technical solution that can run efficiently on ordinary computing devices (such as local CPU environments), achieving low-latency, high-accuracy intelligent completion and auxiliary generation of process instructions. This solution should be truly integrated into the real-time workflow of process engineers while ensuring absolute data security, thereby effectively improving the standardization and efficiency of process design and driving the process towards greater intelligence and standardization. Summary of the Invention

[0008] The present invention adopts the following technical solution: The main technical problem addressed by this invention is that, in the process of compiling process instructions, facing the dual requirements of high accuracy and high efficiency, existing large language models are limited by factors such as computing power requirements, deployment complexity, and response latency, making it difficult to provide real-time and intelligent completion assistance for manually revising process instructions in a local environment. Furthermore, traditional methods struggle to balance data security and the level of intelligence.

[0009] To address the aforementioned technical challenges, this invention proposes an intelligent process instruction completion method based on a lightweight large language model. This method achieves efficient, intelligent, and secure process instruction completion in low-computing-power CPU environments by constructing a domain-specific corpus, training a lightweight model supporting intermediate padding, implementing localized deployment optimization, and building a real-time interactive completion process. Specifically, it includes the following: Firstly, this invention constructs a domain-structured corpus and augmented dataset for pre-training. Addressing the standardized characteristics of process instructions, this invention first establishes a data processing method. This method collects raw text from historical process documents, templates, and specifications, performs cleaning and structured segmentation, forming a high-quality domain-based corpus. Based on this, to enable the model to perform "fill-in-the-blank" tasks, a fill-in-the-middle (FIM) data augmentation strategy is adopted. This involves randomly segmenting some continuous process instruction text into prefix, middle, and suffix parts, and then recombining them into training samples according to a specific format. Through this method, a hybrid dataset containing sequential generation and context-filling tasks is constructed, providing a solid data foundation for the model to learn process logic from scratch.

[0010] Secondly, this invention constructs and trains a domain-specific lightweight large language model. To address the issues of large size and slow inference in general-purpose models, this invention proposes a lightweight model construction and training method. This method does not directly use a massive general-purpose vocabulary, but instead generates a highly simplified domain-specific vocabulary based on the aforementioned domain corpus statistics. The network structure of the large language model is then reconstructed accordingly, significantly reducing redundant parameters. Subsequently, a random initialization strategy is employed, and the lightweight model is pre-trained from scratch using a mixed dataset. This method enables the model to learn domain-specific terminology, writing standards, and contextual reasoning capabilities more purely and efficiently, while maintaining a very low parameter scale, freeing it from the interference of redundant knowledge in general-purpose models.

[0011] Thirdly, the invention implements localized quantization deployment and inference optimization of the model. To achieve smooth operation on the CPUs of ordinary industrial control computers or office computers, this invention proposes a model conversion and deployment method. First, the weights of the trained lightweight model are mapped back to the standard model architecture to ensure compatibility with general inference engines. Second, the model undergoes integer quantization, further compressing the model size and reducing memory usage while maintaining accuracy. Finally, the quantized model is loaded through a localized inference engine, achieving a highly efficient inference environment completely independent of the cloud and where data remains locally, thus ensuring the security of process data.

[0012] Fourthly, this invention enables a context-aware, real-time intelligent completion interaction process. It also provides an intelligent completion interaction method integrated into a process instruction editor. When a user edits a process instruction, the system monitors the cursor position in real time and automatically extracts the preceding and following text as input context. The system dynamically selects the inference mode based on the completeness of the context and calls a local lightweight model to generate predicted content. The generated completion suggestions are displayed in the editor in a non-intrusive manner, allowing users to adopt or modify them with a single click. This method seamlessly integrates complex model inference into the manual compilation process, significantly improving the efficiency and standardization of process instruction compilation.

[0013] In one implementation, the multi-source acquisition and cleaning of process instruction data includes the following steps: Step 111: Collect raw text data from historical process instruction documents and process templates by combining automated scripts with manual collection. Step 112 involves cleaning the collected raw text, specifically including: removing garbled characters, invisible characters, and irrelevant header and footer information from the document; standardizing full-width and half-width symbols and unit of measurement formats; and using regular expressions to remove sensitive fields.

[0014] In one implementation, the structured segmentation and standardization of data includes the following steps: Step 121: Based on the semantic integrity of the process instructions, the cleaned long text is segmented using a multi-level segmentation strategy of "document-paragraph-instruction line". Step 122: Standardize and validate the segmented data, check whether each instruction contains a complete verb and object, remove invalid segments that are too short or semantically incomplete, and form a sequential training corpus.

[0015] In one implementation, constructing an augmented dataset that supports intermediate padding includes the following steps: Step 131: The basic corpus is enhanced using a prefix-suffix-intermediate pattern. For each sample in the basic corpus, an intermediate padding transformation is performed with a 50% probability. Step 132: Randomly segment the selected sample for intermediate padding transformation, and randomly segment a complete process instruction text sequence T into three continuous parts: prefix P, middle M, and suffix S; the segmentation points must satisfy that the lengths of P, M, and S are all greater than the set minimum threshold. Step 133 introduces three special control characters: <|fim_prefix|>, <|fim_suffix|>, and <|fim_middle|>; the segmented fragments are then reassembled into a new training sample sequence according to the following format: ; Step 134: Mix the samples that have undergone intermediate padding transformation with the original sequential samples that have not been transformed to construct the final mixed training dataset.

[0016] In one implementation, constructing a domain-specific thesaurus includes the following steps: Step 211: Perform a full scan and word segmentation statistics on the mixed training dataset. Use the basic word segmenter of the general large model to preprocess the corpus and count all the tokens that appear and their frequencies. Step 212: Filter valid tokens and construct mapping relationships. Extract all tokens that actually appear in the process corpus from the statistical results and forcibly retain necessary special control characters. Combine these tokens into a new special vocabulary. Step 213: Establish a bidirectional mapping index, construct a mapping table from general vocabulary IDs to special vocabulary IDs and a reverse mapping table, which will be used for subsequent model weight migration and recovery.

[0017] In one implementation, reconstructing the lightweight model network architecture includes the following steps: Step 221: Based on the Transformer architecture of the general large model, initialize a new model configuration; keep the original number of layers, number of attention heads, and hidden layer dimensions unchanged; Step 222: Modify the dimensions of the embedding layer at the model input end and the language model head at the output end from the general vocabulary size to the dedicated vocabulary size.

[0018] In one implementation, pre-training the lightweight model includes the following steps: Step 231: A random initialization strategy is adopted, without loading any pre-trained weights of general large models, and instead the model is allowed to learn the semantic representation and logical rules of the process domain from scratch. Step 232: Adopt the autoregressive language modeling objective, introduce the sliding window mechanism, set the window size and step size to ensure that the model can learn long-distance contextual dependencies across lines and paragraphs; Step 233: For training samples with intermediate padding, when calculating cross-entropy loss, only calculate the gradient for the "intermediate content" part after the <|fim_middle|> label, and set the Label of the prefix and suffix parts to ignore values; Step 234: Input the mixed dataset into the lightweight model for multiple rounds of iterative training, monitor the changes in the loss function in real time; periodically evaluate the perplexity and completion accuracy of the model on the validation set, and save the optimal lightweight model weight file.

[0019] In one implementation, mapping the trained lightweight model weights back to the standard model architecture includes the following steps: Step 311: Load the standard configuration file of the general large model and initialize an empty model framework with the full vocabulary size. The number of layers and dimension parameters of this framework are consistent with the lightweight model, but the embedding layer and output layer are restored to their original size. Step 312: Iterate through the weight parameters of the lightweight model. For the weights of the intermediate layer, directly copy them to the corresponding positions in the "full vocabulary model skeleton". For the weights of the embedding layer and the output layer, use the reverse mapping table generated in step 213 to accurately fill the weight vector of each token in the lightweight model into the position of the corresponding general ID in the "full vocabulary model skeleton". For the positions of tokens that do not appear in the special vocabulary, keep their weights in a randomly initialized state or set to zero.

[0020] The beneficial effects of this invention are: This invention provides a localized intelligent process instruction completion method suitable for high-reliability process scenarios, which takes into account data security, intelligence, real-time performance and professionalism. Through dedicated training and local deployment of a lightweight large language model, it effectively improves the efficiency and quality of process instruction compilation, reduces the manual burden, and promotes the standardization and intelligent development of process flow. Attached Figure Description

[0021] To more clearly illustrate the technical solutions implemented in this invention, the accompanying drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this invention. For those skilled in the art, other ideas can be obtained from these drawings without creative effort.

[0022] Figure 1 This is the overall implementation flowchart; Figure 2 It is a data processing flowchart; Figure 3 This is a flowchart of lightweight model building and training; Figure 4 This is a flowchart of the model localization deployment process; Figure 5 This is the flowchart for intelligent auto-completion interaction. Detailed Implementation

[0023] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. It should be understood that the described embodiments are only a part of the embodiments of the present invention, and not all of them. All other embodiments obtained by those skilled in the art without creative effort should fall within the protection scope of the present invention.

[0024] To facilitate understanding of the technical approach of this invention and to highlight the differences from existing process instruction compilation methods, the basic process of the prior art will be briefly described below.

[0025] In current technologies, the compilation of process instructions mainly relies on manual editing, while the industry is actively exploring cloud-based large language models for assisted automatic generation. However, the instructions automatically output by large language models at present often require final manual revision due to insufficient accuracy. Furthermore, cloud models suffer from high computational power dependence, large response latency, and data security risks, making it difficult to meet the needs of localized, real-time, and data-private controllable production environments. Therefore, traditional methods have significant limitations in terms of efficiency, standardization, and security.

[0026] Specifically Figure 1 The overall process of the intelligent completion method for process instructions based on a lightweight large language model is presented, including the following steps: Step 1: Establish a high-quality domain-specific corpus through data collection, cleaning, segmentation, and FIM augmentation. Step 2: Generate a simplified vocabulary based on the domain-specific corpus, reconstruct the model architecture, and train the model. Step 3: Convert the model into a format suitable for local CPU inference using weight backfilling and quantization techniques. Step 4: Integrate an inference plugin into the editor to achieve an interactive workflow of "perception-inference-completion".

[0027] Step 11: Multi-source acquisition and cleaning of process instruction data.

[0028] Step 111: Collect raw text data from the company's historical process instruction documents and standard process templates by combining automated scripts with manual collection.

[0029] Step 112 involves basic cleaning of the collected raw text, specifically including: removing garbled characters, invisible characters, and irrelevant header and footer information from the document; standardizing full-width and half-width symbols and unit of measurement formats; and using regular expressions to remove sensitive fields to ensure data security.

[0030] Step 12: Structured segmentation and standardization of data.

[0031] Step 121: Based on the semantic integrity of the process instructions, the cleaned long text is segmented. This embodiment adopts a multi-level segmentation strategy of "document-paragraph-instruction line", using <|endoftext|> as the document end character to ensure that the model can learn the complete document boundaries.

[0032] Step 122: Standardize and validate the segmented data. Check whether each instruction contains a complete verb and object, remove invalid segments that are too short or semantically incomplete, and form a basic sequential training corpus.

[0033] Step 13: Construct an augmented dataset that supports intermediate padding (FIM).

[0034] Step 131: To enable the model to infer intermediate content from context, this step uses a prefix-suffix-intermediate pattern to enhance the basic corpus, such as... Figure 2 As shown, for each sample in the basic corpus, a 50% probability is used to determine whether to perform an FIM transformation.

[0035] Step 132: Randomly segment the selected samples for FIM transformation. A complete process instruction text sequence T is randomly segmented into three consecutive parts: prefix P, middle M, and suffix S. The segmentation points must satisfy the condition that the lengths of P, M, and S are all greater than a set minimum threshold to ensure that each part contains valid information.

[0036] Step 133: Reassemble the samples and insert control symbols. Three special control symbols are introduced: <|fim_prefix|>, <|fim_suffix|>, and <|fim_middle|>. The segmented fragments are reassembled into a new training sample sequence according to the following format: .

[0037] Step 134 involves mixing the FIM-transformed samples with the original, untransformed sequential samples to construct the final hybrid training dataset. This dataset contains both "continuing from left to right" and "filling in the middle based on the two ends" logical patterns, providing a rich data foundation for subsequent model training.

[0038] Step 21: Construct a domain-specific, minimal vocabulary.

[0039] Step 211: Perform a full scan and word segmentation statistics on the hybrid training dataset constructed in Step 1. Use the basic tokenizer of the general large model to preprocess the corpus and count all the tokens that appear and their frequencies.

[0040] Step 212: Filter valid tokens and construct mapping relationships. Extract all tokens that actually appear in the process corpus from the statistical results, and forcibly retain necessary special control characters. Compile these tokens into a new, very small dedicated vocabulary.

[0041] Step 213: Establish a bidirectional mapping index. Construct a mapping table from general terminology IDs to special terminology IDs, as well as a reverse mapping table, for subsequent model weight migration and recovery.

[0042] Step 22: Reconstruct the lightweight model network architecture.

[0043] Step 221: Based on the Transformer architecture of the general large model, initialize a new model configuration. Keep the original number of layers, attention heads, and hidden size unchanged to inherit the reasoning logic capability of the large model.

[0044] Step 222 modifies the dimensions of the embedding layer at the model input and the language model head (LMHead) at the output from the general vocabulary size to the specific vocabulary size generated in step 212. This operation significantly eliminates unused redundant weight parameters in the model, substantially reducing the total number of parameters. Figure 3 As shown.

[0045] Step 23: Implement domain-adaptive pre-training.

[0046] Step 231: Since the target model structure has changed and the number of parameters is extremely small, this embodiment adopts a random initialization strategy, that is, without loading any pre-trained weights of general large models, the model is allowed to learn the semantic representation and logical rules of the process domain from scratch.

[0047] Step 232: Configure pre-training parameters and strategies. An autoregressive language modeling objective is adopted, and a large learning rate is set to accelerate model convergence in the initial stage. A sliding window mechanism is introduced, with the window size and step size set to ensure the model can learn long-distance contextual dependencies across lines and paragraphs.

[0048] Step 233: Apply the label mask loss calculation strategy. For training samples in FIM format, when calculating the cross-entropy loss, only calculate the gradient for the "middle content" part after the <|fim_middle|> label, and set the labels of the prefix and suffix parts to ignore values.

[0049] Step 234: Perform iterative training and evaluation. Input the mixed dataset into the model for multiple rounds of iterative training, monitoring the changes in the loss function in real time. Periodically evaluate the perplexity and completion accuracy of the model on the validation set, and save the optimal lightweight model weight file.

[0050] Step 31: Since the lightweight model trained in Step 2 uses a non-standard, minimal vocabulary, direct deployment is difficult to be compatible with mainstream inference frameworks. Therefore, it needs to be restored to a standard architecture.

[0051] Step 311: Initialize the full vocabulary model skeleton. Load the standard configuration file of the general large model and initialize an empty model framework with the full vocabulary size. The number of layers, dimensions, and other parameters of this framework are consistent with the lightweight model, but the embedding layer and output layer are restored to their original size.

[0052] Step 312: Perform weight mapping and backfilling. Iterate through the weight parameters of the lightweight model: for the weights of the intermediate layers, directly copy them to the corresponding positions in the "full vocabulary model skeleton"; for the weights of the embedding and output layers, using the reverse mapping table generated in step 213, accurately fill the weight vector of each token in the lightweight model into the corresponding position of the general ID in the "full vocabulary model skeleton"; for token positions not appearing in the dedicated vocabulary, keep their weights in a randomly initialized state or set to zero. Through this step, a logically equivalent but architecturally compatible complete model file is generated, such as... Figure 4 As shown.

[0053] Step 32: Model format conversion and integer quantization.

[0054] Step 321: Convert the reconstructed standard model into the universal GGUF format. This format is optimized for CPU inference and supports single-file storage of model structure, weights, and vocabulary information, facilitating distribution and loading.

[0055] Step 322: Perform integer quantization on the GGUF model. Considering the accuracy requirements of process instruction completion, this embodiment preferably adopts a mixed-precision quantization strategy. The quantization process maps the original FP16 weight parameters to Int4 integers, further compressing the model size with almost no loss of generation quality, and significantly improving the memory read / write speed and matrix operation efficiency on the CPU.

[0056] Step 33: Integration and configuration of the localized inference engine.

[0057] Step 331: Deploy a lightweight local inference engine on the user terminal. This engine does not rely on GPU drivers such as CUDA; it can run efficiently using only the CPU instruction set.

[0058] Step 332: Configure inference service parameters. Load the quantized model file, set the context window size, and enable memory locking to prevent model data from being swapped to disk, ensuring low-latency response during inference. Start the local HTTP service port, providing standard API interfaces for upper-layer applications to call.

[0059] Step 41, integration and status monitoring of editor plugins.

[0060] Step 411: Integrate the intelligent completion plugin into the process instruction editor. This plugin acts as a client, communicating with the local inference service deployed in step 33 via the HTTP protocol.

[0061] Step 412: Monitor user editing behavior in real time. The plugin backend continuously monitors changes in cursor position and keyboard input events. To avoid resource waste caused by frequent requests, a debouncing mechanism is set up to trigger a completion request only after the user stops typing for more than a set threshold.

[0062] Step 42, dynamic routing of context extraction and inference patterns.

[0063] Step 421: Obtain the cursor context. When completion is triggered, the plugin automatically extracts the text before the current cursor position as the "previous text" and the text after the cursor position as the "next text".

[0064] Step 422: The system dynamically selects the reasoning mode based on the content characteristics of the "following text," such as... Figure 5 As shown. Mode A: If "below" is empty or only contains whitespace characters such as line breaks, it is determined that the user is at the end of the document or the beginning of a new line, and the system adopts the standard continuation writing mode. In this case, the prompt word only contains "previous"; Mode B: If "below" contains substantial text content, it is determined that the user is making an insert edit. The system adopts the FIM inference mode, constructing the prompt word according to the format <|fim_prefix|> + previous + <|fim_suffix|> + below + <|fim_middle|>.

[0065] Step 43: Generation and post-processing of the completion results.

[0066] Step 431: Send inference request. Send the constructed prompts to the / completion interface of the local inference engine and set the generation parameters.

[0067] Step 432: Receive and clean the generated results. The inference engine returns the predicted text stream. The plugin cleans the returned content in real time, truncating any duplicate generated content or illegal special tags to ensure that the output text conforms to the format specifications of the process instructions.

[0068] Step 44, Non-intrusive presentation and user adoption.

[0069] Step 441, rendering completion suggestions. The cleaned completion content will be rendered directly after the editor cursor in Ghost Text format. This display method does not change the actual content of the document; it serves only as a visual cue.

[0070] Step 442, User Interaction Decision. Acceptance: If the user presses a specific shortcut key, such as the Tab key, the system converts the gray shadow into formal text and inserts it into the document; Rejection: If the user continues to input other characters or moves the cursor, the system automatically clears the current gray shadow, without interfering with the user's normal operation. Through the above interaction process, a highly efficient "human-computer collaboration" compilation mode is achieved, retaining the flexibility of manual compilation while fully utilizing the intelligent assistance capabilities of the model.

Claims

1. A method for intelligent completion of aircraft manufacturing process instructions based on a lightweight large language model, characterized in that, Includes the following steps: Based on the characteristics of process instructions, raw text is collected from historical process documents and templates, and then cleaned and segmented into structures. A data augmentation strategy involving intermediate padding is employed, which randomly segments some continuous process instruction texts into three parts: prefix, middle, and suffix, and then reassembles them into training samples according to a specific format to form a hybrid training dataset. A dedicated vocabulary is generated based on statistical analysis of technical corpora, and the network structure of the lightweight model is reconstructed based on the dedicated vocabulary to reduce redundant parameters. Subsequently, a random initialization strategy was adopted, and the lightweight model was pre-trained using a mixed dataset; The trained lightweight model weights are mapped back to the standard model architecture, the lightweight model is quantized by integers, and the quantized model is loaded through the local inference engine. When a user edits a process instruction, the system monitors the cursor position in real time and automatically extracts the "preceding text" before the cursor and the "following text" after the cursor as the input context.

2. The method according to claim 1, characterized in that, Multi-source acquisition and cleaning of process instruction data includes the following steps: Step 111: Collect raw text data from historical process instruction documents and process templates by combining automated scripts with manual collection. Step 112 involves cleaning the collected raw text, specifically including: removing garbled characters, invisible characters, and irrelevant header and footer information from the document; standardizing full-width and half-width symbols and unit of measurement formats; and using regular expressions to remove sensitive fields.

3. The method according to claim 1, characterized in that, The structured segmentation and standardization of data includes the following steps: Step 121: Based on the semantic integrity of the process instructions, the cleaned long text is segmented using a multi-level segmentation strategy of "document-paragraph-instruction line". Step 122: Standardize and validate the segmented data, check whether each instruction contains a complete verb and object, remove invalid segments that are too short or semantically incomplete, and form a sequential training corpus.

4. The method according to claim 1, characterized in that, Building an augmented dataset that supports intermediate padding involves the following steps: Step 131: The basic corpus is enhanced using a prefix-suffix-intermediate pattern. For each sample in the basic corpus, an intermediate padding transformation is performed with a 50% probability. Step 132: Randomly segment the selected sample for intermediate padding transformation, and randomly segment a complete process instruction text sequence T into three continuous parts: prefix P, middle M, and suffix S; the segmentation points must satisfy that the lengths of P, M, and S are all greater than the set minimum threshold. Step 133 introduces three special control characters: <|fim_prefix|>, <|fim_suffix|>, and <|fim_middle|>; the segmented fragments are then reassembled into a new training sample sequence according to the following format: ； Step 134: Mix the samples that have undergone intermediate padding transformation with the original sequential samples that have not been transformed to construct the final mixed training dataset.

5. The method according to claim 1, characterized in that, Building a domain-specific thesaurus involves the following steps: Step 211: Perform a full scan and word segmentation statistics on the mixed training dataset. Use the basic word segmenter of the general large model to preprocess the corpus and count all the tokens that appear and their frequencies. Step 212: Filter valid tokens and construct mapping relationships. Extract all tokens that actually appear in the process corpus from the statistical results and forcibly retain necessary special control characters. Combine these tokens into a new special vocabulary. Step 213: Establish a bidirectional mapping index, construct a mapping table from general vocabulary IDs to special vocabulary IDs and a reverse mapping table, which will be used for subsequent model weight migration and recovery.

6. The method according to claim 1, characterized in that, Reconstructing the lightweight model network architecture includes the following steps: Step 221: Based on the Transformer architecture of the general large model, initialize a new model configuration; keep the original number of layers, number of attention heads, and hidden layer dimensions unchanged; Step 222: Modify the dimensions of the embedding layer at the model input end and the language model head at the output end from the general vocabulary size to the dedicated vocabulary size.

7. The method according to claim 4, characterized in that, Pre-training a lightweight model includes the following steps: Step 231: A random initialization strategy is adopted, without loading any pre-trained weights of general large models, and instead the model is allowed to learn the semantic representation and logical rules of the process domain from scratch. Step 232: Adopt the autoregressive language modeling objective, introduce the sliding window mechanism, set the window size and step size to ensure that the model can learn long-distance contextual dependencies across lines and paragraphs; Step 233: For training samples with intermediate padding, when calculating cross-entropy loss, only calculate the gradient for the "intermediate content" part after the <|fim_middle|> label, and set the Label of the prefix and suffix parts to ignore values; Step 234: Input the mixed dataset into the lightweight model for multiple rounds of iterative training, monitor the changes in the loss function in real time; periodically evaluate the perplexity and completion accuracy of the model on the validation set, and save the optimal lightweight model weight file.

8. The method according to claim 5, characterized in that, Mapping the weights of the trained lightweight model back to the standard model architecture includes the following steps: Step 311: Load the standard configuration file of the general large model and initialize an empty model framework with the full vocabulary size. The number of layers and dimension parameters of this framework are consistent with the lightweight model, but the embedding layer and output layer are restored to their original size. Step 312: Iterate through the weight parameters of the lightweight model. For the weights of the intermediate layer, directly copy them to the corresponding positions in the "full vocabulary model skeleton". For the weights of the embedding layer and the output layer, use the reverse mapping table generated in step 213 to accurately fill the weight vector of each token in the lightweight model into the position of the corresponding general ID in the "full vocabulary model skeleton". For the positions of tokens that do not appear in the special vocabulary, keep their weights in a randomly initialized state or set to zero.

Images