Construction method of forklift field knowledge large model with retrieval enhancement and parameter efficient fine-tuning
By constructing a knowledge base in the forklift domain and combining retrieval enhancement and efficient parameter fine-tuning methods, the problems of knowledge dispersion and unstable generation in forklift maintenance and safe operation are solved, and a highly reliable and consistent model generation is achieved under low resource conditions.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- SOUTH CHINA UNIV OF TECH
- Filing Date
- 2026-03-04
- Publication Date
- 2026-06-19
AI Technical Summary
Knowledge related to forklift maintenance and safe operation is scattered and frequently updated. Existing methods struggle to construct high-quality labeled datasets in low-resource domains, leading to model overfitting, untimely knowledge updates, a lack of domain-specific consistency in retrieval enhancements, and unstable generation performance.
By combining retrieval enhancement and efficient parameter fine-tuning, a forklift domain knowledge base is constructed to generate retrieval enhancement input samples. The DoRA module is then introduced into the pre-trained large language model for supervised fine-tuning to learn the forklift domain's expression methods and safety regulations.
By constructing a highly reliable domain knowledge model under low resource conditions, the problems of model overfitting and unstable generation were solved, and a large forklift knowledge domain model with consistent knowledge coverage and high safety and reliability was achieved.
Smart Images
Figure CN122242637A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to a method for constructing a large knowledge model in the forklift field with enhanced retrieval and efficient parameter fine-tuning, belonging to the field of artificial intelligence and natural language processing technology. Background Technology
[0002] Currently, forklifts are a type of special equipment widely used in industrial logistics and warehousing. Their maintenance and safe operation are highly specialized and safety-sensitive. However, forklift-related knowledge is usually scattered across various operation manuals, maintenance specifications, safety standards, and experience documents, resulting in practical problems such as inconsistent structure, frequent updates, and high human learning costs.
[0003] In recent years, large-scale pre-trained language models have been gradually introduced into the fields of industrial knowledge question answering and decision support. However, they still face a series of technical challenges in specific scenarios such as forklift maintenance and safety. First, the scale of domain training data is limited. Forklift maintenance and safety operations are typical low-resource domains, making it difficult to construct large-scale, high-quality labeled datasets. Directly fine-tuning all parameters of a large model is not only costly but also prone to overfitting. Second, efficient parameter fine-tuning carries the risk of degradation under small data conditions. Existing methods such as LoRA and DoRA adjust model behavior by introducing a small number of trainable parameters, but under small-scale domain data conditions, the model is prone to overfitting parameter updates, leading to unstable or degraded generation performance. Simultaneously, it is difficult to balance model parameter memorization and knowledge updating. Forklift maintenance and safety regulations are dynamically updated, and relying solely on model parameter memorization cannot reflect the latest standards and requirements in a timely manner. Furthermore, simple retrieval enhancement methods lack domain representation consistency. Although retrieval enhancement generation methods can introduce external knowledge, the model lacks the ability to learn the language style, safety warning structure, and operational procedure organization methods of the forklift domain, affecting the professionalism and reliability of the answers. Summary of the Invention
[0004] This invention provides a method for constructing a large knowledge model for the forklift domain with enhanced retrieval and efficient parameter fine-tuning, aiming to solve at least one of the technical problems existing in the prior art.
[0005] The technical solution of this invention relates to a method for constructing a large knowledge model for the forklift domain based on retrieval enhancement and efficient parameter fine-tuning. The method according to this invention includes the following steps:
[0006] S100. Collect forklift maintenance manuals, safety operation specifications, and maintenance process documents as raw text data. Clean the raw text data and generate a standardized format file according to the model input requirements to generate a searchable forklift domain knowledge base.
[0007] S200. Obtain the original forklift domain training sample set. For forklift-related questions in the training samples, retrieve knowledge content related to the question from the forklift domain knowledge base, and embed the retrieval results into the model instructions to generate retrieval enhancement input samples containing retrieval enhancement context information.
[0008] S300. Obtain the original pre-trained large language model, introduce a parameter-efficient fine-tuning module into the model to adjust the model's generation behavior in the forklift domain task, so as to obtain the target model architecture with the parameter-efficient fine-tuning module introduced and to be fine-tuned.
[0009] S400. Using the retrieval enhancement input sample as input, supervised fine-tuning training is performed on the target model architecture with the introduced parameter high-efficiency fine-tuning module, so that the model learns the expression methods and safety standards of the forklift domain, and obtains the trained forklift domain knowledge big model and its corresponding fine-tuning parameters.
[0010] S500. Based on the forklift domain knowledge base, use the trained forklift domain knowledge model to answer questions, provide decision support, or perform strategy analysis on user-submitted questions regarding forklift maintenance, safe operation, and troubleshooting.
[0011] Furthermore, in step S300, the parameter efficient fine-tuning module is a DoRA module, and the projection layer weight matrix applied by the DoRA module to the Transformer attention mechanism of the pre-trained large language model includes the Query projection matrix, Key projection matrix, Value projection matrix, and Output projection matrix.
[0012] Furthermore, in step S300, the DoRA module adjusts the weight matrix of any linear layer that needs fine-tuning. Represented as:
[0013] ,
[0014] In the formula, These are the original weights of the pre-trained model, which are frozen and not updated. It is a low-rank update term, usually in LoRA form. ,in , r is the rank. Indicates the dimension of the input features. Indicates the dimension of the output feature; It is a learnable amplitude parameter; It involves normalizing the weights using the norm.
[0015] Furthermore, in step S200, the enhanced input sample is formed by constructing enhanced instructions, which sequentially include: setting the system role as a professional assistant for forklift maintenance and safe operation; clarifying the constraint rule that the model only answers based on the search content and provides corresponding prompts when information is insufficient; listing the Top-K knowledge items obtained through the search module (21); and using the original question as the input part.
[0016] Furthermore, in step S300, the supervised fine-tuning training enables the model to learn how to reference and organize retrieved forklift maintenance knowledge, how to generate structured operational steps such as inspection, confirmation, processing, and re-inspection, how to appropriately generate safety prompts and risk warnings, and how to output standardized responses indicating insufficient information when information is insufficient.
[0017] Furthermore, in step S300, model evaluation is performed during or after the supervised fine-tuning training, wherein the model evaluation operation includes the steps of: testing the model with a forklift domain validation set and calculating at least one evaluation metric including BLEU-4, ROUGE-1, ROUGE-2, and ROUGE-L.
[0018] The present invention also relates to a computer-readable storage medium having program instructions stored thereon, which, when executed by a processor, implement the above-described method.
[0019] The technical solution of the present invention also relates to a large knowledge model system for the forklift field based on retrieval enhancement and efficient parameter fine-tuning, the system including a computer device that includes the aforementioned computer-readable storage medium.
[0020] The technical solution of the present invention also relates to a large knowledge model for the forklift domain based on retrieval enhancement and efficient parameter fine-tuning. The system includes a forklift domain knowledge base module (10), a retrieval enhancement generation module (20), and a model training and evaluation module (30). The forklift domain knowledge base module (10) is used to clean and organize the original text data of forklift maintenance manuals, safety operation specifications, and maintenance process documents into standardized format files to construct a structured searchable knowledge base. The retrieval enhancement generation module (20) is used to retrieve relevant knowledge content from the knowledge base for the problems in the original training samples, and embed the retrieval results into the model instructions according to a preset template to form an enhanced training sample set. The model training and evaluation module (30) is used to introduce an efficient parameter fine-tuning module into the pre-trained large language model, and perform supervised fine-tuning training and performance evaluation of the model based on the enhanced training sample set.
[0021] Furthermore, the model training and evaluation module (30) includes a large language model module (31), a DoRA module (32), and a training and evaluation module (33); the large language model module (31) is a pre-trained model based on the Transformer architecture; the DoRA module (32) is integrated into the large language model module (31) and specifically acts on the attention projection layer weight matrix of the large language model module (31); the training and evaluation module (33) is used to load the enhanced training sample set, perform supervised fine-tuning on the large language model module (31) that integrates the DoRA module (32), and calculate the evaluation index based on the validation set.
[0022] Furthermore, the training evaluation module (33) pre-constructs all enhanced training samples in the training phase using an offline retrieval method; the workflow of the training evaluation module (33) includes: reading the training set after retrieval and enhancement; loading the pre-trained large language model; introducing the DoRA module (32) into the attention projection layer and applying it to the projection matrix; training and updating the low-rank parameters and amplitude parameters using supervised fine-tuning; saving the fine-tuning weights for subsequent inference deployment and evaluating various generated metrics on the validation set.
[0023] Furthermore, it also includes an inference deployment module (40), which is used to receive user input of forklift domain questions, call the trained forklift domain knowledge big model, and combine the forklift domain knowledge base to perform real-time retrieval and answer generation.
[0024] The beneficial effects of this invention are as follows:
[0025] This invention relates to a method and system for constructing a large-scale knowledge model for the forklift domain based on retrieval enhancement and efficient parameter fine-tuning. It combines retrieval enhancement generation with efficient parameter fine-tuning techniques to build a highly reliable domain knowledge model under low-resource domain data conditions. This model enables stable training under low-resource conditions while maintaining knowledge coverage, domain representation consistency, and safety reliability. Furthermore, this invention avoids parameter fine-tuning degradation through external knowledge retrieval, maintaining training stability even under low-resource domain data conditions. This addresses the problems of unstable efficient parameter fine-tuning and lack of domain consistency in retrieval enhancement in existing technologies. Attached Figure Description
[0026] Figure 1 This is a schematic diagram of the overall architecture according to an embodiment of the present invention.
[0027] Figure 2 This is a flowchart illustrating the combination of search enhancement and efficient parameter fine-tuning according to an embodiment of the present invention.
[0028] Figure 3 This is a flowchart illustrating the training and evaluation module according to an embodiment of the present invention.
[0029] Figure label:
[0030] 10. Forklift Domain Knowledge Base Module; 11. Forklift Maintenance Knowledge Module; 12. Data Cleaning and Labeling Module; 20. Retrieval Enhancement Generation Module; 21. Retrieval Module; 22. Retrieval Enhancement Sample Construction Module; 30. Model Training and Evaluation Module; 31. Large Language Model Module; 32. DoRA Module; 33. Training and Evaluation Module; 40. Inference Deployment Module. Detailed Implementation
[0031] The following will provide a clear and complete description of the concept, specific structure, and technical effects of the present invention in conjunction with the embodiments and accompanying drawings, so as to fully understand the purpose, solution, and effects of the present invention.
[0032] It should be noted that, unless otherwise specified, when a feature is referred to as "fixed" or "connected" to another feature, it can be directly fixed or connected to the other feature, or indirectly fixed or connected to the other feature. The singular forms "a," "described," and "the" used herein are also intended to include the plural forms, unless the context clearly indicates otherwise. Furthermore, unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art. The terminology used in this specification is for the purpose of describing particular embodiments only and not for limiting the invention. The term "and / or" as used herein includes any combination of one or more of the associated listed items.
[0033] It should be understood that although the terms first, second, third, etc., may be used to describe various elements in this disclosure, these elements should not be limited to these terms. These terms are used only to distinguish elements of the same type from one another. For example, a first element may also be referred to as a second element without departing from the scope of this disclosure, and similarly, a second element may also be referred to as a first element. Any and all instances or exemplary language (“e.g.,” “such as,” etc.) provided herein are intended only to better illustrate embodiments of the invention and, unless otherwise required, do not impose a limitation on the scope of the invention.
[0034] Reference Figures 1 to 3 In some embodiments, the method for constructing a large model of forklift knowledge domain with retrieval enhancement and efficient parameter fine-tuning according to the present invention includes at least the following steps:
[0035] S100: Collect forklift maintenance manuals, safety operation specifications, and maintenance process documents as raw text data. Clean the raw text data and generate standardized format files according to the model input requirements to generate a searchable forklift domain knowledge base.
[0036] S200. Obtain the original forklift domain training sample set. For forklift-related questions in the training samples, retrieve knowledge content related to the question from the forklift domain knowledge base and embed the retrieval results into the model instructions to generate retrieval-enhanced input samples containing retrieval-enhanced context information.
[0037] S300. Obtain the original pre-trained large language model, introduce a parameter-efficient fine-tuning module into the model to adjust the model's generation behavior in the forklift domain task, so as to obtain the target model architecture with the parameter-efficient fine-tuning module introduced and to be fine-tuned.
[0038] S400: Using the enhanced input samples as input, supervised fine-tuning training is performed on the target model architecture with the introduced parameter fine-tuning module, so that the model learns the expression and safety specifications of the forklift domain, and obtains the trained forklift domain knowledge model and its corresponding fine-tuning parameters.
[0039] S500: Based on the forklift domain knowledge base, the trained forklift domain knowledge model is used to answer questions, provide decision support, or perform strategy analysis on user-submitted questions regarding forklift maintenance, safe operation, and troubleshooting.
[0040] Reference Figures 1 to 3 In some embodiments, the forklift knowledge domain large model training system based on retrieval enhancement and efficient parameter fine-tuning according to the present invention includes a forklift domain knowledge base module 10, a retrieval enhancement generation module 20, and a model training method and evaluation module 30.
[0041] The forklift knowledge base module 10 includes a forklift maintenance knowledge acquisition module 11 and a data cleaning and labeling module 12. The acquisition methods of the forklift maintenance knowledge module 11 include textual knowledge such as forklift maintenance manuals, safe operating procedures, troubleshooting processes, and maintenance cycles. The data cleaning and labeling module 12 mainly organizes the data into a standardized JSON format that the model can recognize.
[0042] The retrieval enhancement generation module 20 includes a retrieval module 21 and a retrieval enhancement sample construction module 22. The retrieval enhancement generation module 20 is used to retrieve relevant knowledge content from the knowledge base for the questions in the original training samples, and embed the retrieval results into the model instructions according to a preset template to form an enhanced training sample set.
[0043] The evaluation module 30 includes a large language model module 31, a DoRA module 32, and a training and evaluation module 33. The model training and evaluation module 30 introduces an efficient parameter fine-tuning module into the pre-trained large language model and performs supervised fine-tuning training and performance evaluation based on an enhanced training sample set. The large language model module 31 is a pre-trained model based on the Transformer architecture; the DoRA module 32 is integrated into the large language model module 31 and specifically acts on the attention projection layer weight matrix of the large language model module 31; the training and evaluation module 33 loads the enhanced training sample set, performs supervised fine-tuning of the large language model module 31 integrating the DoRA module 32, and calculates evaluation metrics based on the validation set. During the training phase, the training and evaluation module 33 pre-constructs all enhanced training samples using an offline retrieval method and freezes the original weights of the large language model module 31 during training, updating only the trainable parameters introduced by the DoRA module 32.
[0044] Furthermore, the system of the present invention also includes an inference deployment module 40, which is used to receive user input of forklift domain questions, call the trained forklift domain knowledge big model, and perform real-time retrieval and answer generation in conjunction with the forklift domain knowledge base.
[0045] This invention presents a retrieval-augmented generation (RAG) method and system for training large-scale forklift knowledge models in the field of forklift maintenance and safe operation. It combines RAG with efficient parameter fine-tuning techniques (such as Doray tuning) to build a highly reliable domain knowledge model under low-resource domain data conditions. This results in a large-scale forklift knowledge model that can be stably trained under low-resource conditions while maintaining knowledge coverage, domain representation consistency, and safety reliability. Furthermore, this invention avoids parameter fine-tuning degradation through external knowledge retrieval, maintaining training stability even under low-resource domain data conditions. This solves the problems of unstable efficient parameter fine-tuning and lack of domain consistency in retrieval enhancement in existing technologies.
[0046] In some embodiments of the present invention, the model training method of the present invention performs offline RAG on the original instruction, input, and output format JSON file, adds RAG instructions and retrieved forklift maintenance knowledge to the instruction part, and generates a new JSON file for subsequent training. Thus, based on the forklift domain knowledge base, for each question in the training set, a retrieval method is used to obtain highly relevant knowledge items, and the retrieval results and corresponding instruction templates are embedded into the instruction field of the original data to form an enhanced training sample.
[0047] Specifically, this invention addresses the problem of limited training data scale and sparse knowledge distribution in the field of forklift maintenance and safe operation. By introducing a retrieval-enhanced generation mechanism, the model prioritizes the use of external domain knowledge rather than relying on parameter memory during training and inference, effectively avoiding the performance degradation and instability that easily occur under small data conditions when efficiently fine-tuning parameters.
[0048] In some embodiments of the present invention, the model training method uses new data processed by offline RAG as model input, and then trains it using DoRA. Supervised fine-tuning is performed on the base large language model integrating the DoRA module using training samples constructed with retrieval enhancement. Only the low-rank matrix and magnitude parameters in DoRA are updated, while the original pre-trained weights are frozen. Specifically, the RAG-processed data pre-feeds the model with external knowledge highly relevant to the answer during training, transforming it into an output based on knowledge text and the question. That is, its output becomes context-based generation, essentially changing from memory-based generation to conditional generation or reading comprehension, thereby effectively improving the stability of the DoRA method under small data training conditions.
[0049] In some embodiments of the present invention, the model training method of the present invention performs model evaluation during or after supervised fine-tuning training. The model evaluation operation includes the steps of: testing the model using a forklift domain validation set and calculating the evaluation metrics of the introduced evaluation mechanism. Further, the introduced evaluation mechanism includes evaluation metrics such as BLEU-4, ROUGE-1, ROUGE-2, and ROUGE-L, used to quantify the quality of the model-generated results. Specifically, BLEU-4 represents the matching degree between a fragment consisting of four consecutive words and the answer, used to measure whether the expression is the same as the database output; ROUGE-1 represents word coverage, used to measure whether the information points are fully covered; ROUGE-2 represents consecutive two-word coverage, used to measure whether key phrases are consistent; and ROUGE-L represents the longest common subsequence, used to measure whether sentence structures are similar.
[0050] Specifically, this invention calculates the above-mentioned metrics on a validation set to measure the performance of the generated text in terms of phrase matching, word coverage, key phrase consistency, and sentence structure similarity, and analyzes the difference in effect between using DoRA alone and combining RAG+DoRA through comparative experiments.
[0051] Through comparative experiments, it can be seen that under the data conditions of low-resource forklifts, the existing methods, when using parameter fine-tuning alone, exhibit unstable or even degraded model performance. However, after introducing the retrieval enhancement mechanism in this invention, the model generation results show more stable performance in terms of knowledge coverage completeness and structural consistency. Therefore, the RAG combined with DoRA fine-tuning method of this invention has superiority.
[0052] In some embodiments of the present invention, see Figure 2 By combining the retrieval enhancement generation module 20 with the model training method and evaluation module 30, a combination of RAG and DoRA fine-tuning is achieved. Specifically, compared to the traditional use of DoRA fine-tuning alone, this invention provides the model with external knowledge context highly relevant to the forklift problem during training and inference, enabling the model to shift from relying on parameter memory to generating answers under given evidence conditions. Thus, through the retrieval enhancement mechanism, the model's dependence on parameter memory can be significantly reduced under low-resource forklift data conditions, improving the knowledge coverage and safety specification consistency of the answers.
[0053] In some embodiments of the present invention, the retrieval module 21 employs an offline TF-IDF retrieval method to ensure reproducibility. Specifically, the knowledge entries are vectorized into TF-IDF representations through the offline retrieval method of the retrieval module 21, the cosine similarity between the question vector and the knowledge vector is calculated, and the Top-K representation is taken; then, the Top-K knowledge entries are pre-retrieved and solidified for each question in the training set and validation set, thereby effectively avoiding the uncertainty of online retrieval.
[0054] In some embodiments of the present invention, the retrieval enhancement sample construction module embeds the retrieved knowledge content into the model input according to a unified template. For example, it constructs an enhanced instruction_rag. Specifically, firstly, the system role is set as a professional assistant for forklift maintenance and safe operation. Then, the constraint rule is defined so that the model only answers based on the retrieved content and provides corresponding prompts when information is insufficient. Subsequently, the retrieval module 21 lists the Top-K knowledge entries obtained from the retrieval, each of which includes a title and content. Finally, the original question is used as the input. Further, a complete training sample is formed on this basis, where the instruction part adopts the aforementioned constructed instruction_rag, the input part is the original question, and the output part corresponds to the reference answer, thereby constructing a training data format suitable for subsequent supervised fine-tuning.
[0055] In some embodiments of the present invention, the large language model module 31 adopts the Transformer multi-head attention mechanism. In each layer of the Transformer, the multi-head self-attention module typically includes a linear transformation of the input latent state vector X to generate Query, Key, and Value, which are represented as follows:
[0056]
[0057] In the formula, , , These represent the projection matrices for Query, Key, and Value, respectively. The attention output is also included. It is expressed as follows:
[0058]
[0059] After multi-head attention calculation and concatenation, the output projection matrix is used. The integrated vector of the final output of the current layer obtained by performing a linear transformation. It is expressed as follows:
[0060]
[0061] In the formula, the function This indicates a concatenation operation, which joins multiple vectors of different dimensions together in sequence to form a larger vector. This represents the memory calculation result for each head. Indicates the number of heads.
[0062] in, It is the location of key parameters that affect the model's attention behavior and generation style.
[0063] In some embodiments of the present invention, the DoRA module 32, based on the decoupling of the direction and magnitude of the weights, primarily adjusts the direction through low-rank updates, while introducing magnitude parameters to enhance expressive power, thereby achieving effective fine-tuning with a small number of trainable parameters. Specifically, for any linear layer weight matrix that needs fine-tuning... The DoRA module represents it as:
[0064]
[0065] In the formula, These are the original weights of the pre-trained model, which are frozen and not updated. It is a low-rank update term, usually in LoRA form. ,in , r is the rank. Indicates the dimension of the input features. Indicates the dimension of the output feature; It is a learnable amplitude parameter. Specifically, settings can be configured for each output channel or the entire matrix. This involves normalizing the weights using the norm to make the updates more stable.
[0066] It should be noted that the DoRA applied to the projection layer weights of the Transformer attention mechanism in this invention includes at least the Query projection matrix, Key projection matrix, Value projection matrix, and Output projection matrix. By introducing the DoRA structure onto the above matrices for each attention module, the organization of answers in the forklift domain can be learned more effectively.
[0067] In some embodiments of the present invention, the update strategy adopted by the DoRA module during training is as follows: during training, the original weights of the pre-trained model are frozen. Only update the low-rank parameters. and and amplitude parameters The model uses supervised learning to predict the output under the condition of instruction + rag. That is, it uses supervised learning to predict the output answer under the input condition composed of enhanced instructions and retrieved knowledge, so that the model learns to generate a response that conforms to forklift maintenance and safety specifications under the constraint of retrieved evidence.
[0068] Understandably, in the context of small-scale training data in the forklift field, relying solely on the DoRA module to allow the model to memorize a large amount of maintenance fact details can easily lead to undergeneralization or overfitting to specific representations. However, this invention introduces a retrieval enhancement mechanism, where knowledge content is dynamically provided by an external knowledge base. Specifically, the model fine-tuning process focuses on enabling the model to learn how to reference and organize retrieved forklift maintenance knowledge, how to generate structured operational steps such as inspection, confirmation, processing, and re-inspection, how to appropriately generate safety prompts and risk warnings, and how to output standardized responses when information is insufficient. Based on these mechanisms, this invention delegates the knowledge supply task to the retrieval enhancement generation module, while entrusting the representation learning and standardization tasks to the DoRA module. This effectively reduces the performance degradation risk of parameter fine-tuning under low-resource conditions and significantly improves the model's deployability and reliability in real-world industrial scenarios.
[0069] In some embodiments of the present invention, the training evaluation module pre-constructs enhanced retrieval samples using an offline retrieval method during the training phase to ensure the stability and reproducibility of the training process. See also Figure 3 The workflow of the training and evaluation module includes at least the following: reading the training set after retrieval enhancement; loading the pre-trained large language model; introducing the DoRA module into the attention projection layer and applying it to the projection matrix. The low-rank parameters are trained and updated using a supervised fine-tuning method. With amplitude parameter Save the fine-tuned weights for subsequent inference deployment and for evaluating various generated metrics on the validation set.
[0070] Furthermore, in the model inference stage, the system first performs domain knowledge retrieval on the forklift question input by the user, embedding the retrieved forklift maintenance and safety knowledge as context into the model input, and then generating the final answer from a large language model that has undergone efficient parameter fine-tuning. Specifically, based on the optimized model obtained from the training and evaluation module, the model inference process of this invention includes at least the following: invoking the model that has completed retrieval enhancement and DoRA training as an expert system during inference; receiving the forklift question proposed by the user; retrieving the most relevant knowledge entries from the knowledge base; constructing enhanced retrieval input; and generating and outputting the answer by the model. The model inference process of this invention ensures that the model is always under the constraint of external knowledge during inference, thereby effectively improving the reliability and security of the generated answer.
[0071] This invention constructs a forklift domain knowledge base. It collects texts from forklift maintenance manuals, safety operation specifications, and maintenance process documents, cleans this data, and generates standardized format files according to model input requirements, thus forming a searchable forklift domain knowledge base. Furthermore, by generating retrieval-enhanced input samples, forklift-related questions in the training samples, it retrieves relevant knowledge content from the knowledge base and embeds the retrieval results into the model instructions, thereby forming retrieval-enhanced input. Based on the retrieval-enhanced parameter fine-tuning model training, this invention performs supervised fine-tuning training on the model incorporating the DoRA module under retrieval-enhanced input conditions, enabling the model to effectively learn forklift domain expressions and safety specifications. Finally, this invention utilizes the trained model to perform question-and-answer or decision-aid assistance on forklift maintenance, safe operation, and troubleshooting issues, realizing effective model reasoning and application.
[0072] It should be noted that this invention deploys the system on a traditional forklift, enabling high-precision real-time identification of industrial pallets and fork positions. Combined with the integrated visual interface provided by the interactive auxiliary pickup unit, it can effectively assist relevant operators in completing various tasks conveniently. This not only significantly improves the efficiency of pallet pickup operations but also reduces the reliance on the professional skills of forklift operators in actual operation.
[0073] This invention dynamically incorporates forklift maintenance knowledge and safety regulations into the model input via retrieval, reducing the model's need for parameterized storage of domain-specific knowledge. This makes the model's generated results less sensitive to changes in the training data distribution, thereby improving the system's reliability and maintainability in real-world industrial environments. Furthermore, by providing the model with forklift maintenance steps, safety warnings, and operating procedures highly relevant to the problem through a retrieval enhancement mechanism, the model's generated responses are more reasonable and consistent in terms of knowledge coverage, operational sequence, and expression of safety precautions, making them suitable for forklift operation scenarios with high safety requirements. Moreover, compared to simply applying constraint functions to the amplitude of parameters through an efficient parameter fine-tuning module to improve stability, this invention, by introducing a retrieval enhancement mechanism, can replace or weaken the impact of parameter constraints on model performance when the effectiveness of parameter constraints is limited under low-resource conditions, thus avoiding the applicability boundaries of a single parameter adjustment scheme at the system level.
[0074] It should be understood that the method steps in the embodiments of the present invention can be implemented or carried out by computer hardware, a combination of hardware and software, or by computer instructions stored in a non-transitory computer-readable storage medium. The method can use standard programming techniques. Each program can be implemented in a high-level procedural or object-oriented programming language to communicate with the computer system. However, if necessary, the program can be implemented in assembly or machine language. In any case, the language can be a compiled or interpreted language. Furthermore, for this purpose, the program can run on a programmed application-specific integrated circuit (ASIC).
[0075] Furthermore, the procedures described herein may be performed in any suitable order unless otherwise indicated herein or otherwise clearly contradicted by the context. The procedures described herein (or variations and / or combinations thereof) may be executed under the control of one or more computer systems configured with executable instructions, and may be implemented by hardware or a combination thereof as code (e.g., executable instructions, one or more computer programs, or one or more applications) that commonly executes on one or more processors. The computer program comprises a plurality of instructions executable by one or more processors.
[0076] Furthermore, the method can be implemented in any suitable type of computing platform, including but not limited to personal computers, minicomputers, mainframes, workstations, networked or distributed computing environments, standalone or integrated computer platforms, or in communication with charged particle tools or other imaging devices, etc. Aspects of the invention can be implemented as machine-readable code stored on a non-transitory storage medium or device, whether removable or integrated into a computing platform, such as a hard disk, optical read and / or write storage medium, RSM, ROM, etc., such that it can be read by a programmable computer, and when the storage medium or device is read by the computer, it can be used to configure and operate the computer to perform the processes described herein. Furthermore, the machine-readable code, or portions thereof, can be transmitted via wired or wireless networks. The invention described herein includes these and other different types of non-transitory computer-readable storage media when such media comprises instructions or programs that implement the steps described above in conjunction with a microprocessor or other data processor. When programmed according to the methods and techniques described in the invention, the invention may also include the computer itself.
[0077] A computer program can be applied to input data to perform the functions described herein, thereby transforming the input data to generate output data stored in non-volatile memory. The output information can also be applied to one or more output devices, such as a display. In a preferred embodiment of the invention, the transformed data represents physical and tangible objects, including specific visual depictions of physical and tangible objects generated on the display.
[0078] The above description is merely a preferred embodiment of the present invention. The present invention is not limited to the above-described embodiments. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention, as long as they achieve the technical effects of the present invention by the same means, should be included within the scope of protection of the present invention. Within the scope of protection of the present invention, the technical solutions and / or implementation methods can have various modifications and variations.
Claims
1. A method for constructing a large knowledge model for the forklift domain based on retrieval enhancement and efficient parameter fine-tuning, characterized in that: The method includes the following steps: S100. Collect forklift maintenance manuals, safety operation specifications, and maintenance process documents as raw text data. Clean the raw text data and generate a standardized format file according to the model input requirements to generate a searchable forklift domain knowledge base. S200. Obtain the original forklift domain training sample set. For forklift-related questions in the training samples, retrieve knowledge content related to the question from the forklift domain knowledge base, and embed the retrieval results into the model instructions to generate retrieval enhancement input samples containing retrieval enhancement context information. S300. Obtain the original pre-trained large language model, introduce a parameter-efficient fine-tuning module into the model to adjust the model's generation behavior in the forklift domain task, so as to obtain the target model architecture with the parameter-efficient fine-tuning module introduced and to be fine-tuned. S400. Using the retrieval enhancement input sample as input, supervised fine-tuning training is performed on the target model architecture with the introduced parameter high-efficiency fine-tuning module, so that the model learns the expression methods and safety standards of the forklift domain, and obtains the trained forklift domain knowledge big model and its corresponding fine-tuning parameters. S500. Based on the forklift domain knowledge base, use the trained forklift domain knowledge model to answer questions, provide decision support, or perform strategy analysis on user-submitted questions regarding forklift maintenance, safe operation, and troubleshooting.
2. The method according to claim 1, characterized in that, In step S300, the parameter fine-tuning module is a DoRA module, and the projection layer weight matrix applied by the DoRA module to the Transformer attention mechanism of the pre-trained large language model includes the Query projection matrix, Key projection matrix, Value projection matrix and Output projection matrix.
3. The method according to claim 2, characterized in that, In step S300, the DoRA module adjusts the weight matrix of any linear layer that needs fine-tuning. Represented as: ， In the formula, These are the original weights of the pre-trained model, which are frozen and not updated. It is a low-rank update term, usually in LoRA form. ,in , r is the rank. Indicates the dimension of the input features. Indicates the dimension of the output feature; It is a learnable amplitude parameter; It involves normalizing the weights using the norm.
4. The method according to claim 1, characterized in that, In step S200, the enhanced input sample is formed by constructing enhanced instructions, which include: setting the system role as a professional assistant for forklift maintenance and safe operation; clarifying the constraint rule that the model only answers based on the search content and gives corresponding prompts when the information is insufficient; listing the Top-K knowledge items obtained by the search through the search module (21); and taking the original question as the input part.
5. The method according to claim 1, characterized in that, In step S300, the supervised fine-tuning training enables the model to learn how to reference and organize retrieved forklift maintenance knowledge, how to generate structured operational steps such as inspection, confirmation, processing, and re-inspection, how to appropriately generate safety prompts and risk warnings, and how to output standardized responses for insufficient information when information is insufficient.
6. The method according to claim 3, characterized in that, In step S300, model evaluation is performed during or after the supervised fine-tuning training, wherein the model evaluation operation includes the steps of: testing the model with a forklift domain validation set and calculating at least one evaluation metric including BLEU-4, ROUGE-1, ROUGE-2, and ROUGE-L.
7. A large-scale knowledge model system for the forklift domain based on retrieval enhancement and efficient parameter fine-tuning, characterized in that: The method described in any one of claims 1 to 6 is performed during execution; The system includes: a forklift domain knowledge base module (10), a retrieval enhancement generation module (20), and a model training and evaluation module (30); The forklift domain knowledge base module (10) is used to clean and organize the original text data of forklift maintenance manuals, safety operation specifications and maintenance process documents into standardized format files to build a structured searchable knowledge base. The retrieval enhancement generation module (20) is used to retrieve relevant knowledge content from the knowledge base for the questions in the original training samples, and embed the retrieval results into the model instructions according to the preset template to form an enhanced training sample set; The model training and evaluation module (30) is used to introduce a parameter-efficient fine-tuning module into the pre-trained large language model and to perform supervised fine-tuning training and performance evaluation of the model based on the enhanced training sample set.
8. The system according to claim 7, characterized in that, The model training and evaluation module (30) includes a large language model module (31), a DoRA module (32), and a training and evaluation module (33). The large language model module (31) is a pre-trained model based on the Transformer architecture. The DoRA module (32) is integrated into the large language model module (31) and specifically acts on the attention projection layer weight matrix of the large language model module (31). The training and evaluation module (33) is used to load the enhanced training sample set, perform supervised fine-tuning on the large language model module (31) that integrates the DoRA module (32), and calculate the evaluation index based on the validation set.
9. The system according to claim 8, characterized in that, The training evaluation module (33) pre-constructs all enhanced training samples in the training phase using an offline retrieval method; The workflow of the training evaluation module (33) includes: reading the training set after retrieval enhancement; Load the pre-trained large language model; introduce the DoRA module (32) into the attention projection layer and apply it to the projection matrix; train and update the low-rank parameters and magnitude parameters using supervised fine-tuning; save the fine-tuned weights for subsequent inference deployment and to evaluate various generation metrics on the validation set.
10. The system according to claim 7, characterized in that, It also includes an inference deployment module (40), which is used to receive user input of forklift domain questions, call the trained forklift domain knowledge big model, and combine the forklift domain knowledge base to perform real-time retrieval and answer generation.