Automobile Q&A Method with Multi-Adapter Fine-Tuning, Computer Device and Storage Medium

Abstract

The present invention provides a method for fine-tuning a multi-adapter for automotive Q&A, a computer device, and a storage medium. The method includes: establishing and pre-training a base large model; fine-tuning the base large model based on a maintenance fault problem dataset to obtain a maintenance fault adapter; fine-tuning the base large model based on a vehicle usage problem dataset to obtain a vehicle usage problem adapter; fine-tuning the base large model based on a general domain problem dataset to obtain a general domain problem adapter; and fine-tuning the base large model based on an intent recognition task dataset to obtain an intent recognition adapter. The present invention can perform separate fine-tuning according to problems in different fields, ensure the specificity of each field, thereby improving the accuracy of field adaptation and ensuring consistent performance. The present invention is a major improvement over the existing single fine-tuning method and can effectively solve the problem of domain deviation.

Description

Technical Field

[0001] The present invention belongs to the technical field of automotive information and communication technology, and particularly relates to a method for fine-tuning a multi-adapter (Muti-Adapter) in an automotive question-answering system, a computer device, and a storage medium. Background Art

[0002] With the development of large language models, question-answering systems based on large models have shown excellent performance in handling complex questions. Large models have powerful natural language processing capabilities, can understand and generate rich language expressions, and greatly improve the performance of question-answering systems. However, most current automotive question-answering systems are trained with pre-set rules and a large amount of labeled data, which makes them inefficient or inaccurate in handling various questions. Although large models perform well in general fields, they still face challenges in adapting to specific fields. Especially in the automotive field, which involves a large amount of professional knowledge and specific domain questions, general large models are difficult to accurately answer these professional questions.

[0003] Existing automotive question-answering systems have a single function and cannot handle maintenance and troubleshooting questions, vehicle usage questions, and general domain questions simultaneously. When facing different types of questions, they often need to rely on multiple (more than 1) independent systems or modules, resulting in resource waste and inconvenience in use. Moreover, current automotive intelligent question-answering systems mainly rely on pre-built knowledge bases for question-answering interactions, and this method shows obvious limitations when facing new questions or questions not included in the knowledge base. In addition, the fine-tuning techniques of existing systems usually do not distinguish between domains and use a single fine-tuning method. This leads to possible domain biases when dealing with questions in different domains, because the dataset scales and characteristics of each domain are different, and consistent performance cannot be guaranteed. This method ignores the specificity of different domains and is difficult to achieve precise domain adaptation.

[0004] The construction and maintenance of the said knowledge base require a large amount of manpower and resources. Especially when dealing with frequently updated automotive technologies and fault information, the cost further increases. Among them, data collection, annotation, and system update are all time-consuming and laborious processes, increasing the overall operating cost. In addition, the existing full-parameter fine-tuning method is costly, not only requiring a large amount of computing resources, but also the fine-tuned model is not pluggable and cannot be updated or replaced at any time, which further increases the system maintenance and upgrade costs.

[0005] Traditional rule- and knowledge-base-based methods are inefficient in handling multi-turn questions and complex context-related questions. The system needs to frequently update the knowledge base when dealing with new questions, affecting the response speed and user experience. At the same time, existing fine-tuning methods cannot efficiently adapt to questions in different domains, resulting in low efficiency of the system in dealing with complex and diverse questions. Summary of the Invention

[0006] In view of the above problems, the present invention provides a method for fine-tuning a Muti-Adapter for automotive question answering, a computer device, and a storage medium.

[0007] The method for fine-tuning a multi-adapter for automotive question answering provided by the present invention is characterized in that a base large model is used to obtain and apply a maintenance fault adapter, a vehicle use question adapter, a general domain question adapter, and an intent recognition adapter, and then question answering for automotive problems is performed, including the steps of:

[0008] S1. Establishment and pre-training of the base large model;

[0009] S2. Fine-tuning the base large model based on a maintenance fault question data set to obtain the maintenance fault adapter;

[0010] S3. Fine-tuning the base large model based on a vehicle use question data set to obtain the vehicle use question adapter;

[0011] S4. Fine-tuning the base large model based on a general domain question data set to obtain the general domain question adapter;

[0012] S5. Fine-tuning the base large model based on an intent recognition task data set to obtain the intent recognition adapter.

[0013] Furthermore,

[0014] The base large model is based on AutoMasterGPT v3.5, but its model architecture includes 40 self-attention mechanism layers and 40 feed-forward neural network layers;

[0015] The number of attention heads in the self-attention mechanism layer is 32, and the number of hidden units in the feed-forward neural network layer is 8192;

[0016] The context length of the base large model in the inference stage is 8192 tokens.

[0017] Furthermore,

[0018] The step S2 includes the following steps:

[0019] S21. Data preparation: Collect and annotate automotive maintenance fault question data and organize it into a maintenance fault question data set;

[0020] S22. Fine-tuning training: Use the maintenance fault question data set to fine-tune and train the base large model to obtain a maintenance fault adapter;

[0021] S23. Evaluation and optimization: Evaluate the performance of the maintenance fault adapter on maintenance fault questions and perform corresponding optimizations, specifically including the following steps:

[0022] S231. Evaluation Preparation: Select the maintenance fault verification set and test set;

[0023] S232. Determine the maintenance fault evaluation criteria and baseline for measuring whether the performance of the maintenance fault adapter meets the expectations;

[0024] S233. Evaluation Index Calculation: Evaluate the performance of the maintenance fault adapter using Accuracy, Precision, Recall, and F1-score;

[0025] S234. Calculate the response time of the base large model for different types of maintenance fault problems;

[0026] S235. Performance Analysis: Analyze the performance of the maintenance fault adapter for different types of maintenance fault problems according to the calculation results of the evaluation metrics in step S233;

[0027] S236. Tuning: Adjust the hyperparameters of the maintenance fault adapter according to the analysis results in step S235; Perform data augmentation, including increasing the diversity and quantity of maintenance fault problem data; Retrain the base large model and perform fine-tuning;

[0028] S237. Iterative Evaluation: Repeat steps S231 to S236 until the performance of the maintenance fault adapter meets the expected standard;

[0029] S238. Finally, evaluate on an independent maintenance fault test set to verify the performance of the maintenance fault adapter in a real scenario,

[0030] Step S3 includes the following steps:

[0031] S31. Data Preparation: Collect and label vehicle-related problem data and organize it into a vehicle problem dataset;

[0032] S32. Fine-tuning Training: Use the vehicle problem dataset to fine-tune the base large model to obtain a vehicle problem adapter;

[0033] S33. Evaluation and Tuning: Evaluate the performance of the vehicle problem adapter for vehicle problems and perform corresponding optimizations, specifically including the following steps:

[0034] S331. Evaluation Preparation: Select the vehicle problem verification set and test set;

[0035] S332. Determine the vehicle problem evaluation criteria and baseline for measuring whether the performance of the vehicle problem adapter meets the expectations;

[0036] S333. Calculation of evaluation metrics: Evaluate the performance of the vehicle-related problem adapter using accuracy, precision, recall, and F1-score;

[0037] S334. Calculate the response time of the vehicle-related problem adapter for different types of vehicle-related problems;

[0038] S335. Performance analysis: Analyze the performance of the vehicle-related problem adapter for different types of vehicle-related problems based on the calculation results of the evaluation metrics in step S333;

[0039] S336. Tuning: Adjust the hyperparameters of the vehicle-related problem adapter according to the analysis results in step S335; perform data augmentation, including increasing the diversity and quantity of vehicle-related problem data; retrain the base large model and perform fine-tuning;

[0040] S337. Iterative evaluation: Repeat steps S331 to S336 until the performance of the vehicle-related problem adapter meets the expected standard;

[0041] S338. Finally, evaluate on an independent vehicle-related problem test set to verify the performance of the vehicle-related problem adapter in a real scenario,

[0042] Step S4 includes the following steps:

[0043] S41. Data preparation: Collect and label general domain problem data and organize it into a general domain problem dataset;

[0044] S42. Fine-tuning training: Use the general domain problem dataset to fine-tune the base large model to obtain a general domain problem adapter;

[0045] S43. Evaluation and tuning: Evaluate the performance of the general domain problem adapter on general domain problems and perform corresponding optimizations, specifically including the following steps:

[0046] S431. Evaluation preparation: Select a general domain problem validation set and a test set;

[0047] S432. Determine the general domain problem evaluation criteria and baseline for measuring whether the performance of the general domain problem adapter meets the expected goal;

[0048] S433. Calculation of evaluation metrics: Evaluate the performance of the general domain problem adapter using accuracy, precision, recall, and F1-score;

[0049] S434. Calculate the response time of the base large model for different types of general domain problems;

[0050] S435. Performance analysis: Analyze the performance of the general domain problem adapter for various general domain problems according to the calculation results of the evaluation metrics in step S433;

[0051] S436. Tuning: Adjust the hyperparameters of the general domain problem adapter according to the analysis results in step S435; Perform data augmentation, including generating more types of general domain problem data to increase the diversity of training data; Retrain the base large model and perform fine-tuning;

[0052] S437. Iterative evaluation: Repeat steps S431 to S436 until the performance of the general domain problem adapter reaches the expected standard;

[0053] S438. Finally, conduct a comprehensive evaluation on an independent general domain problem test set to verify the performance of the general domain problem adapter in a real scenario,

[0054] Step S5 includes the following steps:

[0055] S51. Data preparation: Collect and annotate intent recognition task data and organize it into an intent recognition dataset;

[0056] S52. Fine-tuning training: Use the intent recognition dataset to perform fine-tuning training on the base large model to obtain an intent recognition adapter;

[0057] S53. Evaluation and tuning: Evaluate the performance of the intent recognition adapter in the intent recognition task and perform corresponding optimizations. The specific steps are as follows:

[0058] S531. Evaluation preparation: Select an intent recognition validation set and a test set, and determine the intent recognition evaluation criteria and baseline;

[0059] S532. Evaluation metric calculation: Evaluate the performance of the intent recognition adapter using accuracy, precision, recall, and F1-score, and calculate the response time of the base large model for different types of intent recognition problems;

[0060] S533. Performance analysis: Analyze the performance of the intent recognition adapter according to the calculation results of the evaluation metrics in step S532;

[0061] S534. Tuning: According to the results of the analysis in step S533, adjust the hyperparameters of the intent recognition adapter; perform data augmentation, including increasing the diversity and quantity of intent recognition problem data; retrain the base large model and perform fine-tuning.

[0062] S535. Iterative Evaluation: Repeat steps S531 to S534 until the performance of the intent recognition adapter reaches the expected standard, and verify it on an independent intent recognition test set to ensure reliability.

[0063] Furthermore,

[0064] The maintenance fault adapter, vehicle problem adapter, general domain problem adapter, and intent recognition adapter are based on a common adapter architecture.

[0065] Furthermore,

[0066] The construction of the adapter architecture includes the steps of:

[0067] LoRA Module Design: Design a low-rank adapter module, and the low-rank adapter module adopts an adjusted LoRA model.

[0068] Adapter Integration: Integrate multiple Lora modules into the base large model by means of program reference and call.

[0069] Furthermore,

[0070] The fine-tuning training in steps S2 to S5 includes the following steps:

[0071] SS1. Initialize Parameters: Initialize parameters from the pre-trained base large model.

[0072] SS2. Initialize Low-Rank Matrices: Initialize two low-rank matrices for the low-rank adapter module A and B , where and , r ≪ d , where

[0073] : indicates that the low-rank matrix A is a d row r column matrix,

[0074] : indicates that the low-rank matrix B is a d row r column matrix,

[0075] r≪ d : represents r The value of is much less than d , r and d are non - negative integers,

[0076] Randomly initialize the low - rank matrix using a Gaussian distribution A and B :

[0077] ,

[0078] ,

[0079] where, is the standard deviation, which is set to a preset small value according to the working conditions, i 1 and j 1 are integers, and 1 ≤ i 1 ≤ d and 1 ≤ j 1 ≤ r ,

[0080] SS3. Input data: Input the processed data into the base large - model, so that the base large - model extracts features from the input data;

[0081] SS4. Forward propagation: Calculate the output result after LoRA adjustment , including each calculation layer of the neural network, including the self - attention layer and the feed - forward neural network layer, extracts features by gradually processing the input data to generate the model output, that is, obtain the output result , at the same time, adjust the weight matrix by adding the product of the low - rank matrix A and B to the weight matrix:

[0082] ,

[0083] In the above formula, is the adjusted weight matrix, W is the initial weight matrix, "·" represents matrix dot - product, B’ is B 's transpose matrix,

[0084] The output result satisfies:

[0085] ,

[0086] In the above formula, f is a mapping function, x is the input feature,f Its specific form varies according to the task requirements and is specifically determined by the technical personnel according to the working conditions;

[0087] SS5. Calculate the cross-entropy loss, and calculate the cross-entropy loss function according to the output result and the true label: :

[0088] ,

[0089] In the above formula, represents the cross-entropy loss function, N is the number of samples, C is the number of classes, that is, the number of types of adapters, is the i th j true label of the th i sample for the j th class, i and j are integers, and log(X) is the logarithm of X to the base 10;

[0090] SS6. Calculate the gradient of the loss function with respect to any element W θ of the weight matrix, and update the elements of the low-rank matrix A and B respectively,

[0091] ,

[0092] In the above formula, is the gradient of the loss function with respect to any element W θ of the weight matrix,

[0093] For any element A θ of the low-rank matrix A and any element B θ of the low-rank matrix B , use the optimization algorithm Adam for update,

[0094] SS7. Iterative training: Repeat steps SS4 to SS6 until the performance of the base large model on the validation set reaches the expected standard.

[0095] Furthermore,

[0096] The optimization algorithm Adam is updated through the following steps:

[0097] Parameter setting: Initial learning rate: 1e-3, batch size: 8;

[0098] SS61. Calculate the gradient and , that is, the loss function with respect to any element A of the low-rank matrix A θ and any element B of the low-rank matrix B θ partial derivative:

[0099] ,

[0100] ;

[0101] SS62. Update the first moment estimate,

[0102] Denote A θ and B θ collectively as parameter θ , denote t representing the t-th (i.e., the current) iteration calculation, where t is an integer greater than 1, update the first moment estimate of the gradient of the loss function corresponding to the parameter θ as follows:

[0103] ,

[0104] where m t-1 is the first moment estimate of the previous iteration, is the current gradient of the loss function with respect to θ , β 1 is the first exponential decay rate hyperparameter;

[0105] SS63. Update the second moment estimate,

[0106] Update the second moment estimate of the gradient of the loss function corresponding to the parameter θ as follows:

[0107] ,

[0108] where is the second moment estimate of the previous iteration, is the loss function with respect to θThe square of the current gradient, β 2 is the second exponential decay rate hyperparameter,

[0109] SS64. Calculate m t and for bias correction,

[0110] Since m t and have biases in the initial stage and need bias correction:

[0111] , ,

[0112] is the corrected first moment estimate, is the corrected second moment estimate,

[0113] SS65. Update the parameter θ ,

[0114] Use and to update the parameter θ :

[0115] ,

[0116] In the above formula, α represents the learning rate, Є is a constant.

[0117] The present invention also provides a computer device, which includes a memory, a first processor, and a first computer program stored on the memory and executable on the first processor. When the first computer program is executed by the first processor, it implements the above-mentioned multi-adapter fine-tuning automotive Q&A method.

[0118] The present invention also provides a computer-readable storage medium, which is used to store a second computer program. The second computer program can be executed by at least one second processor, so that the at least one second processor executes the above-mentioned multi-adapter fine-tuning automotive Q&A method.

[0119] The Muti-Adapter fine-tuning method for automotive Q&A provided by the present invention is proposed for the first time. This method can perform separate fine-tuning according to questions in different fields, ensuring the specificity of each field, thereby improving the accuracy of domain adaptation and ensuring consistent performance. This is a significant improvement over the existing single fine-tuning method and can effectively solve the domain deviation problem. The present invention divides automotive domain questions into maintenance faults, driving experiences, and general questions, and performs Lora fine-tuning on the base large model respectively to generate different Adapter adapters. At the same time, an intent recognition adapter is trained to determine which adapter to use to answer user questions, thereby significantly enhancing the adaptability and accuracy of the system in handling domain diversification and complex questions.

[0120] Through the large model and multi-adapter fine-tuning method, the present invention realizes the efficient recognition and processing of new questions, breaking through the limitations of existing automotive intelligent Q&A systems when facing new questions or questions not included in the knowledge base. This will greatly improve the response speed of the system and the user experience.

[0121] The fine-tuned model in the present invention is pluggable and can be updated or replaced at any time, which will significantly reduce the system's maintenance and upgrade costs. This is a significant improvement over the existing full-parameter fine-tuning method. It can not only save a large amount of computing resources but also solve the problem that the construction and maintenance of the knowledge base require a large amount of manpower and resources. Especially when dealing with frequently updated automotive technologies and fault information, the cost can be greatly reduced.

[0122] The present invention proposes a method that can efficiently handle new questions, does not require frequent updates to the knowledge base, and improves the response speed and user experience. This effectively solves the problem of low efficiency of traditional rule-based and knowledge-base-based methods in dealing with multi-turn Q&A and complex context-related questions, and can improve the efficiency of handling complex and diverse questions.

[0123] Other features and advantages of the present invention will be described in the following specification, and, in part, will be obvious from the specification, or will be understood by implementing the present invention. The objectives and other advantages of the present invention can be realized and obtained through the structures pointed out in the specification, claims, and drawings. Brief Description of the Drawings

[0124] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the following drawings are some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0125] Figure 1The flowchart of the Muti-Adapter fine-tuning automotive question-answering method according to an embodiment of the present invention is shown. Detailed implementation manners

[0126] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are some, but not all, of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0127] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the technical field to which this application belongs; the terms used in the specification of this application are only for the purpose of describing specific embodiments and are not intended to limit this application; the terms "including" and "having" and any variations thereof in the specification and claims of this application and the above drawings are intended to cover non-exclusive inclusion. The terms "first", "second", "third", etc. in the specification and claims of this application or the above drawings are used to distinguish different objects and are not used to describe a specific order or primary-secondary relationship. The term "a plurality" as used in this application means two or more (including two).

[0128] Figure 1 The flowchart of the Muti-Adapter fine-tuning automotive question-answering method provided by the present invention. After obtaining a maintenance fault adapter, a vehicle usage question adapter, a general domain question adapter, and an intent recognition adapter by using a base large model in the Muti-Adapter fine-tuning automotive question-answering method, automotive question answering is performed after using the maintenance fault adapter, the vehicle usage question adapter, the general domain question adapter, and the intent recognition adapter. The Muti-Adapter fine-tuning automotive question-answering method includes the following steps.

[0129] S1. Establishment and pre-training of the base large model.

[0130] The base large model (which can be briefly denoted as the model) can be obtained by using AutoMasterGPT v3.5 as the basis and optimizing it. AutoMasterGPT v3.5 is a large natural language processing model based on Transformer. The architecture of the base large model has a higher number of layers and parameters, reaching the order of 14B parameters, so as to enhance the expressiveness and generalization ability of the model in processing complex language tasks.

[0131] Specifically, the base large model is based on AutoMasterGPT v3.5, but its model architecture includes 40 layers of self-attention mechanism layers and 40 layers of feed-forward neural network layers. The addition of these layers significantly improves the model's representation ability and the ability to handle complexity when understanding and generating language. At the same time, the number of model parameters reaches 14B (14 billion), which enables the model to have stronger expressive ability and generalization ability, especially performing more excellently when dealing with complex natural language tasks.

[0132] In addition, the number of attention heads in the self-attention mechanism layer is increased from the original 12 to 32 to better capture long-distance dependencies and detailed information. The number of hidden units in the feed-forward neural network layer is increased from the original 3072 to 8192, which significantly enhances the model's computing ability and generalization ability when dealing with a large amount of knowledge related to the automotive field.

[0133] For the pre-training stage, a corpus containing a large amount of knowledge in the automotive field is used for training, including general and professional encyclopedic knowledge, social media content, automotive repair manuals, etc. Through these improvements, the base large model has stronger understanding ability and higher answer accuracy when facing professional questions in the automotive field.

[0134] In addition, the context length of the model in the inference stage is extended from the original 2048 tokens to 8192 tokens.

[0135] These improvements significantly enhance the depth and complexity of the model, enabling it to better adapt to professional knowledge and different types of questions in the automotive field. The pre-training is carried out using a large amount of general text data (including general and automotive field encyclopedic knowledge information, social media content, books, blogs and other corpora), enabling the base large model to have broad language understanding and generation capabilities.

[0136] After pre-training the base large model, a large amount of general domain data and automotive field data are used to perform supervised fine-tuning (sft) on the base large model. This includes:

[0137] 1. Obtain the pre-trained model parameters through the above pre-training tasks;

[0138] 2. Use supervised data (10 million + automotive Q&A data and general dialogue data) to perform supervised fine-tuning on the pre-trained model.

[0139] 3. During the fine-tuning process, the model's parameters are updated according to the supervised data to optimize the model's performance in automotive answers.

[0140] 4. The objective function used in the fine-tuning stage is usually the loss function of the supervised learning task, such as the cross-entropy loss function.

[0141] S2. Fine-tune the base large model based on the maintenance fault problem dataset to obtain a maintenance fault adapter, which specifically includes the following steps:

[0142] S21. Data preparation: Collect and annotate 100,000 automotive maintenance fault problem data and organize them into a maintenance fault problem dataset.

[0143] S22. Fine-tuning training: Use the maintenance fault problem dataset to fine-tune the base large model to obtain a maintenance fault adapter.

[0144] S23. Evaluation and tuning: Evaluate the performance of the adapter on maintenance fault problems and perform corresponding optimizations. Specifically, it includes the following steps:

[0145] S231. Evaluation preparation: Select a validation set and a test set that contain diverse maintenance fault problems to comprehensively cover different scenarios and ensure the comprehensiveness and representativeness of the evaluation.

[0146] S232. Determine the evaluation criteria and baseline for measuring whether the performance of the adapter meets the expectations.

[0147] S233. Evaluation metric calculation: Evaluate the performance of the adapter using metrics such as accuracy, precision, recall, and F1-score.

[0148] S234. Calculate the response time of the model on different types of problems to ensure that the adapter is not only accurate but also responds quickly.

[0149] S235. Performance analysis: According to the calculation results of the evaluation metrics, analyze the performance of the adapter on different types of problems and find performance bottlenecks. Analyze the types of errors that occur during the evaluation of the adapter through visualization tools (such as confusion matrices) to determine which specific categories of problems the adapter performs poorly in evaluating. Evaluate the performance of the adapter under boundary conditions, such as evaluating unconventional or rare maintenance fault problems, to ensure the robustness of the model.

[0150] S236. Tuning: According to the results of the performance analysis, adjust the hyperparameters of the adapter (such as learning rate, batch size, etc.) to optimize the convergence and stability of the model. Perform data augmentation, including increasing the diversity and quantity of data, to improve the generalization ability of the adapter in different scenarios. Retrain the model, focusing on the problems of the categories where the adapter performs poorly during evaluation in step S235, and perform targeted fine-tuning on the model to improve the performance of the adapter in specific domains.

[0151] S237. Iterative Evaluation: Repeat the evaluation and tuning steps until the performance of the adapter meets the expected standards. Among them, record the evaluation results of each iteration, compare the performance improvements between different iterations, and ensure that the optimization direction is correct and effective.

[0152] S238. Finally, evaluate on an independent test set to verify the performance of the adapter in real scenarios to ensure the reliability and effectiveness of its application.

[0153] S3. Fine-tune the base large model based on the vehicle usage problem dataset to obtain a vehicle usage problem adapter, which specifically includes the following steps:

[0154] S31. Data Preparation: Collect and annotate a large amount of vehicle usage problem data and organize it into a vehicle usage problem dataset.

[0155] S32. Fine-tuning Training: Use the vehicle usage problem dataset to fine-tune the base large model to obtain a vehicle usage problem adapter.

[0156] S33. Evaluation and Tuning: Evaluate the performance of the adapter on vehicle usage problems and perform corresponding optimizations. Specifically, it includes the following steps:

[0157] S331. Evaluation Preparation: Select a validation set and a test set containing diverse vehicle usage problems to ensure coverage of different scenarios and comprehensively evaluate the performance of the adapter.

[0158] S332. Determine the evaluation criteria and baseline for measuring whether the performance of the adapter meets the expectations.

[0159] S333. Evaluation Metric Calculation: Use metrics such as Accuracy, Precision, Recall, and F1-score to evaluate the performance of the adapter.

[0160] S334. Calculate the response time of the adapter for different types of vehicle usage problems to ensure fast response on the basis of accuracy.

[0161] S335. Performance Analysis: According to the evaluation results, analyze the performance of the adapter for different types of vehicle usage problems and identify the parts with insufficient performance. Use visualization tools (such as confusion matrices) to identify the types of errors that occur during the evaluation of the adapter and determine which categories of problems the adapter performs poorly in evaluating. Evaluate the performance of the adapter under boundary conditions, such as when facing unconventional or rare vehicle usage problems, to ensure the robustness of the model.

[0162] S336. Tuning: According to the performance analysis results, adjust the hyperparameters of the adapter (such as learning rate, batch size, etc.) to optimize the convergence and stability of the model. Perform data augmentation to increase the diversity and quantity of data, so as to improve the generalization ability of the adapter in different scenarios. Retrain and fine-tune the model for the problems of the categories with poor performance during the adapter evaluation in step S335 to improve the performance of the adapter in specific domains.

[0163] S337. Iterative Evaluation: Repeat the evaluation and tuning steps until the performance of the adapter reaches the expected standard. Among them, record the evaluation results of each iteration, analyze the performance improvement situation, and ensure that the optimization direction is correct and effective.

[0164] S338. Finally, evaluate on an independent test set to verify the performance of the adapter in real scenarios and ensure the reliability and effectiveness of the application.

[0165] S4. Fine-tune the base large model based on the general domain question dataset to obtain a general domain question adapter, which specifically includes the following steps:

[0166] S41. Data Preparation: Collect and annotate a large amount of general domain question data and organize it into a general domain question dataset.

[0167] S42. Fine-tuning Training: Use the general domain question dataset to fine-tune the base large model to obtain a vehicle problem adapter.

[0168] S43. Evaluation and Tuning: Evaluate the performance of the adapter on general domain questions and perform corresponding optimizations. Specifically, it includes the following steps:

[0169] S431. Evaluation Preparation: Select a validation set and a test set containing a wide range of general domain questions to ensure that the coverage of the evaluation is comprehensive enough to cover different types of general questions.

[0170] S432. Determine the evaluation criteria and baseline to measure whether the performance of the adapter reaches the expected goal.

[0171] S433. Evaluation Metric Calculation: Use metrics such as accuracy, precision, recall, F1-score, etc. to evaluate the performance of the adapter.

[0172] S434. Calculate the response time of the model on different types of general domain questions to ensure that the adapter has efficient response capabilities while maintaining accuracy.

[0173] S435. Performance Analysis: According to the evaluation metrics, analyze the performance of the adapter on various general domain problems and identify the poorly performing parts. Use visualization tools (such as confusion matrices and error distribution plots) to analyze the types of errors that occur during the evaluation of the adapter, find the root causes of the problems, and determine the categories of problems for which the adapter performs poorly during evaluation. Evaluate the performance of the adapter when dealing with boundary conditions, such as when facing rare or difficult-to-understand general problems, to ensure the robustness and adaptability of the adapter.

[0174] S436. Tuning: According to the performance analysis results, adjust the hyperparameters of the adapter (such as learning rate, regularization coefficient, batch size, etc.) to optimize the convergence speed and stability of the model. Perform data augmentation, including generating more types of general domain problems to increase the diversity of the training data, in order to improve the generalization ability of the adapter. Retrain and fine-tune the model for the categories of problems for which the adapter performed poorly during the evaluation in step S435 to improve the performance of the adapter in these areas.

[0175] S437. Iterative Evaluation: Repeat the evaluation and tuning steps until the performance of the adapter meets the expected standards. Among them, record the evaluation results of each iteration, analyze the performance changes between different iterations, and ensure the effectiveness of the tuning and the correctness of the optimization direction.

[0176] S438. Finally, conduct a comprehensive evaluation on the independent test set to verify the performance of the adapter in the real scenario to ensure the stability and application reliability of the adapter.

[0177] S5. Fine-tune the base large model based on the intent recognition task dataset to obtain an intent recognition adapter, which specifically includes the following steps:

[0178] S51. Data Preparation: Collect and annotate a large amount of intent recognition task data and organize it into an intent recognition dataset.

[0179] S52. Fine-tuning Training: Use the intent recognition dataset to fine-tune the base large model to obtain an intent recognition adapter.

[0180] S53. Evaluation and Tuning: Evaluate the performance of the adapter on the intent recognition task and perform corresponding optimizations. The specific steps are as follows:

[0181] S531. Evaluation Preparation: Select the validation set and the test set to ensure coverage of diverse intent recognition task scenarios, and determine the evaluation criteria and baseline.

[0182] S532. Evaluation Metric Calculation: Evaluate the adapter using metrics such as accuracy, precision, recall, and F1-score, and calculate the response time of the model on different types of problems.

[0183] S533. Performance Analysis: Analyze the performance of the adapter based on the evaluation results, identify performance bottlenecks, and use visualization tools (such as confusion matrices) to analyze the types or categories of errors that occur during adapter evaluation to determine the direction for improvement.

[0184] S534. Tuning: Adjust the hyperparameters of the adapter (such as learning rate, batch size, etc.) to optimize the convergence speed and stability of the model. Perform data augmentation to increase the diversity and quantity of data to enhance the generalization ability of the adapter in different scenarios. Retrain and fine-tune the model to optimize for the categories of errors that occurred during adapter evaluation in step S533.

[0185] S535. Iterative Evaluation: Repeat the evaluation and tuning steps until the adapter performance meets the expected standards, and verify on an independent test set to ensure reliability.

[0186] Among them,

[0187] Steps S21, S31, S41, S51 also include: performing data cleaning on the collected data: removing sensitive information, removing invalid characters, etc., and preprocessing it into a fine-tuning data format.

[0188] The annotation in steps S21, S31, S41 is to annotate the answers for the corresponding questions, and the annotation in step S51 is to annotate the user intent for the corresponding intent recognition task.

[0189] The adapters involved in steps S2 to S5 are based on a common adapter architecture, and the construction of the adapter architecture includes the steps:

[0190] SA1. Lora Module Design: Design a low-rank adapter module, which is an adjusted LoRA (Low-Rank Adaptation) model. As a fine-tuning technology, the LoRA model, the present invention adjusts the weights of the original LoRA model by adding a low-rank matrix to obtain the adjusted LoRA model, thereby achieving model adaptability while maintaining computational efficiency.

[0191] SA2. Adapter Integration: Integrate multiple Lora modules into the base large model by means of program reference and call. The integration is achieved through the model's dynamic loading mechanism, enabling the adapter module to be quickly called when needed. The number of multiple LoRA modules usually ranges from 3 to 10, and the specific quantity is adjusted according to the actual needs of the system and the problem complexity in different fields. This can ensure the efficient processing of different types of problems by the adapter and guarantee the overall performance of the model.

[0192] The fine-tuning training in steps S2 to S5 includes the following steps:

[0193] SS1. Initialize parameters: Initialize parameters from a pre-trained base large model.

[0194] SS2. Initialize low-rank matrices: Initialize two low-rank matrices for each low-rank adapter module generated in steps S2 to S5 A and B , where and , r ≪ d , and is determined by the technician according to the working conditions, where

[0195] : represents that the matrix A is a d row r column matrix,

[0196] : represents that the matrix B is a d row r column matrix,

[0197] r ≪ d : represents that r is much smaller than d , meaning the matrix is low-rank and has a small dimension. r and d are non-negative integers.

[0198] The present invention randomly initializes the low-rank matrices A and B using a Gaussian distribution (normal distribution):

[0199] ,

[0200] ,

[0201] where is the standard deviation, usually set by the technician according to the working conditions to a preset small value (such as 0.01 or 0.02) to ensure that the initial weights are small and facilitate subsequent model optimization, i 1 and j 1 are integers, and 1 ≤ i 1 ≤ d and 1 ≤ j 1 ≤ rThe random initialization injects an appropriate perturbation into the matrix to avoid the problem that the adapter makes no contribution due to all elements being zero. The randomness of the random initialization provides diverse optimization paths for the model, which helps with convergence. Among them, the mean of the initial distribution of the random initialization is zero to ensure that the adapter does not cause a deviation from the base model in the initial state. The random initialization does not rely on prior knowledge of a specific domain and is applicable to various tasks, including the automotive domain and intent recognition tasks. At the same time, it also reduces the need for complex preprocessing and improves the adaptability of the adapter.

[0202] SS3. Input data: Input the processed data into the base large model so that the base large model can extract features from the input data.

[0203] SS4. Forward propagation: Calculate the output result after adjustment by LoRA 。Each computational layer of the neural network, including the self-attention layer, feed-forward neural network layer, etc., generates the model output by gradually processing the input data for feature extraction, that is, obtaining the output prediction, which is the output result 。At the same time, by adding the product of the low-rank matrix A and B to the weight matrix of these computational layers to adjust the weight matrix to complete the adapter fine-tuning and further improve the model performance. The adjustment is as follows:

[0204] ,

[0205] In the above formula, is the adjusted parameter matrix, that is, the weight matrix, W is the initial parameter matrix (determined by the technician according to the working conditions), A and B are the low-rank matrices in step SS2. "·" represents the matrix dot product, B’ is B 's transpose matrix. By introducing the low-rank matrices A and B , the weight matrix is lightly adjusted, which can significantly reduce the number of parameters during fine-tuning training while maintaining the model performance.

[0206] The output result is the model output satisfies:

[0207] ,

[0208] In the above formula, f is a mapping function, x is the input feature. fThe specific form of may vary according to the task requirements and may be determined by technical personnel based on the working conditions. For example, it may be a linear transformation or a nonlinear activation function.

[0209] SS5, calculate the cross entropy loss, according to the output results Calculate the cross entropy loss function with the true label :

[0210] ,

[0211] In the above formula, represents the cross entropy loss function (the cross entropy loss is used to measure the difference between the model's predicted distribution and the true distribution, that is, the error. By minimizing this error, the model's predicted value is as close to the true value as possible). N is the sample size, C is the number of categories (i.e. types of adapters), It is i The sample j The true label of the class, It is i The sample j The predicted probability of the class, i and j is an integer, log(X) is the logarithm of X to the base 10.

[0212] SS6. Calculate the loss function by back propagation calculation method Relative to any element of the weight matrix W θ The gradient of and update the low-rank matrix A and B elements.

[0213] ,

[0214] In the above formula, is the loss function Relative to any element of the weight matrix W θ The gradient of , which indicates the update direction of the weight matrix elements.

[0215] For low rank matrices A Any element of A θ and low-rank matrix B Any element of B θ, it is updated using the optimized computing algorithm Adam. Adam (Adaptive Moment Estimation) is an optimization computing method that combines momentum and adaptive learning rate, which can accelerate convergence and avoid getting stuck in local optima. When performing parameter updates, Adam calculates the first moment estimate (i.e., momentum) and the second moment estimate (i.e., exponential moving average of the square of the gradient) of the low-rank matrix parameter gradient, so as to better adjust the learning rate. For any element of the low-rank matrix A and any element of the low-rank matrix A θ B and any element of the low-rank matrix B θ , Adam updates through the following steps:

[0216] Parameter settings:

[0217] Initial learning rate: 1e-3;

[0218] Batch size: 8 (the number of samples randomly selected in each training epoch),

[0219] SS61. Calculate the gradients and , that is, the partial derivatives of the loss function with respect to any element of the low-rank matrix A and any element of the low-rank matrix A θ and any element of the low-rank matrix B and any element of the low-rank matrix B θ . These two gradients represent the element update directions of the low-rank matrices A , B :

[0220] ,

[0221] .

[0222] SS62. Update the first moment estimate (momentum).

[0223] Denote A θ and B θ collectively as the parameter θ . Denote t as the t-th (i.e., the current) iteration calculation, where t is an integer greater than 1. Update the first moment estimate (i.e., momentum) of the gradient of the corresponding loss function according to the following formula: θ

[0224] ,

[0225] where,​​m t-1 is the first moment estimate of the previous iteration, is the loss function of the current gradient, β 1 is the first exponential decay rate hyperparameter, which is adjusted and set by the technical personnel according to the working conditions.

[0226] SS63. Update the second moment estimate.

[0227] Update according to the following formula θ the second moment estimate of the gradient of the corresponding loss function:

[0228] ,

[0229] where, is the second moment estimate of the previous iteration, is the loss function the square of the current gradient of, β 2 is the second exponential decay rate hyperparameter, which is adjusted and set by the technical personnel according to the working conditions.

[0230] SS64. Calculate m t and of the bias correction.

[0231] Since m t and have biases in the initial stage, bias correction is required:

[0232] , ,

[0233] is the corrected first moment estimate, is the corrected second moment estimate.

[0234] SS65. Update the parameter θ .

[0235] Use the corrected and to update the parameter θ :

[0236] .

[0237] Specifically for the low-rank matrix element A θ and B θ the update is:

[0238] ,

[0239] ,

[0240] wherein, α represents the learning rate, which is used to control the step size of parameter θ update, and it is a key hyperparameter that affects the convergence speed and stability. Є is a small constant (set by technicians according to the working conditions) used to prevent the denominator from being zero. A θt+1 and A θt are respectively A θ 's next iteration value and current iteration value. B θt+1 and B θt are respectively B θ 's next iteration value and current iteration value. and are respectively the first moment estimate and second moment estimate corresponding to the low-rank matrix element A θ in the current iteration calculation. and are respectively the first moment estimate and second moment estimate corresponding to the low-rank matrix element B θ in the current iteration calculation. The momentum m t is used to smooth the estimated value of the loss function gradient, reduce oscillations, and accelerate convergence. The second moment estimate is used to adjust the learning rate of each parameter to adapt to the gradient changes of different parameters. Through the above steps, the Adam optimization calculation method can effectively adjust A θ and B θ values to gradually reduce the loss function to achieve the optimization goal.

[0241] SS7. Iterative training: Repeat steps SS4 to SS6 until the performance of the model on the validation set reaches the expected standard, and finally obtain a fine-tuned trained model.

[0242] In the above-mentioned steps, indicators such as accuracy, precision, recall, and F1-score are used to comprehensively evaluate each adapter. The specific forms of accuracy, precision, recall, and F1-score are:

[0243] Accuracy = (TP + TN) / (TP + TN + FP + FN),

[0244] Precision = TP / (TP + FP),

[0245] Recall = TP / (TP + FN),

[0246] F1 - score = (2 × Precision × Recall) / (Precision + Recall).

[0247] Among them, TP (True Positive) represents the number of samples that are truly positive (i.e., accurate) and are correctly predicted as positive, TN (True Negative) represents the number of samples that are truly negative (inaccurate, or rather, wrong) and are correctly predicted as negative, FP (False Positive) represents the number of samples that are truly negative but are wrongly predicted as positive, and FN (False Negative) represents the number of samples that are truly positive but are wrongly predicted as negative.

[0248] If the above - mentioned indicators all reach the expected values, a practical base large - model required for the Muti - Adapter fine - tuning automotive Q&A method provided by the present invention can be obtained. Further, when the Muti - Adapter fine - tuning automotive Q&A method performs Q&A on automotive questions based on an operable system installed with the practical base large - model, it includes the following steps:

[0249] SA. User question input: The user inputs a question through the interface of the operable system, and the question can cover various types of questions such as automotive repair and fault questions, vehicle - using questions, general - field questions, etc.

[0250] SB. The operable system uses the intent recognition adapter of the practical base large - model to perform intent recognition on the question input by the user, and determines which field the question belongs to among automotive repair and fault questions, vehicle - using questions, and general - field questions. For example, if the user asks a question about an automotive fault, the intent recognition adapter will recognize that the question belongs to the repair and fault category of questions.

[0251] SC. The operable system processes it using the corresponding adapter (repair and fault adapter, vehicle - using adapter, general - field question adapter) according to the result of intent recognition. For example, if the question is recognized as a repair and fault category of questions, the repair and fault category of question adapter will be selected for processing.

[0252] The SD and the operating system generate an answer to the question according to the processing result of the corresponding adapter, and then output the answer to the user. For example, if the intent recognition selects the adapter for repair and fault questions, the system will generate the corresponding answer through this adapter and output the answer to the user.

[0253] The present invention also provides a computer device, which includes a memory, a first processor, and a first computer program stored on the memory and executable on the first processor. When the first computer program is executed by the first processor, it implements the above-mentioned multi-adapter fine-tuning method for automotive Q&A.

[0254] The present invention also provides a computer-readable storage medium for storing a second computer program, which can be executed by at least one second processor, so that the at least one second processor executes the above-mentioned multi-adapter fine-tuning method for automotive Q&A.

[0255] The present invention can:

[0256] 1. Solve the domain adaptation problems in different fields: The fine-tuning techniques of existing systems usually do not distinguish between domains and adopt a single fine-tuning method, resulting in possible domain biases when dealing with problems in different fields. The present invention introduces a multi-adapter fine-tuning method, which performs separate fine-tuning according to problems in different fields (such as automotive maintenance, vehicle use, general domain problems, etc.), ensures the specificity of each field, thereby improving the accuracy of domain adaptation and ensuring consistent performance.

[0257] 2. Solve the limitations of existing automotive intelligent Q&A systems when facing new or questions not included in the knowledge base, and achieve efficient recognition and processing of new questions. At the same time, in view of the possible domain biases between different fields in the fine-tuning technique, a fine-tuning method that can accurately adapt to each field is proposed.

[0258] 3. Solve the problem that the construction and maintenance of the knowledge base require a large amount of manpower and resources, especially when dealing with frequently updated automotive technologies and fault information, and reduce costs. At the same time, solve the problem of the high cost of existing full-parameter fine-tuning methods, and propose a method that not only requires less computing resources, but also the fine-tuned model is pluggable and can be updated or replaced at any time, which will reduce the maintenance and upgrade costs of the system.

[0259] 4. Solve the problem of low efficiency of traditional rule-based and knowledge-base-based methods when dealing with multi-turn Q&A and complex context association problems, and propose a method that can efficiently handle new questions, does not require frequent updates of the knowledge base, improves the response speed and user experience. At the same time, propose a method that can efficiently adapt to problems in different fields and improve the efficiency of dealing with complex and diverse problems.

[0260] Reference to "embodiments" in this specification means that the particular features, structures, or characteristics described in connection with the embodiments can be included in at least one embodiment of the present application. The phrase appears in various places in the specification and does not necessarily refer to the same embodiment each time, nor is it an independent or alternative embodiment mutually exclusive of other embodiments. Those skilled in the art will explicitly and implicitly understand that the embodiments described herein can be combined with other embodiments.

[0261] Although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements for some of the technical features; and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A multi-adapter fine-tuning automobile question-and-answer method, characterized in that: The base model is used to obtain and apply the maintenance fault adapter, vehicle problem adapter, general domain problem adapter and intention recognition adapter to answer questions about automobile problems, including the following steps: S1, establishment and pre-training of the base large model; S2. Fine-tuning the base large model based on the maintenance fault problem data set to obtain the maintenance fault adapter; S3, fine-tuning the base large model based on the car use problem data set to obtain the car use problem adapter; S4, fine-tuning the base large model based on the general domain problem dataset to obtain the general domain problem adapter; S5, fine-tuning the base large model based on the intention recognition task data set to obtain the intention recognition adapter; The fine-tuning training in steps S2 to S5 includes the following steps: SS1. Initialization parameters: Initialize parameters from the pre-trained base large model; SS2, low-rank matrix initialization: Initialize two low-rank matrices A and B for the low-rank adapter module, where and , r≪d, where : indicates that the low-rank matrix A is a matrix with d rows and r columns, : Indicates that the low-rank matrix B is a matrix with d rows and r columns, r≪d: indicates that the value of r is much smaller than d, and r and d are integers not less than 0. The low-rank matrices A and B are randomly initialized using Gaussian distribution: ， ， in, is the standard deviation, which is set to a preset small value according to the working conditions, i1 and j1 are integers, and 1≤i1≤d and 1≤j1≤r, SS3, input data: input the processed data into the base large model, so that the base large model performs feature extraction on the input data; SS4, forward propagation: calculate the output result after LoRA adjustment The various computing layers of the neural network including the self-attention layer and the feedforward neural network layer generate the model output by performing feature extraction on the input data step by step, that is, obtaining the output result At the same time, the weight matrix is ​​adjusted by adding the product of the low-rank matrices A and B to the weight matrix of each calculation layer: ， In the above formula, is the adjusted weight matrix, W is the initial weight matrix, "·" represents the matrix dot product, B' is the transposed matrix of B, The output result satisfy: ， In the above formula, f is a mapping function, x is the input feature, and the specific form of f varies according to the task requirements and is determined by the technicians according to the working conditions; SS5, calculate the cross entropy loss, according to the output results Calculate the cross entropy loss function with the true label : ， In the above formula, represents the cross entropy loss function, N is the number of samples, C is the number of categories, i.e., the number of adapter types, is the true label of the jth class of the ith sample, is the predicted probability of the jth class of the ith sample, i and j are integers, and log(X) is the logarithm of X to the base 10; SS6. Calculate the loss function through the back propagation algorithm With respect to any element W of the weight matrix θ The gradient of , and update the elements of the low-rank matrices A and B respectively, ， In the above formula, is the loss function With respect to any element W of the weight matrix θ The gradient of For any element A of the low-rank matrix A θ and any element B of the low-rank matrix B θ , updated using the optimization calculation method Adam, SS7. Iterative training: Repeat steps SS4 to SS6 until the performance of the large base model on the validation set reaches the expected standard.

2. The automobile question-and-answer method for multi-adapter fine-tuning according to claim 1, characterized in that: The base model is based on AutoMasterGPTv3.5, but its model architecture includes 40 layers of self-attention mechanism and 40 layers of feedforward neural network; The number of attention heads in the self-attention mechanism layer is 32, and the number of hidden units in the feedforward neural network layer is 8192; The context length of the base large model during the inference phase is 8192 tokens.

3. The automobile question-and-answer method for multi-adapter fine-tuning according to claim 1 or 2, characterized in that: The step S2 comprises the following steps: S21. Data preparation: Collect and annotate automobile maintenance failure problem data and organize them into maintenance failure problem data sets; S22, fine-tuning training: using the maintenance fault problem data set to perform fine-tuning training on the base large model to obtain a maintenance fault adapter; S23, evaluation and optimization: evaluating the performance of the repair fault adapter on the repair fault problem and performing corresponding optimization, specifically including the following steps: S231, Evaluation preparation: Select the maintenance fault verification set and test set; S232, determining a repair fault assessment standard and a baseline for measuring whether the performance of the repair fault adapter meets expectations; S233, evaluation index calculation: using accuracy, precision, recall, and F1-score to evaluate the performance of the repair fault adapter; S234, calculating the response time of the large base model to different types of maintenance fault problems; S235, performance analysis: analyzing the performance of the repair fault adapter on different types of repair fault problems according to the calculation results of the evaluation index in step S233; S236, tuning: according to the analysis result of step S235, adjusting the hyper parameters of the maintenance fault adapter; performing data enhancement, including increasing the diversity and quantity of maintenance fault problem data; retraining the base large model, and performing fine tuning; S237, iterative evaluation: repeating the steps S231 to S236 until the performance of the repaired fault adapter reaches the expected standard; S238. Finally, an evaluation is performed on an independent maintenance fault test set to verify the performance of the maintenance fault adapter in a real scenario. The step S3 comprises the following steps: S31. Data preparation: Collect and annotate car-related problem data and organize them into a car-related problem dataset; S32, fine-tuning training: fine-tuning the base large model using the vehicle use problem data set to obtain a vehicle use problem adapter; S33, evaluation and optimization: evaluating the performance of the vehicle use problem adapter on the vehicle use problem and performing corresponding optimization, specifically including the following steps: S331. Evaluation preparation: Select the validation set and test set of the car usage problem; S332, determining a vehicle use problem evaluation standard and a baseline for measuring whether the performance of the vehicle use problem adapter meets expectations; S333, evaluation index calculation: use accuracy, precision, recall, and F1-score to evaluate the performance of the vehicle-related problem adapter; S334, calculating the response time of the vehicle use problem adapter to different types of vehicle use problems; S335, performance analysis: analyzing the performance of the vehicle use problem adapter on different types of vehicle use problems according to the calculation results of the evaluation index of step S333; S336, tuning: according to the analysis result of step S335, adjusting the hyper parameters of the vehicle usage problem adapter; performing data enhancement, including increasing the diversity and quantity of vehicle usage problem data; retraining the base large model, and performing fine tuning; S337, iterative evaluation: repeating the steps S331 to S336 until the performance of the vehicle-related problem adapter reaches the expected standard; S338. Finally, an evaluation is performed on an independent car-use problem test set to verify the performance of the car-use problem adapter in a real scenario. The step S4 comprises the following steps: S41. Data preparation: Collect and annotate common domain problem data and organize them into common domain problem datasets; S42, fine-tuning training: fine-tuning the base large model using the general domain problem dataset to obtain a general domain problem adapter; S43, evaluation and tuning: evaluating the performance of the general domain problem adapter on the general domain problem and performing corresponding optimization, specifically including the following steps: S431. Evaluation preparation: Select the validation set and test set of common domain problems; S432, determining a general domain problem evaluation standard and a baseline for measuring whether the performance of the general domain problem adapter meets the expected goal; S433, evaluation index calculation: use accuracy, precision, recall, and F1-score to evaluate the performance of the general domain problem adapter; S434, calculating the response time of the base large model to different types of general field problems; S435, performance analysis: analyzing the performance of the general domain problem adapter on various general domain problems according to the calculation results of the evaluation index of step S433; S436, tuning: according to the analysis result of step S435, adjusting the hyper parameters of the general domain problem adapter; performing data enhancement, including generating more types of general domain problem data to increase the diversity of training data; retraining the base large model and performing fine tuning; S437, iterative evaluation: repeating the steps S431 to S436 until the performance of the general domain problem adapter reaches the expected standard; S438. Finally, a comprehensive evaluation is performed on an independent general domain problem test set to verify the performance of the general domain problem adapter in real scenarios. The step S5 comprises the following steps: S51, data preparation: collect and annotate intent recognition task data and organize them into intent recognition datasets; S52, fine-tuning training: fine-tuning the base large model using the intent recognition data set to obtain an intent recognition adapter; S53, evaluation and tuning: Evaluate the performance of the intent recognition adapter on the intent recognition task and perform corresponding optimization. The specific steps are as follows: S531, Evaluation preparation: Select the intent recognition verification set and test set, and determine the intent recognition evaluation criteria and baseline; S532, evaluation index calculation: use accuracy, precision, recall, and F1-score to evaluate the performance of the intent recognition adapter, and calculate the response time of the base large model on different types of intent recognition problems; S533, performance analysis: analyzing the performance of the intent recognition adapter according to the calculation result of the evaluation index of step S532; S534, tuning: according to the analysis results of step S533, adjusting the hyperparameters of the intent recognition adapter; performing data enhancement, including increasing the diversity and quantity of intent recognition problem data; retraining the base large model, and performing fine-tuning; S535, iterative evaluation: repeat steps S531 to S534 until the performance of the intent recognition adapter reaches the expected standard, and verify it on an independent intent recognition test set to ensure reliability.

4. The automobile question-and-answer method for multi-adapter fine-tuning according to claim 3, characterized in that: The maintenance fault adapter, vehicle usage problem adapter, general domain problem adapter and intent recognition adapter are based on a common adapter architecture.

5. The automobile question-and-answer method for multi-adapter fine-tuning according to claim 4, characterized in that: The construction of the adapter architecture includes the steps of: LoRA module design: A low-rank adapter module is designed, which adopts the adjusted LoRA model; Adapter integration: Integrate multiple Lora modules into the base model through program reference and calling.

6. The automobile question-and-answer method for multi-adapter fine-tuning according to claim 1, characterized in that: The optimization calculation method Adam is updated through the following steps: Parameter settings: initial learning rate: 0.001, batch size: 8; SS61, Calculate Gradient and , which is the loss function For any element A of the low-rank matrix A θ and any element B of the low-rank matrix B θ The partial derivative of : ， ； SS62, Update first-order moment estimate, A θ and B θ It is uniformly denoted as parameter θ, t represents the tth, i.e., the current, iteration calculation, t is an integer greater than 1, and the first-order moment estimate of the gradient of the loss function corresponding to the parameter θ is updated according to the following formula: ， Among them, m t-1 is the first-order moment estimate of the previous iteration, is the loss function The current gradient of θ, β1 is the first exponential decay rate hyperparameter; SS63, Updated Second-Order Moment Estimates, Update the second-order moment estimate of the gradient of the loss function corresponding to the parameter θ according to the following formula: ， in, is the second-order moment estimate of the previous iteration, is the loss function The square of the current gradient of θ, β2 is the second exponential decay rate hyperparameter, SS64, calculate m t and The deviation correction of Due to m t and There are deviations in the initial stage, which need to be corrected: ， ， is the modified first-order moment estimate, is the modified second-order moment estimate, SS65, update the parameter θ, use and Update the parameter θ: ， In the above formula, α represents the learning rate and Є is a constant.

7. Computer device, characterized in that The invention comprises a memory, a first processor and a first computer program stored in the memory and executable on the first processor, wherein the first computer program, when executed by the first processor, implements the automobile question-and-answer method for multi-adapter fine-tuning according to any one of claims 1 to 6.

8. A computer-readable storage medium, characterized in that The computer-readable storage medium is used to store a second computer program, and the second computer program can be executed by at least one second processor to enable the at least one second processor to execute the multi-adapter fine-tuning automobile question-and-answer method according to any one of claims 1-5.

Images