Vehicle-mounted offline question-answering system and method, electronic equipment and vehicle

Abstract

The invention discloses a vehicle-mounted offline question-answering system and method, electronic equipment and a vehicle, and belongs to the technical field of vehicles. The system at least comprises a user terminal, a database and a vehicle-mounted server. The user terminal is at least used for acquiring a query instruction input by a user so as to convert the query instruction into first format data and then upload the first format data to the vehicle-mounted server; and receiving response data, converting the response data into second format data, and feeding back the second format data to the user in a preset feedback form. The database is at least for storing vehicle utility data. And the vehicle-mounted server is at least used for receiving the first format data, so that the pre-trained large language model generates response data based on the first format data and the called vehicle practical data and issues the response data to the user terminal. The technical problems that an existing electronic manual is low in information retrieval efficiency and poor in user experience, has security risks, is difficult to maintain and update, is limited in system training and optimization, needs networking use and the like can be solved, and the user experience and query efficiency can be improved.

Description

Technical Field

[0001] Embodiments of the present invention relate to the field of vehicle technology, and in particular to an in-vehicle offline question-answering system, method, electronic device, and vehicle. Background Art

[0002] In the automotive industry, traditional user manuals are mostly provided in paper form, which usually contain information on vehicle operation, daily maintenance, troubleshooting, etc. In recent years, with the continuous development of digital technology, electronic manuals that can provide users with a more convenient query method have gradually replaced paper manuals. Existing electronic manuals mainly have the following forms:

[0003] (1) PDF format electronic manual: Provided through the automobile manufacturer's website or mobile application, users can download or view the user manual in PDF format online.

[0004] (2) Electronic manuals in mobile applications: Some automobile manufacturers have developed dedicated mobile applications where users can search for relevant information. These applications usually have a search function and an interface that combines text and images.

[0005] (3) Online help system: Some manufacturers provide an online help system where users can enter questions and search through a web interface, and the system will return relevant help documents.

[0006] However, although the existing electronic manuals have improved the user experience to some extent, there are still many problems and shortcomings:

[0007] (1) Low information retrieval efficiency: When users search for specific questions, the results returned by the system often contain a large amount of irrelevant information. The search results are not relevant and require users to filter them by themselves, which affects the user experience. In addition, most existing systems are based on keyword search and lack natural language understanding (i.e., it is difficult to understand users' natural language queries), resulting in inaccurate answers to complex questions.

[0008] (2) Poor user experience: The interface interactions of some mobile applications and online systems are complex and cumbersome to operate, which leads to a poor user experience, especially for elderly users or users who are not technically proficient. In addition, some systems rely on cloud servers to process queries. Network latency and server load can cause the system to respond slowly and fail to meet users’ instant query needs.

[0009] (3) Security risks exist: Many systems rely on cloud processing, which means that the user's query data needs to be uploaded to the server, which poses security risks such as data privacy leakage. As users have higher and higher requirements for the privacy of their personal information and query content, cloud processing can hardly fully meet user needs.

[0010] (4) Difficulty in maintenance and updating: The existing electronic manual content needs to be updated manually, which is a cumbersome and error-prone process and cannot reflect the latest vehicle information and troubleshooting solutions in a timely manner.

[0011] (5) Limited system training and optimization: The online help system and mobile applications have limited intelligence, making it difficult to perform adaptive learning and optimization, and unable to continuously improve system performance based on user feedback.

[0012] (6) Requires an Internet connection: The mobile application and online help system must be connected to the Internet in real time to use.

[0013] To sum up, existing electronic manuals have many problems and pain points in terms of information retrieval efficiency, user experience, data privacy and security, system maintenance and updates, etc. A new technical solution is urgently needed to solve at least one of the above problems in order to improve user experience and query efficiency. Summary of the invention

[0014] The embodiments of the present invention provide an in-vehicle offline question-and-answer system, method, electronic device and vehicle to at least solve the technical problems of low information retrieval efficiency, poor user experience, security risks, maintenance and updating difficulties, limited system training and optimization, and the need for Internet access in existing electronic manuals, thereby improving user experience and query efficiency.

[0015] In a first aspect, an embodiment of the present invention provides a vehicle-mounted offline question-answering system, comprising at least a user terminal, a database, and a vehicle-mounted server;

[0016] The user terminal is connected to the vehicle-mounted server and is at least used to obtain a query instruction input by a user, convert the query instruction into a first format data and then upload it to the vehicle-mounted server; and receive response data, convert the response data into a second format data and then feed it back to the user in a preset feedback form;

[0017] The database is connected to the vehicle-mounted server and is at least used to store vehicle practical data for the vehicle-mounted server to call;

[0018] The vehicle-mounted server is configured with a pre-trained large language model, and is at least used to receive the first format data, so that the pre-trained large language model generates the response data based on the first format data and the called vehicle practical data, and then sends the response data to the user terminal;

[0019] The pre-training includes at least two of pruning training, distillation training and low-rank adaptation training.

[0020] Optionally, the pre-training includes at least the pruning training and the low-rank adaptation training;

[0021] The pre-training process of the large language model includes at least the following steps:

[0022] SA1. Calculate important index parameters of each weight in the large language model;

[0023] SA2. Resetting to zero at least one weight of the important indicator parameter in the large language model that is lower than a preset threshold;

[0024] SA3. Re-performing performance training on the large language model using the first performance training data;

[0025] SA4. Perform incremental training on the large language model through low-rank matrix decomposition based on the first incremental training data.

[0026] Optionally, the pre-training includes at least the pruning training and the low-rank adaptation training;

[0027] The pre-training process of the large language model includes at least the following steps:

[0028] SB1. Evaluate the contribution of each neuron in the large language model;

[0029] SB2. deleting at least one neuron in the large language model whose contribution is lower than a preset contribution;

[0030] SB3. Re-performing performance training on the large language model using the second performance training data;

[0031] SB4. Perform incremental training on the large language model through low-rank matrix decomposition based on the second incremental training data.

[0032] Optionally, the pre-training includes at least the pruning training and the low-rank adaptation training;

[0033] The pre-training process of the large language model includes at least the following steps:

[0034] SC1. determining a model influencing parameter of each structural unit in the large language model;

[0035] SC2. removing at least one structural unit in the large language model whose model influencing parameter is higher than a preset parameter value;

[0036] SC3. Re-performing performance training on the large language model using third performance training data;

[0037] SC4. Perform incremental training on the large language model through low-rank matrix decomposition based on the third incremental training data.

[0038] Optionally, the pre-training includes at least the pruning training and the low-rank adaptation training;

[0039] The pre-training process of the large language model includes at least the following steps:

[0040] SA1. Calculate important index parameters of each weight in the large language model;

[0041] SA2. Resetting to zero at least one weight of the important indicator parameter in the large language model that is lower than a preset threshold;

[0042] SA3. Re-performing performance training on the large language model using the first performance training data;

[0043] SB1. Evaluate the contribution of each neuron in the large language model;

[0044] SB2. deleting at least one neuron in the large language model whose contribution is lower than a preset contribution;

[0045] SB3. Re-performing performance training on the large language model using the second performance training data;

[0046] SB5. Performing incremental training on the large language model through low-rank matrix decomposition based on the fourth incremental training data;

[0047] or,

[0048] The pre-training process of the large language model includes at least the following steps:

[0049] SA1. Calculate important index parameters of each weight in the large language model;

[0050] SA2. Resetting to zero at least one weight of the important indicator parameter in the large language model that is lower than a preset threshold;

[0051] SA3. Re-performing performance training on the large language model using the first performance training data;

[0052] SC1. determining a model influencing parameter of each structural unit in the large language model;

[0053] SC2. removing at least one structural unit in the large language model whose model influencing parameter is higher than a preset parameter value;

[0054] SC3. Re-performing performance training on the large language model using third performance training data;

[0055] SC5. Performing incremental training on the large language model through low-rank matrix decomposition based on the fifth incremental training data;

[0056] or,

[0057] The pre-training process of the large language model includes at least the following steps:

[0058] SB1. Evaluate the contribution of each neuron in the large language model;

[0059] SB2. deleting at least one neuron in the large language model whose contribution is lower than a preset contribution;

[0060] SB3. Re-performing performance training on the large language model using the second performance training data;

[0061] SC1. determining a model influencing parameter of each structural unit in the large language model;

[0062] SC2. removing at least one structural unit in the large language model whose model influencing parameter is higher than a preset parameter value;

[0063] SC3. Re-performing performance training on the large language model using third performance training data;

[0064] SC6. Perform incremental training on the large language model through low-rank matrix decomposition based on the sixth incremental training data.

[0065] Optionally, the pre-training includes at least the pruning training and the low-rank adaptation training;

[0066] The pre-training process of the large language model includes at least the following steps:

[0067] SA1. Calculate important index parameters of each weight in the large language model;

[0068] SA2. Resetting to zero at least one weight of the important indicator parameter in the large language model that is lower than a preset threshold;

[0069] SA3. Re-performing performance training on the large language model using the first performance training data;

[0070] SB1. Evaluate the contribution of each neuron in the large language model;

[0071] SB2. deleting at least one neuron in the large language model whose contribution is lower than a preset contribution;

[0072] SB3. Re-performing performance training on the large language model using the second performance training data;

[0073] SC1. determining a model influencing parameter of each structural unit in the large language model;

[0074] SC2. removing at least one structural unit in the large language model whose model influencing parameter is higher than a preset parameter value;

[0075] SC3. Re-performing performance training on the large language model using third performance training data;

[0076] SC7. Perform incremental training on the large language model through low-rank matrix decomposition based on the seventh incremental training data.

[0077] Optionally, the pre-training includes at least the distillation training and the low-rank adaptation training;

[0078] The pre-training process of the large language model includes at least the following steps:

[0079] SD1. Use the teacher model training data to train the large language model to obtain a teacher large language model;

[0080] SD2. Input the preset in-vehicle question and answer data into the teacher's large language model to obtain the data probability distribution of the preset in-vehicle question and answer data;

[0081] SD3. Establish a student language model, and perform performance training on the student language model based on the data probability distribution;

[0082] SD4. Perform incremental training on the student large language model through low-rank matrix decomposition based on the eighth incremental training data.

[0083] In a second aspect, an embodiment of the present invention further provides an in-vehicle offline question-answering method, which is performed by using the in-vehicle offline question-answering system described in the first aspect, and the method at least includes:

[0084] Acquiring a query instruction input by a user through a user terminal, converting the query instruction into data in a first format and uploading the data to the vehicle-mounted server;

[0085] Receiving the first format data through the vehicle-mounted server and calling the vehicle practical data stored in the database, so as to generate response data based on the first format data and the called vehicle practical data through the pre-trained large language model, and then sending the response data to the user terminal;

[0086] Receiving the response data through the user terminal, converting the response data into second format data and then feeding back to the user in a preset feedback form;

[0087] The pre-training includes at least two of pruning training, distillation training and low-rank adaptation training.

[0088] In a third aspect, an embodiment of the present invention further provides an electronic device, comprising a processor and a memory, wherein the memory stores computer-readable instructions, and when the computer-readable instructions are executed by the processor, the steps in the in-vehicle offline question-and-answer method described in the second aspect are executed.

[0089] In a fourth aspect, an embodiment of the present invention further provides a vehicle, which is integrated with at least the in-vehicle offline question-answering system described in any one of the first aspects.

[0090] The embodiment of the present invention provides an in-vehicle offline question-answering system, method, electronic device and vehicle. First, the user terminal obtains the query instruction input by the user, converts the query instruction into a first format data and uploads it to the in-vehicle server; then, the in-vehicle server receives the first format data and calls the vehicle practical data stored in the database, so as to generate response data based on the first format data and the called vehicle practical data through a large language model pre-trained by at least two types of pruning training, distillation training and low-rank adaptation training, and then sends the response data to the user terminal; finally, the user terminal receives the response data, converts the response data into a second format data and feeds it back to the user in a preset feedback form.

[0091] It can be understood that the embodiments of the present invention can make the in-vehicle offline question and answer system adapt to application scenarios of complex natural language queries by using a pre-trained large language model (LLM), which is more conducive to the system understanding user intentions and then performing accurate question and answer responses, solving the technical problem of low efficiency of information retrieval in existing electronic manuals, and is conducive to improving the query efficiency of the in-vehicle offline question and answer system.

[0092] In addition, since the LLM in the embodiment of the present invention has undergone a pre-training process including at least two methods of pruning training, distillation training and low-rank adaptive training, the model structure of the pre-trained LLM can be streamlined and optimized. On the one hand, it can reduce the computing and storage requirements of the model when it is deployed locally, so that the pre-trained LLM can be used locally offline under the existing computing power of the vehicle computer; on the other hand, it can also continuously learn and iterate according to the user's query data, realize the automatic update of the database in the vehicle offline question and answer system, ensure the accuracy and timeliness of the answer, and facilitate maintenance and update. At the same time, because the pre-trained LLM can be used locally offline, the embodiment of the present invention effectively reduces the answer response time of the vehicle offline question and answer system, which is conducive to meeting the user's instant question and answer needs and improving the user experience. In addition, since all data processing in the vehicle offline question and answer system can be completed locally, data upload to the cloud is avoided, which can greatly reduce the risk of data leakage and help protect the user's privacy and data security. BRIEF DESCRIPTION OF THE DRAWINGS

[0093] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0094] Figure 1 is a structural diagram of an in-vehicle offline question-answering system provided by Embodiment 1 of the present invention;

[0095] Figure 2 This is a pre-training flow chart of an LLM provided in the second embodiment of the present invention;

[0096] Figure 3 This is a pre-training flow chart of an LLM provided in Embodiment 3 of the present invention;

[0097] Figure 4 This is a pre-training flow chart of an LLM provided by Embodiment 4 of the present invention;

[0098] Figure 5 This is a pre-training flow chart of an LLM provided in Embodiment 5 of the present invention;

[0099] Figure 6 is another LLM pre-training flow chart provided in Embodiment 5 of the present invention;

[0100] Figure 7 This is another LLM pre-training flow chart provided in Embodiment 5 of the present invention;

[0101] Figure 8 This is a pre-training flow chart of an LLM provided in Embodiment 6 of the present invention;

[0102] Fig. 9 This is a pre-training flow chart of an LLM provided in Embodiment 7 of the present invention;

[0103] Fig.10 This is a pre-training flow chart of an LLM provided in Embodiment 8 of the present invention;

[0104] Fig.11 This is a pre-training flow chart of an LLM provided by Embodiment 9 of the present invention;

[0105] Fig.12 is a flow chart of an in-vehicle offline question-answering method provided by Embodiment 10 of the present invention;

[0106] Fig.13 It is a structural schematic diagram of an electronic device provided in the eleventh embodiment of the present invention. DETAILED DESCRIPTION

[0107] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application will be further described in detail below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present application.

[0108] The terms used in the embodiments of the present application are only for the purpose of describing specific embodiments, and are not intended to limit the present application. The singular forms "a", "said" and "the" used in the embodiments of the present application and the appended claims are also intended to include plural forms, unless the context clearly indicates other meanings, and "multiple" generally includes at least two.

[0109] It should be understood that the term "and / or" used in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship.

[0110] It should be understood that although the terms first, second, third, etc. may be used to describe in the embodiments of the present application, these descriptions should not be limited to these terms. These terms are only used to distinguish the descriptions. For example, without departing from the scope of the embodiments of the present application, the first may also be referred to as the second, and similarly, the second may also be referred to as the first.

[0111] As used herein, the words "if" and "if" may be interpreted as "at the time of" or "when" or "in response to determining" or "in response to detecting", depending on the context. Similarly, the phrases "if it is determined" or "if (stated condition or event) is detected" may be interpreted as "when it is determined" or "in response to determining" or "when detecting (stated condition or event)" or "in response to detecting (stated condition or event)", depending on the context.

[0112] It should also be noted that the term "includes", "comprising" or any other variation thereof is intended to cover non-exclusive inclusion, so that a commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprising a ..." do not exclude the existence of other identical elements in the commodity or device including the elements.

[0113] It should be particularly noted that any symbols and / or numbers in the specification that are not marked in the accompanying drawings are not drawing marks.

[0114] Embodiment 1

[0115] Figure 1 1 is a schematic diagram of the structure of an in-vehicle offline question-and-answer system provided in the first embodiment of the present invention. This embodiment is applicable to the query and answer scenarios between the user in the car cabin and the vehicle computer regarding vehicle driving operation, vehicle daily maintenance, vehicle troubleshooting, route navigation, future weather changes, etc. The system can be implemented in software and / or hardware. Figure 1 As shown, the vehicle-mounted offline question-answering system provided in this embodiment at least includes a user terminal 110, a database 120 and a vehicle-mounted server 130 (Vehicle-mounted Server, VMS).

[0116] The user terminal 110 is connected to the vehicle-mounted server 130, and is used at least to obtain the query instruction input by the user, convert the query instruction into a first format data and upload it to the vehicle-mounted server 130; and receive response data, convert the response data into a second format data and then feed it back to the user in a preset feedback form.

[0117] The database 120 is connected to the vehicle-mounted server 130 and is at least used to store vehicle practical data for the vehicle-mounted server 130 to call.

[0118] The vehicle-mounted server 130 is configured with a pre-trained large language model (for example, a large language model based on the Transformer architecture, such as Ali Qwen-2, etc.), which is at least used to receive the first format data, so that the pre-trained large language model generates response data based on the first format data and the called vehicle practical data, and then sends the response data to the user terminal 110.

[0119] Among them, pre-training includes at least two of pruning training, distillation training and low-rank adaptation (LORA) training.

[0120] It can be known that the user terminal 110 may refer to the voice interaction system, the central control display screen, the infotainment screen, the rear seat left entertainment screen, the rear seat right entertainment screen, etc. in the vehicle intelligent cockpit system architecture; the query command input by the user may be a voice command (for example, the user may input it through the aforementioned voice interaction system), a text command (for example, the user may manually input it through the aforementioned infotainment screen), etc. In other words, the user may input text or voice queries through the vehicle system or mobile device, such as manually inputting "how to change the engine oil", or asking "what should I do if the engine fault light is on", or directly taking a photo of the instrument fault code and uploading it to the user terminal 110.

[0121] In addition, the first format data may be data in a standard natural language processing format. For example, after the user inputs a voice query through the voice interaction system, the user terminal 110 may convert the voice query into a text format through speech-to-text (STT) technology, and then further process the text format into data in a standard natural language processing format through redundant word deletion, word segmentation, grammatical analysis and other processing steps, and then upload the data in a standard natural language processing format to the VMS. It is understandable that the database 120 may be integrated into the aforementioned vehicle intelligent cockpit system architecture, and the vehicle practical data may include, but is not limited to, weather forecasts, the vehicle's car manual, the vehicle's user manual, the vehicle's car repair manual, etc.

[0122] Specifically, after the VMS receives the first format data sent by the user terminal 110, the pre-trained LLM begins to analyze the user query semantics involved in the first format data, understand the user's intention, and then generates a response content (i.e., response data, which may include, for example, the operation steps of the vehicle in a specific scenario, the tools and precautions required for vehicle maintenance, etc.) in combination with the vehicle practical data such as the car manual in the database 120. After that, the VMS sends the response data to the user terminal 110 (at this time, the response data may be a data compression package). Further, the user terminal 110 converts the response data into a second format data (the second format data may be text format data), and feeds back to the user through a preset feedback form (the preset feedback form may be a voice broadcast form, a text display form, a voice broadcast plus a text display form, etc.), and the user may choose to view detailed steps, listen to voice guidance, etc.

[0123] Exemplarily, the workflow of the in-vehicle offline question-answering system may be as follows:

[0124] First, the user terminal 110 obtains the query instruction input by the user, converts the query instruction into the first format data and uploads it to the VMS. Then, the VMS receives the first format data and calls the vehicle practical data stored in the database 120, so as to generate response data based on the first format data and the called vehicle practical data through at least two pre-trained large language models of pruning training, distillation training and low-rank adaptation training, and then sends the response data to the user terminal 110. Finally, the user terminal 110 receives the response data, converts the response data into the second format data and feeds it back to the user in a preset feedback form.

[0125] To sum up, this embodiment uses the pre-trained LLM to make the in-vehicle offline question and answer system adaptable to application scenarios of complex natural language queries, which is more conducive to the system understanding the user's intention and then performing accurate question and answer responses, solving the technical problem of low efficiency of existing electronic manual information retrieval, and is conducive to improving the query efficiency of the in-vehicle offline question and answer system.

[0126] At the same time, since the LLM in this embodiment has undergone a pre-training process including at least two methods of pruning training, distillation training and LORA training, the model structure of the pre-trained LLM is streamlined and optimized. On the one hand, it can reduce the computing and storage requirements of the model when it is deployed locally, so that the pre-trained LLM can be used locally offline under the existing computing power of the vehicle computer; on the other hand, it can also perform continuous learning and iteration based on the user's query data, realize automatic updating of the database in the vehicle-mounted offline question-and-answer system, ensure the accuracy and timeliness of the answers, and facilitate maintenance and updating.

[0127] In addition, because the pre-trained LLM can be used locally offline, this embodiment effectively reduces the response time of the in-vehicle offline question and answer system, which is conducive to meeting the user's instant question and answer needs and improving the user experience. Moreover, since all data processing in the in-vehicle offline question and answer system can be completed locally, data uploading to the cloud is avoided, thereby greatly reducing the risk of data leakage and helping to protect user privacy and data security.

[0128] It should be noted that, in a specific implementation, taking the scenario where a user inquires about the oil replacement steps as an example, the specific implementation of this embodiment may be as follows:

[0129] 1. User query:

[0130] The user enters a text query into the vehicle system or mobile device: "How do I change the oil?" or enters the same question via voice.

[0131] 2. System processing:

[0132] The user terminal receives the input and converts the query into a standard natural language processing format. The pre-trained large-scale language model on the local server receives the query and processes it.

[0133] 3. Model answer generation:

[0134] The LLM model optimized by pruning, distillation and / or LORA technology analyzes user questions and generates detailed answers based on relevant information in the database. The answers include specific replacement steps, required tools, and precautions.

[0135] 4. Result feedback:

[0136] The system will feed back the generated answer to the user through the user terminal, providing two feedback methods: text and voice. The user can choose to view the detailed steps or listen to the voice guidance to perform the oil change operation.

[0137] In another specific implementation, taking the scenario where a user eliminates the cause of a vehicle failure as an example, the specific implementation of this embodiment may be as follows:

[0138] 1. User query:

[0139] Users can enter text queries through the vehicle system or mobile device: "What should I do if the engine fault light is on?" or enter the same question through voice. Or they can directly take a photo and send it to the vehicle computer, using the picture to assist in asking questions.

[0140] 2. System processing:

[0141] The user terminal receives the input and converts the query into a standard natural language processing format. The pre-trained large-scale language model on the local server receives the query and processes it.

[0142] 3. Model answer generation:

[0143] The LLM model optimized by LoRa technology analyzes user questions and generates detailed answers based on the troubleshooting information in the database. The answers include possible causes of the faulty light, preliminary inspection steps, and whether it is necessary to contact professional maintenance personnel.

[0144] 4. Result feedback:

[0145] The system will feed back the generated answers to the user through the user terminal, providing two feedback methods: text and voice. The user can choose to view detailed steps or listen to voice guidance to perform troubleshooting operations.

[0146] Embodiment 2

[0147] Figure 2 This is a pre-training flow chart of an LLM provided in the second embodiment of the present invention. This embodiment is refined based on the first embodiment. Figure 2 As shown, optionally, the pre-training includes at least pruning training (ie, steps SA1 to SA3) and low-rank adaptation training (ie, step SA4).

[0148] The pre-training process of a large language model includes at least the following steps:

[0149] SA1. Calculate the important indicator parameters for each weight in the large language model.

[0150] The important indicator parameter of the weight may refer to the absolute value of the weight, for example. It can be understood that step SA1 is essentially to calculate the importance of each weight in the LLM.

[0151] SA2. Reset at least one weight of an important indicator parameter in the large language model that is lower than a preset threshold to zero.

[0152] The preset threshold value may be adaptively configured according to the pruning training effect to be achieved by the LLM, which is not limited in this embodiment.

[0153] It can be understood that when the absolute value of a weight in LLM is lower than the preset threshold, it means that the importance of the weight is low. This low-importance weight can be set to zero to achieve weight pruning (that is, by deleting those connections with small absolute weight values ​​or unimportant connections to simplify the network structure of LLM, thereby at least reducing the computational cost and memory usage of LLM).

[0154] SA3. Use the first performance training data to re-train the performance of the large language model.

[0155] The first performance training data refers to new data that is different from the training data involved in the initial establishment process of the LLM. It can be understood that the large language model in step SA3 refers to the LLM with the weight pruning strategy added (after processing in steps SA1 and SA2), and step SA3 is essentially to retrain the LLM with the weight pruning strategy added based on new data that has not been used to restore the performance of the pruned model.

[0156] SA4. Perform incremental training on the large language model through low-rank matrix decomposition based on the first incremental training data.

[0157] The first incremental training data may be instruction manuals, user manuals, maintenance manuals, etc. of various models of automobiles produced by different manufacturers.

[0158] It is understandable that the large language model in step SA4 refers to the LLM with restored model performance after adding the weight pruning strategy (after processing in step SA3). The essence of step SA4 is to use LORA to fine-tune the pruned model with restored model performance so that it has good adaptability in the actual offline question-and-answer process with users. At the same time, LORA can use the first incremental training data to make small-scale adaptive adjustments to the relevant parameters and weights of the LLM after weight pruning through low-rank matrix decomposition, so that it can still run efficiently without significantly increasing computing resources.

[0159] In summary, after reducing the complexity of LLM through weight pruning, this embodiment provides additional optimization space in model performance based on LORA incremental training, which is not a simple combination of pruning method and LORA algorithm. The multi-level optimization scheme provided in this embodiment can ensure that the model can still handle complex tasks with less computing and storage resources, and has good scalability. Due to the flexibility of LORA incremental training, the vehicle-mounted offline question-and-answer system can gradually improve the performance of the model as the data is updated, adapt to new usage scenarios and requirements, and ensure that the system is continuously optimized during the life cycle. Through the sequential optimization scheme of weight pruning and LORA incremental training, the computing and storage requirements of the model can be significantly reduced without losing model performance, while enhancing the adaptability, flexibility and scalability of the system. This optimization strategy is very suitable for scenarios that require efficient, lightweight and sustainable operation, such as Internet of Vehicles, smart devices and edge computing.

[0160] Embodiment 3

[0161] Figure 3 This is a pre-training flow chart of an LLM provided in the third embodiment of the present invention. This embodiment is refined based on the first embodiment. Figure 3 As shown, optionally, the pre-training includes at least pruning training (ie, steps SB1 to SB3) and low-rank adaptation training (ie, step SB4).

[0162] The pre-training process of a large language model includes at least the following steps:

[0163] SB1. Evaluate the contribution of each neuron in a large language model.

[0164] The contribution of a neuron may refer to the activation value of the neuron or the degree of influence of the neuron on the final output of the LLM. The larger the activation value, the higher the contribution, and the greater the influence of the neuron on the final output of the LLM.

[0165] SB2. Delete at least one neuron in the large language model whose contribution is lower than a preset contribution.

[0166] The preset contribution degree may be adaptively adjusted according to the pruning training effect to be achieved by the LLM, and this embodiment does not limit this.

[0167] It can be understood that when the contribution of a neuron in LLM is lower than the preset contribution, it means that the neuron has little effect on the final output of LLM, and the neuron with low contribution can be deleted to achieve neuron pruning (that is, by deleting those neurons or channels with smaller contributions to simplify the network structure of LLM, thereby at least reducing the computational cost and memory usage of LLM).

[0168] SB3. Use the second performance training data to re-perform performance training on the large language model.

[0169] The second performance training data refers to new data that is different from the training data involved in the initial establishment process of the LLM. It can be understood that the large language model in step SB3 refers to the LLM with the neuron pruning strategy added (after processing in steps SB1 and SB2), and step SB3 is essentially to retrain the LLM with the neuron pruning strategy added based on new data that has not been used to restore the performance of the pruned model.

[0170] SB4. Perform incremental training on the large language model through low-rank matrix decomposition based on the second incremental training data.

[0171] The second incremental training data may be instruction manuals, user manuals, maintenance manuals, etc. of various models of automobiles produced by different manufacturers.

[0172] It is understandable that the large language model in step SB4 refers to the LLM with restored model performance after adding the neuron pruning strategy (after processing in step SB3). The essence of step SB4 is to use LORA to fine-tune the pruned model with restored model performance so that it has good adaptability in the actual offline question-and-answer process with users. At the same time, LORA can use the second incremental training data to make small-scale adaptive adjustments to the relevant parameters and weights of the LLM pruned by neurons through low-rank matrix decomposition, so that it can still run efficiently without significantly increasing computing resources.

[0173] It can be seen that after reducing the complexity of LLM through neuron pruning, this embodiment provides additional optimization space in model performance based on LORA incremental training, which is not a simple combination of pruning method and LORA algorithm. The multi-level optimization scheme provided in this embodiment can ensure that the model can still handle complex tasks with less computing and storage resources, and has good scalability. Due to the flexibility of LORA incremental training, the vehicle-mounted offline question-and-answer system can gradually improve the performance of the model as the data is updated, adapt to new usage scenarios and requirements, and ensure that the system is continuously optimized during the life cycle. Through the sequential optimization scheme of neuron pruning and LORA incremental training, the computing and storage requirements of the model can be significantly reduced without losing model performance, while enhancing the adaptability, flexibility and scalability of the system. This optimization strategy is very suitable for scenarios that require efficient, lightweight and sustainable operation, such as Internet of Vehicles, smart devices and edge computing.

[0174] Embodiment 4

[0175] Figure 44 is a pre-training flow chart of an LLM provided in the fourth embodiment of the present invention. This embodiment is refined based on the first embodiment. Figure 4 As shown, optionally, the pre-training includes at least pruning training (ie, steps SC1 to SC3) and low-rank adaptation training (ie, step SC4).

[0176] The pre-training process of a large language model includes at least the following steps:

[0177] SC1. Determine the model impact parameters of each structural unit in the large language model.

[0178] The structural unit may refer to a convolution kernel, a channel or a layer. Generally speaking, the structural unit with more parameters has a greater impact on the model performance of LLM, and the corresponding model impact parameter is also greater.

[0179] SC2. Remove at least one structural unit in the large language model whose model influence parameter is higher than a preset parameter value.

[0180] Among them, the preset parameter value can be adaptively set according to the pruning training effect to be achieved by LLM, which is not limited in this embodiment.

[0181] It can be understood that when the model influencing parameter of a certain structural unit in LLM is higher than the preset parameter value, it means that the parameter complexity of the structural unit and its influence on the model performance of LLM are relatively high. This structural unit with high parameter complexity can be removed to achieve structured pruning, thereby reducing the computational cost and storage requirements of LLM to a greater extent.

[0182] SC3. Use the third performance training data to re-train the large language model.

[0183] The third performance training data refers to new data different from the training data involved in the initial establishment process of the LLM. It can be understood that the large language model in step SC3 refers to the LLM with the structured pruning strategy added (after processing in steps SC1 and SC2), and step SC3 is essentially to retrain the LLM with the structured pruning strategy added based on new data that has not been used to restore the performance of the pruned model.

[0184] SC4. Incrementally train the large language model through low-rank matrix decomposition based on the third incremental training data.

[0185] The third incremental training data may be instruction manuals, user manuals, maintenance manuals, etc. of various models of automobiles produced by different manufacturers.

[0186] It can be understood that the large language model in step SC4 refers to the LLM with restored model performance after adding the structured pruning strategy (after processing in step SC3). The essence of step SC4 is to use LORA to fine-tune the pruned model with restored model performance so that it has good adaptability in the actual offline question-and-answer process with users. At the same time, LORA can use the third incremental training data to make small-scale adaptive adjustments to the relevant parameters and weights of the structured pruned LLM through low-rank matrix decomposition, so that it can still run efficiently without significantly increasing computing resources.

[0187] In summary, after reducing the complexity of LLM through structured pruning, this embodiment provides additional optimization space in model performance based on LORA incremental training, which is not a simple combination of pruning method and LORA algorithm. The multi-level optimization scheme provided in this embodiment can ensure that the model can still handle complex tasks with less computing and storage resources, and has good scalability. Due to the flexibility of LORA incremental training, the vehicle-mounted offline question-and-answer system can gradually improve the performance of the model as the data is updated, adapt to new usage scenarios and requirements, and ensure that the system is continuously optimized during the life cycle. Through the sequential optimization scheme of structured pruning and LORA incremental training, the computing and storage requirements of the model can be significantly reduced without losing model performance, while enhancing the adaptability, flexibility and scalability of the system. This optimization strategy is very suitable for scenarios that require efficient, lightweight and sustainable operation, such as Internet of Vehicles, smart devices and edge computing.

[0188] Embodiment 5

[0189] Figure 5 This is a pre-training flow chart of an LLM provided in the fifth embodiment of the present invention. This embodiment is refined based on the first embodiment. Figure 5 As shown, optionally, the pre-training includes at least pruning training (ie, steps SA1 to SB3) and low-rank adaptation training (ie, step SB5).

[0190] The pre-training process of a large language model includes at least the following steps:

[0191] SA1. Calculate the important indicator parameters for each weight in the large language model.

[0192] SA2. Reset at least one weight of an important indicator parameter in the large language model that is lower than a preset threshold to zero.

[0193] SA3. Use the first performance training data to re-train the performance of the large language model.

[0194] SB1. Evaluate the contribution of each neuron in a large language model.

[0195] The large language model in step SB1 refers to the LLM to which the weight pruning strategy has been added (after being processed by steps SA1 to SA3) and to which the model performance has been restored.

[0196] SB2. Delete at least one neuron in the large language model whose contribution is lower than a preset contribution.

[0197] SB3. Use the second performance training data to re-perform performance training on the large language model.

[0198] The large language model in step SB3 refers to the LLM to which the weight pruning strategy has been added (after processing in steps SA1 to SB2) and the model performance after weight pruning has been restored, and the neuron pruning strategy is added again. It can be understood that the second performance training data and the first performance training data need to use completely different data to optimize the performance recovery of the model after neuron pruning.

[0199] SB5. Perform incremental training on the large language model through low-rank matrix decomposition based on the fourth incremental training data.

[0200] The fourth incremental training data may be instruction manuals, user manuals, maintenance manuals, etc. of various models of automobiles produced by different manufacturers.

[0201] It can be understood that the large language model in step SB5 refers to the LLM to which the weight pruning strategy has been added (after processing in steps SA1 to SB3) and the model performance after weight pruning has been restored, and the neuron pruning strategy has been added again and the model performance after neuron pruning has been restored. The essence of step SB5 is to use LORA to fine-tune the pruned model with restored model performance, so that it has good adaptability in the actual offline question-and-answer process with users. At the same time, LORA can use the fourth incremental training data to make small-scale adaptive adjustments to the relevant parameters and weights of the LLM after weight pruning and neuron pruning through low-rank matrix decomposition, so that it can still run efficiently without significantly increasing computing resources.

[0202] In summary, after reducing the complexity of LLM through weight pruning and neuron pruning, this embodiment provides additional optimization space in model performance based on LORA incremental training, which is not a simple combination of pruning method and LORA algorithm. The multi-level optimization scheme provided in this embodiment can ensure that the model can still handle complex tasks with less computing and storage resources, and has good scalability. Due to the flexibility of LORA incremental training, the vehicle-mounted offline question-and-answer system can gradually improve the performance of the model as the data is updated, adapt to new usage scenarios and requirements, and ensure that the system is continuously optimized during the life cycle. Through the sequential optimization scheme of weight pruning, neuron pruning and LORA incremental training, the computing and storage requirements of the model can be significantly reduced without losing model performance, while enhancing the adaptability, flexibility and scalability of the system. This optimization strategy is very suitable for scenarios that require efficient, lightweight and sustainable operation, such as Internet of Vehicles, smart devices and edge computing.

[0203] It should be noted that, in a specific implementation, Figure 6 is another LLM pre-training flow chart provided in Embodiment 5 of the present invention, Figure 7 This is another LLM pre-training flow chart provided in Embodiment 5 of the present invention. Figure 6 , steps SA1 to SA3 and steps SB1 to SB3 can be executed synchronously; see Figure 7 , steps SB1 to SB3 can be executed before steps SA1 to SA3; they will not be repeated here.

[0204] Embodiment 6

[0205] Figure 8 This is a pre-training flow chart of an LLM provided in the sixth embodiment of the present invention. This embodiment is refined based on the first embodiment. Figure 8 As shown, optionally, the pre-training includes at least pruning training (ie, steps SA1 to SC3) and low-rank adaptation training (ie, step SC5).

[0206] The pre-training process of a large language model includes at least the following steps:

[0207] SA1. Calculate the important indicator parameters for each weight in the large language model.

[0208] SA2. Reset at least one weight of an important indicator parameter in the large language model that is lower than a preset threshold to zero.

[0209] SA3. Use the first performance training data to re-train the performance of the large language model.

[0210] SC1. Determine the model impact parameters of each structural unit in the large language model.

[0211] The large language model in step SC1 refers to the LLM to which the weight pruning strategy is added (after being processed by steps SA1 to SA3) and to which the model performance has been restored.

[0212] SC2. Remove at least one structural unit in the large language model whose model influence parameter is higher than a preset parameter value.

[0213] SC3. Use the third performance training data to re-train the large language model.

[0214] The large language model in step SC3 refers to the LLM to which the weight pruning strategy has been added (after processing in steps SA1 to SC2) and the model performance after weight pruning has been restored, and the structural pruning strategy is added again. It can be understood that the third performance training data and the first performance training data need to use completely different data to achieve the optimization of the performance recovery of the model after structural pruning.

[0215] SC5. Incrementally train the large language model through low-rank matrix decomposition based on the fifth incremental training data.

[0216] The fifth incremental training data may be instruction manuals, user manuals, maintenance manuals, etc. of various models of automobiles produced by different manufacturers.

[0217] It can be understood that the large language model in step SC5 refers to the LLM to which the weight pruning strategy has been added (after processing in steps SA1 to SC3) and the model performance after weight pruning has been restored, and the structured pruning strategy has been added again and the model performance after structured pruning has been restored. The essence of step SC5 is to use LORA to fine-tune the pruned model with restored model performance, so that it has good adaptability in the actual offline question-and-answer process with users. At the same time, LORA can use the fifth incremental training data to make small-scale adaptive adjustments to the relevant parameters and weights of the LLM after weight pruning and structured pruning through low-rank matrix decomposition, so that it can still run efficiently without significantly increasing computing resources.

[0218] In summary, after reducing the complexity of LLM through weight pruning and structured pruning, this embodiment provides additional optimization space in model performance based on LORA incremental training, which is not a simple combination of pruning method and LORA algorithm. The multi-level optimization scheme provided in this embodiment can ensure that the model can still handle complex tasks with less computing and storage resources, and has good scalability. Due to the flexibility of LORA incremental training, the vehicle-mounted offline question-and-answer system can gradually improve the performance of the model as the data is updated, adapt to new usage scenarios and requirements, and ensure that the system is continuously optimized during the life cycle. Through the sequential optimization scheme of weight pruning, structured pruning and LORA incremental training, the computing and storage requirements of the model can be significantly reduced without losing model performance, while enhancing the adaptability, flexibility and scalability of the system. This optimization strategy is very suitable for scenarios that require efficient, lightweight and sustainable operation, such as Internet of Vehicles, smart devices and edge computing.

[0219] It should be noted that, in a specific implementation, Figure 8 Steps SA1 to SA3 and steps SC1 to SC3 may be executed synchronously; or, steps SC1 to SC3 may be executed before steps SA1 to SA3; details will not be given here.

[0220] Embodiment 7

[0221] Fig. 9 7 is a pre-training flow chart of an LLM provided in the seventh embodiment of the present invention. This embodiment is refined based on the first embodiment. Fig. 9 As shown, optionally, the pre-training includes at least pruning training (ie, steps SB1 to SC3) and low-rank adaptation training (ie, step SC6).

[0222] The pre-training process of a large language model includes at least the following steps:

[0223] SB1. Evaluate the contribution of each neuron in a large language model.

[0224] SB2. Delete at least one neuron in the large language model whose contribution is lower than a preset contribution.

[0225] SB3. Use the second performance training data to re-perform performance training on the large language model.

[0226] SC1. Determine the model impact parameters of each structural unit in the large language model.

[0227] The large language model in step SC1 refers to the LLM to which the neuron pruning strategy has been added (after being processed by steps SB1 to SB3) and to which the model performance has been restored.

[0228] SC2. Remove at least one structural unit in the large language model whose model influence parameter is higher than a preset parameter value.

[0229] SC3. Use the third performance training data to re-train the large language model.

[0230] The large language model in step SC3 refers to the LLM to which the neuron pruning strategy has been added (after processing in steps SB1 to SC2) and the model performance after neuron pruning has been restored, and the structural pruning strategy is added again. It can be understood that the third performance training data and the second performance training data need to use completely different data to achieve the optimization of the performance recovery of the model after structural pruning.

[0231] SC6. Incrementally train the large language model through low-rank matrix decomposition based on the sixth incremental training data.

[0232] The sixth incremental training data may be instruction manuals, user manuals, maintenance manuals, etc. of various models of automobiles produced by different manufacturers.

[0233] It can be understood that the large language model in step SC6 refers to the LLM to which the neuron pruning strategy has been added (after processing in steps SB1 to SC3) and the model performance after neuron pruning has been restored, and the structural pruning strategy has been added again and the model performance after structural pruning has been restored. The essence of step SC6 is to use LORA to fine-tune the pruned model with restored model performance, so that it has good adaptability in the actual offline question-and-answer process with users. At the same time, LORA can use the sixth incremental training data to make small-scale adaptive adjustments to the relevant parameters and weights of the LLM after neuron pruning and structural pruning through low-rank matrix decomposition, so that it can still run efficiently without significantly increasing computing resources.

[0234] In summary, after reducing the complexity of LLM through neuron pruning and structured pruning, this embodiment provides additional optimization space in model performance based on LORA incremental training, which is not a simple combination of pruning method and LORA algorithm. The multi-level optimization scheme provided in this embodiment can ensure that the model can still handle complex tasks with less computing and storage resources, and has good scalability. Due to the flexibility of LORA incremental training, the vehicle-mounted offline question-and-answer system can gradually improve the performance of the model as the data is updated, adapt to new usage scenarios and requirements, and ensure that the system is continuously optimized during its life cycle. Through the sequential optimization scheme of neuron pruning, structured pruning and LORA incremental training, the computing and storage requirements of the model can be significantly reduced without losing model performance, while enhancing the adaptability, flexibility and scalability of the system. This optimization strategy is very suitable for scenarios that require efficient, lightweight and sustainable operation, such as Internet of Vehicles, smart devices and edge computing.

[0235] It should be noted that, in a specific implementation, Fig. 9 Steps SB1 to SB3 and steps SC1 to SC3 may be executed synchronously; or, steps SC1 to SC3 may be executed before steps SB1 to SB3; details will not be given here.

[0236] Embodiment 8

[0237] Fig.10 8 is a pre-training flow chart of an LLM provided in the eighth embodiment of the present invention. This embodiment is refined based on the first embodiment. Fig.10 As shown, optionally, the pre-training includes at least pruning training (ie, steps SA1 to SC3) and low-rank adaptation training (ie, step SC7).

[0238] The pre-training process of a large language model includes at least the following steps:

[0239] SA1. Calculate the important indicator parameters for each weight in the large language model.

[0240] SA2. Reset at least one weight of an important indicator parameter in the large language model that is lower than a preset threshold to zero.

[0241] SA3. Use the first performance training data to re-train the performance of the large language model.

[0242] SB1. Evaluate the contribution of each neuron in a large language model.

[0243] The large language model in step SB1 refers to the LLM to which the weight pruning strategy has been added (after being processed by steps SA1 to SA3) and to which the model performance has been restored.

[0244] SB2. Delete at least one neuron in the large language model whose contribution is lower than a preset contribution.

[0245] SB3. Use the second performance training data to re-perform performance training on the large language model.

[0246] The large language model in step SB3 refers to the LLM to which the weight pruning strategy has been added (after processing in steps SA1 to SB2) and the model performance after weight pruning has been restored, and the neuron pruning strategy is added again. It can be understood that the second performance training data and the first performance training data need to use completely different data to optimize the performance recovery of the model after neuron pruning.

[0247] SC1. Determine the model impact parameters of each structural unit in the large language model.

[0248] Among them, the large language model in step SC1 refers to the LLM to which the weight pruning strategy has been added and the model performance has been restored after weight pruning (after being processed by steps SA1 to SB3), and the LLM to which the neuron pruning strategy has been added again and the model performance has been restored after neuron pruning.

[0249] SC2. Remove at least one structural unit in the large language model whose model influence parameter is higher than a preset parameter value.

[0250] SC3. Use the third performance training data to re-train the large language model.

[0251] Among them, the large language model in step SC3 refers to the LLM to which the weight pruning strategy has been added (after processing in steps SA1 to SC2) and the model performance after weight pruning has been restored, the neuron pruning strategy has been added again and the model performance after neuron pruning has been restored, and the structural pruning strategy has been added again. It can be understood that the third performance training data, the second performance training data and the first performance training data all need to use completely different data to achieve the optimization of the performance recovery of the model after structural pruning.

[0252] SC7. Incrementally train the large language model through low-rank matrix decomposition based on the seventh incremental training data.

[0253] The seventh incremental training data may be instruction manuals, user manuals, maintenance manuals, etc. of various models of cars produced by different manufacturers.

[0254] It can be understood that the large language model in step SC7 refers to the LLM that has added the weight pruning strategy and restored the model performance after weight pruning (after processing in steps SA1 to SC3), added the neuron pruning strategy again and restored the model performance after neuron pruning, and added the structured pruning strategy again and restored the model performance after structured pruning. The essence of step SC7 is to use LORA to fine-tune the pruned model with restored model performance, so that it has good adaptability in the actual offline question-and-answer process with users. At the same time, LORA can use the seventh incremental training data to make small-scale adaptive adjustments to the relevant parameters and weights of the LLM after weight pruning, neuron pruning, and structured pruning through low-rank matrix decomposition, so that it can still run efficiently without significantly increasing computing resources.

[0255] In summary, after reducing the complexity of LLM through weight pruning, neuron pruning and structured pruning, this embodiment provides additional optimization space in model performance based on LORA incremental training, which is not a simple combination of pruning method and LORA algorithm. The multi-level optimization scheme provided in this embodiment can ensure that the model can still handle complex tasks with less computing and storage resources, and has good scalability. Due to the flexibility of LORA incremental training, the vehicle-mounted offline question-and-answer system can gradually improve the performance of the model as the data is updated, adapt to new usage scenarios and requirements, and ensure that the system is continuously optimized during the life cycle. Through the sequential optimization scheme of weight pruning, neuron pruning, structured pruning and LORA incremental training, the computing and storage requirements of the model can be significantly reduced without losing model performance, while enhancing the adaptability, flexibility and scalability of the system. This optimization strategy is very suitable for scenarios that require efficient, lightweight and sustainable operation, such as Internet of Vehicles, smart devices and edge computing.

[0256] It should be noted that, in a specific implementation, Fig.10Steps SA1 to SA3, steps SB1 to SB3, and steps SC1 to SC3 in the process may be executed synchronously; or, they may be executed in the order of step SB1, step SB2, step SB3, step SA1, step SA2, step SA3, step SC1, step SC2, step SC3, and step SC7; or, they may be executed in the order of step SC1, step SC2, step SC3, step SB1, step SB2, step SB3, step SA1, step SA2, step SA3, and step SC7; or, they may be executed in the order of step SC1, step SC2, step SC3, step SA1, step SA2, step SA3, step SB1, step SB2, step SB3 3. Step SC7 is executed sequentially; or, steps SA1-SA3 and steps SB1-SC3 can be executed synchronously; or, steps SA1-SA3 and steps SC1-SB3 (i.e., the execution order of step SC1, step SC2, step SC3, step SB1, step SB2, step SB3) can be executed synchronously; or, steps SB1-SB3 and steps SA1-SC3 can be executed synchronously; or, steps SB1-SB3 and steps SC1-SA3 can be executed synchronously; or, steps SC1-SC3 and steps SA1-SB3 can be executed synchronously; or, steps SC1-SC3 and steps SA1-SB3 can be executed synchronously; or, steps SC1-SC3 and steps SB1-SA3 can be executed synchronously; no further details will be given.

[0257] Embodiment 9

[0258] Fig.11 1 is a pre-training flow chart of an LLM provided in the ninth embodiment of the present invention. This embodiment is refined based on the above embodiments. Fig.11 As shown, optionally, the pre-training includes at least distillation training (ie, steps SD1 to SD3) and low-rank adaptation training (ie, step SD4).

[0259] The pre-training process of a large language model includes at least the following steps:

[0260] SD1. Use the teacher model training data to train the large language model to obtain the teacher large language model.

[0261] Among them, the teacher model training data may, for example, have the data scale and type used to train ChatGPT 4o; the large language model in step SD1 may refer to the commercially available Alibaba Qwen-2 model, etc.

[0262] SD2. Input the preset in-vehicle question and answer data into the teacher's large language model to obtain the data probability distribution of the preset in-vehicle question and answer data.

[0263] The preset vehicle question and answer data may be a vehicle electronic manual or other materials that the pre-trained LLM intends to use in practice. Adaptively, the data probability distribution of the preset vehicle question and answer data may refer to the probability distribution of each chapter category in the vehicle electronic manual that the pre-trained LLM intends to use in practice.

[0264] Specifically, the essence of step SD2 is to generate prediction results (i.e. soft labels) by processing preset in-vehicle question-and-answer data through the commercially available Alibaba Qwen-2 model. The soft labels not only contain classification probabilities, but also similarity information between categories. This information is more comprehensive and richer than hard labels (i.e. direct classification results), and can lay a solid foundation for the subsequent training of student large language models.

[0265] SD3. Establish a student language model and perform performance training on the student language model based on the data probability distribution.

[0266] The specific situation of the student language model in step SD3 is related to the combination of steps SD1 to SD4 with the solutions of the aforementioned embodiments.

[0267] Exemplarily, if steps SD1 to SD4 are to describe the pre-training process of the large language model based on the solution described in Example 1, then the student large language model in step SD3 may refer to any small LLM whose model architecture scale is much smaller than the commercially available Ali Qwen-2 model (i.e., the aforementioned teacher large language model); if steps SD1 to SD4 are to describe the pre-training process of the large language model based on the solution described in Example 2 and steps SD1 to SD4 are performed after step SA4, then the student large language model in step SD3 may refer to an LLM that has undergone weight pruning and LORA incremental training (e.g., For example, it may be the Ali Qwen-2 model that has undergone weight pruning and LORA incremental training and is already commercially available); if steps SD1 to SD4 are to describe the pre-training process of the large language model based on the scheme described in Example 8 and steps SD1 to SD4 are executed after step SC7, then the student large language model in step SD3 may refer to an LLM that has undergone weight pruning, neuron pruning, structured pruning and LORA incremental training (for example, it may be the Ali Qwen-2 model that has undergone weight pruning, neuron pruning, structured pruning and LORA incremental training and is already commercially available); and so on, no further details will be given.

[0268] It can be understood that the student large language model performs performance training based on the soft labels generated by the teacher large language model, which enables the student large language model to directly learn more detailed category information and features.

[0269] SD4. Perform incremental training on the student large language model through low-rank matrix decomposition based on the eighth incremental training data.

[0270] The eighth incremental training data may be instruction manuals, user manuals, maintenance manuals, etc. of various models of automobiles produced by different manufacturers.

[0271] It is understandable that step SD4 is actually to use LORA to fine-tune the student language model so that it has good adaptability in the actual offline question-answering process with users. At the same time, LORA can use the eighth incremental training data to make small-scale adaptive adjustments to the relevant parameters and weights of the student language model through low-rank matrix decomposition, so that it can still run efficiently without significantly increasing computing resources.

[0272] In summary, this embodiment can at least reduce the complexity of LLM through pruning and distillation (this is related to the combination of steps SD1 to SD4 with the scheme of the aforementioned embodiment), and provide additional optimization space in model performance based on LORA incremental training, which is not a simple combination of pruning, distillation and LORA algorithm. The multi-level optimization scheme provided in this embodiment can ensure that the model can still handle complex tasks with less computing and storage resources, and has good scalability. Due to the flexibility of LORA incremental training, the vehicle-mounted offline question-and-answer system can gradually improve the performance of the model as the data is updated, adapt to new usage scenarios and requirements, and ensure that the system is continuously optimized during its life cycle. Through the sequential optimization scheme of pruning, distillation and LORA incremental training, the computing and storage requirements of the model can be significantly reduced without losing model performance, while enhancing the adaptability, flexibility and scalability of the system. This optimization strategy is very suitable for scenarios that require efficient, lightweight and sustainable operation, such as Internet of Vehicles, smart devices and edge computing.

[0273] It should be noted that, based on the above embodiments, the vehicle-mounted offline question-and-answer system can also be optimized by combining quantization, hybrid cloud architecture, dedicated automobile manual QA model, and edge computing technology.

[0274] Exemplarily, on the basis of the sequential optimization scheme of pruning, distillation and LORA incremental training, the model parameters of LLM are further quantified to reduce the computational and storage requirements of LLM. Alternatively, through question-answering score judgment and user privacy judgment, the model reasoning and answer generation steps that cannot be answered by LLM after pruning, distillation and / or LORA incremental training (i.e., pre-training) and do not involve privacy are placed in the cloud for processing, and the model reasoning and answer generation steps with high local answer scores or involving privacy are performed locally on the vehicle server. This setting not only ensures the system response speed, but also utilizes the computing resources of the cloud. Alternatively, a question-answering model specifically for car manuals is developed. Although this may not be as good as LLM in natural language processing capabilities, it can achieve efficient question-answering through rules and knowledge bases, and perform local model training of small model parameters, but the question-answering effect may not be as good as that of large model parameters. Alternatively, the local chip of the vehicle machine is used as an edge computing device for model reasoning and answer generation to improve the response speed and computing power of the system.

[0275] Embodiment 10

[0276] This embodiment provides a vehicle-mounted offline question-answering method. Fig.12 This is a flow chart of an in-vehicle offline question-and-answer method provided in the tenth embodiment of the present invention. This embodiment is applicable to the query and answer scenarios between the user in the car cabin and the vehicle computer regarding vehicle driving operation, vehicle daily maintenance, vehicle troubleshooting, route navigation, future weather changes, etc. The in-vehicle offline question-and-answer method can be, but is not limited to, executed by the in-vehicle offline question-and-answer system in the aforementioned embodiment as the execution subject, and the execution subject can be implemented in software and / or hardware. Fig.12 As shown, the vehicle-mounted offline question-answering method comprises at least the following steps:

[0277] S1. Obtaining a query instruction input by a user through a user terminal, converting the query instruction into first format data and uploading it to a vehicle-mounted server.

[0278] S2. Receive the first format data through the vehicle server and call the vehicle practical data stored in the database, so as to generate response data based on the first format data and the called vehicle practical data through the pre-trained large language model, and then send the response data to the user terminal.

[0279] S3. Receive response data through the user terminal, convert the response data into second format data, and then feed back the data to the user in a preset feedback form.

[0280] Among them, pre-training includes at least two of pruning training, distillation training and low-rank adaptation training.

[0281] Optionally, the pre-training includes at least pruning training and low-rank adaptation training;

[0282] The pre-training process of a large language model includes at least the following steps:

[0283] SA1. Calculate the important indicator parameters of each weight in the large language model;

[0284] SA2, reset at least one weight of an important indicator parameter in the large language model that is lower than a preset threshold;

[0285] SA3. Re-train the performance of the large language model using the first performance training data;

[0286] SA4. Perform incremental training on the large language model through low-rank matrix decomposition based on the first incremental training data.

[0287] Optionally, the pre-training includes at least pruning training and low-rank adaptation training;

[0288] The pre-training process of a large language model includes at least the following steps:

[0289] SB1. Evaluate the contribution of each neuron in a large language model.

[0290] SB2, delete at least one neuron in the large language model whose contribution is lower than a preset contribution;

[0291] SB3, re-performing performance training on the large language model using the second performance training data;

[0292] SB4. Perform incremental training on the large language model through low-rank matrix decomposition based on the second incremental training data.

[0293] Optionally, the pre-training includes at least pruning training and low-rank adaptation training;

[0294] The pre-training process of a large language model includes at least the following steps:

[0295] SC1, determine the model influence parameters of each structural unit in the large language model;

[0296] SC2, remove at least one structural unit in the large language model whose model influencing parameter is higher than the preset parameter value;

[0297] SC3, re-training the large language model using the third performance training data;

[0298] SC4. Incrementally train the large language model through low-rank matrix decomposition based on the third incremental training data.

[0299] Optionally, the pre-training includes at least pruning training and low-rank adaptation training;

[0300] The pre-training process of a large language model includes at least the following steps:

[0301] SA1. Calculate the important indicator parameters of each weight in the large language model;

[0302] SA2, reset at least one weight of an important indicator parameter in the large language model that is lower than a preset threshold;

[0303] SA3. Re-train the performance of the large language model using the first performance training data;

[0304] SB1. Evaluate the contribution of each neuron in a large language model.

[0305] SB2, delete at least one neuron in the large language model whose contribution is lower than a preset contribution;

[0306] SB3, re-performing performance training on the large language model using the second performance training data;

[0307] SB5. Incrementally training the large language model by low-rank matrix decomposition based on the fourth incremental training data;

[0308] or,

[0309] The pre-training process of a large language model includes at least the following steps:

[0310] SA1. Calculate the important indicator parameters of each weight in the large language model;

[0311] SA2, reset at least one weight of an important indicator parameter in the large language model that is lower than a preset threshold;

[0312] SA3. Re-train the performance of the large language model using the first performance training data;

[0313] SC1, determine the model influence parameters of each structural unit in the large language model;

[0314] SC2, remove at least one structural unit in the large language model whose model influencing parameter is higher than the preset parameter value;

[0315] SC3, re-training the large language model using the third performance training data;

[0316] SC5. Incrementally train the large language model based on the fifth incremental training data by low-rank matrix decomposition.

[0317] or,

[0318] The pre-training process of a large language model includes at least the following steps:

[0319] SB1. Evaluate the contribution of each neuron in a large language model.

[0320] SB2, delete at least one neuron in the large language model whose contribution is lower than a preset contribution;

[0321] SB3, re-performing performance training on the large language model using the second performance training data;

[0322] SC1, determine the model influence parameters of each structural unit in the large language model;

[0323] SC2, remove at least one structural unit in the large language model whose model influencing parameter is higher than the preset parameter value;

[0324] SC3, re-training the large language model using the third performance training data;

[0325] SC6. Incrementally train the large language model through low-rank matrix decomposition based on the sixth incremental training data.

[0326] Optionally, the pre-training includes at least pruning training and low-rank adaptation training;

[0327] The pre-training process of a large language model includes at least the following steps:

[0328] SA1. Calculate the important indicator parameters of each weight in the large language model;

[0329] SA2, reset at least one weight of an important indicator parameter in the large language model that is lower than a preset threshold;

[0330] SA3. Re-train the performance of the large language model using the first performance training data;

[0331] SB1. Evaluate the contribution of each neuron in a large language model.

[0332] SB2, delete at least one neuron in the large language model whose contribution is lower than a preset contribution;

[0333] SB3, re-performing performance training on the large language model using the second performance training data;

[0334] SC1, determine the model influence parameters of each structural unit in the large language model;

[0335] SC2, remove at least one structural unit in the large language model whose model influencing parameter is higher than the preset parameter value;

[0336] SC3, re-training the large language model using the third performance training data;

[0337] SC7. Incrementally train the large language model through low-rank matrix decomposition based on the seventh incremental training data.

[0338] Optionally, the pre-training includes at least distillation training and low-rank adaptation training;

[0339] The pre-training process of a large language model includes at least the following steps:

[0340] SD1. Use the teacher model training data to train the large language model to obtain the teacher large language model;

[0341] SD2. Input the preset in-vehicle question and answer data into the teacher's large language model to obtain the data probability distribution of the preset in-vehicle question and answer data;

[0342] SD3. Establish a student language model and perform performance training on the student language model based on the data probability distribution;

[0343] SD4. Perform incremental training on the student large language model through low-rank matrix decomposition based on the eighth incremental training data.

[0344] To sum up, this embodiment uses the pre-trained LLM to make the in-vehicle offline question and answer system adaptable to application scenarios of complex natural language queries, which is more conducive to the system understanding the user's intention and then performing accurate question and answer responses, solving the technical problem of low efficiency of existing electronic manual information retrieval, and is conducive to improving the query efficiency of the in-vehicle offline question and answer system.

[0345] At the same time, since the LLM in this embodiment has undergone a pre-training process including at least two methods of pruning training, distillation training and LORA training, the model structure of the pre-trained LLM is streamlined and optimized. On the one hand, it can reduce the computing and storage requirements of the model when it is deployed locally, so that the pre-trained LLM can be used locally offline under the existing computing power of the vehicle computer; on the other hand, it can also perform continuous learning and iteration based on the user's query data, realize automatic updating of the database in the vehicle-mounted offline question-and-answer system, ensure the accuracy and timeliness of the answers, and facilitate maintenance and updating.

[0346] In addition, because the pre-trained LLM can be used locally offline, this embodiment effectively reduces the response time of the in-vehicle offline question and answer system, which is conducive to meeting the user's instant question and answer needs and improving the user experience. Moreover, since all data processing in the in-vehicle offline question and answer system can be completed locally, data uploading to the cloud is avoided, thereby greatly reducing the risk of data leakage and helping to protect user privacy and data security.

[0347] Embodiment 11

[0348] This embodiment provides an electronic device, Fig.13 1 is a schematic diagram of the structure of an electronic device provided by the eleventh embodiment of the present invention. Fig.13The electronic device 1000 includes a processor 1001 and a memory 1002. The memory 1002 stores computer-readable instructions. When the computer-readable instructions are executed by the processor 1001, the steps of any method included in the tenth embodiment are executed.

[0349] Through the above technical solution, the processor 1001 and the memory 1002 are interconnected and communicate with each other through a communication bus and / or other forms of connection mechanisms (not marked), and the memory 1002 stores a computer program executable by the processor. When the electronic device 1000 is running, the processor 1001 executes the computer program to execute the method in any optional implementation method of Example 10 to achieve the following functions: obtaining a query instruction input by a user through a user terminal, converting the query instruction into a first format data and uploading it to a vehicle-mounted server; receiving the first format data through the vehicle-mounted server and calling the vehicle practical data stored in the database, so as to generate response data based on the first format data and the called vehicle practical data through a pre-trained large language model, and then sending the response data to the user terminal; receiving the response data through the user terminal, converting the response data into a second format data and then feeding back to the user in a preset feedback form.

[0350] Embodiment 12

[0351] This embodiment provides a vehicle, which is integrated with at least the vehicle-mounted offline question-answering system described in any one of Embodiments 1 to 9.

[0352] The above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some of the technical features therein with equivalents, or change the order of the method steps. These modifications, replacements or changes in the order of steps do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. An in-vehicle offline question-answering system, characterized in that: At least includes a user terminal, a database and an on-vehicle server; The user terminal is connected to the vehicle-mounted server and is at least used to obtain a query instruction input by a user, so as to convert the query instruction into data in a first format and then upload the data to the vehicle-mounted server; and receiving response data, converting the response data into second format data and feeding back the data to the user in a preset feedback form; The database is connected to the vehicle-mounted server and is at least used to store vehicle practical data for the vehicle-mounted server to call; The vehicle-mounted server is configured with a pre-trained large language model, and is at least used to receive the first format data, so that the pre-trained large language model generates the response data based on the first format data and the called vehicle practical data, and then sends the response data to the user terminal; The pre-training includes at least two of pruning training, distillation training and low-rank adaptation training.

2. The vehicle-mounted offline question-answering system according to claim 1, characterized in that: The pre-training includes at least the pruning training and the low-rank adaptation training; The pre-training process of the large language model includes at least the following steps: SA1. Calculate important index parameters of each weight in the large language model; SA2. Resetting to zero at least one weight of the important indicator parameter in the large language model that is lower than a preset threshold; SA3. Re-performing performance training on the large language model using the first performance training data; SA4. Perform incremental training on the large language model through low-rank matrix decomposition based on the first incremental training data.

3. The vehicle-mounted offline question-answering system according to claim 1, characterized in that: The pre-training includes at least the pruning training and the low-rank adaptation training; The pre-training process of the large language model includes at least the following steps: SB1. Evaluate the contribution of each neuron in the large language model; SB2. deleting at least one neuron in the large language model whose contribution is lower than a preset contribution; SB3. Re-performing performance training on the large language model using the second performance training data; SB4. Perform incremental training on the large language model through low-rank matrix decomposition based on the second incremental training data.

4. The vehicle-mounted offline question-answering system according to claim 1, characterized in that: The pre-training includes at least the pruning training and the low-rank adaptation training; The pre-training process of the large language model includes at least the following steps: SC1. determining a model influencing parameter of each structural unit in the large language model; SC2. removing at least one structural unit in the large language model whose model influencing parameter is higher than a preset parameter value; SC3. Re-performing performance training on the large language model using third performance training data; SC4. Perform incremental training on the large language model through low-rank matrix decomposition based on the third incremental training data.

5. The vehicle-mounted offline question-answering system according to claim 1, characterized in that: The pre-training includes at least the pruning training and the low-rank adaptation training; The pre-training process of the large language model includes at least the following steps: SA1. Calculate important index parameters of each weight in the large language model; SA2. Resetting to zero at least one weight of the important indicator parameter in the large language model that is lower than a preset threshold; SA3. Re-performing performance training on the large language model using the first performance training data; SB1. Evaluate the contribution of each neuron in the large language model; SB2. deleting at least one neuron in the large language model whose contribution is lower than a preset contribution; SB3. Re-performing performance training on the large language model using the second performance training data; SB5. Performing incremental training on the large language model through low-rank matrix decomposition based on the fourth incremental training data; or, The pre-training process of the large language model includes at least the following steps: SA1. Calculate important index parameters of each weight in the large language model; SA2. Resetting to zero at least one weight of the important indicator parameter in the large language model that is lower than a preset threshold; SA3. Re-performing performance training on the large language model using the first performance training data; SC1. determining a model influencing parameter of each structural unit in the large language model; SC2. removing at least one structural unit in the large language model whose model influencing parameter is higher than a preset parameter value; SC3. Re-performing performance training on the large language model using third performance training data; SC5. Performing incremental training on the large language model through low-rank matrix decomposition based on the fifth incremental training data; or, The pre-training process of the large language model includes at least the following steps: SB1. Evaluate the contribution of each neuron in the large language model; SB2. deleting at least one neuron in the large language model whose contribution is lower than a preset contribution; SB3. Re-performing performance training on the large language model using the second performance training data; SC1. determining a model influencing parameter of each structural unit in the large language model; SC2. removing at least one structural unit in the large language model whose model influencing parameter is higher than a preset parameter value; SC3. Re-performing performance training on the large language model using third performance training data; SC6. Perform incremental training on the large language model through low-rank matrix decomposition based on the sixth incremental training data.

6. The vehicle-mounted offline question-answering system according to claim 1, characterized in that: The pre-training includes at least the pruning training and the low-rank adaptation training; The pre-training process of the large language model includes at least the following steps: SA1. Calculate important index parameters of each weight in the large language model; SA2. Resetting to zero at least one weight of the important indicator parameter in the large language model that is lower than a preset threshold; SA3. Re-performing performance training on the large language model using the first performance training data; SB1. Evaluate the contribution of each neuron in the large language model; SB2. deleting at least one neuron in the large language model whose contribution is lower than a preset contribution; SB3. Re-performing performance training on the large language model using the second performance training data; SC1. determining a model influencing parameter of each structural unit in the large language model; SC2. removing at least one structural unit in the large language model whose model influencing parameter is higher than a preset parameter value; SC3. Re-performing performance training on the large language model using third performance training data; SC7. Perform incremental training on the large language model through low-rank matrix decomposition based on the seventh incremental training data.

7. The vehicle-mounted offline question-answering system according to any one of claims 1 to 6, characterized in that: The pre-training includes at least the distillation training and the low-rank adaptation training; The pre-training process of the large language model includes at least the following steps: SD1. Use the teacher model training data to train the large language model to obtain a teacher large language model; SD2. Input the preset in-vehicle question and answer data into the teacher's large language model to obtain the data probability distribution of the preset in-vehicle question and answer data; SD3. Establish a student language model, and perform performance training on the student language model based on the data probability distribution; SD4. Perform incremental training on the student large language model through low-rank matrix decomposition based on the eighth incremental training data.

8. A vehicle-mounted offline question-answering method, characterized in that: The method is performed using the vehicle-mounted offline question-answering system of claim 1, and the method at least comprises: Acquiring a query instruction input by a user through a user terminal, converting the query instruction into data in a first format and uploading the data to the vehicle-mounted server; Receiving the first format data through the vehicle-mounted server and calling the vehicle practical data stored in the database, so as to generate response data based on the first format data and the called vehicle practical data through the pre-trained large language model, and then sending the response data to the user terminal; Receiving the response data through the user terminal, converting the response data into second format data and then feeding back to the user in a preset feedback form; The pre-training includes at least two of pruning training, distillation training and low-rank adaptation training.

9. An electronic device, characterized in that: The method comprises a processor and a memory, wherein the memory stores computer-readable instructions. When the computer-readable instructions are executed by the processor, the steps in the in-vehicle offline question-answering method as claimed in claim 8 are executed.

10. A vehicle, characterized in that: At least the vehicle-mounted offline question-answering system as described in any one of claims 1 to 7 is integrated.

Images