Intelligent query method and system based on large language model

Abstract

The present invention discloses an intelligent query method and system based on a large language model, and relates to the field of intelligent query technology. According to the data query question and the intended type of the response, a data generation method is adopted to respectively generate fine-tuning data sets suitable for the foreground interaction scenario, the parameter parsing interaction scenario and the time parsing interaction scenario; the large language model is fine-tuned using the fine-tuning data sets to obtain the foreground model, the parameter parsing model and the time parsing model, which are deployed and managed to obtain the large language model service; when receiving the user query input, the foreground model, the parameter parsing model and the time parsing model judge the query insertion, perform parameter parsing, and time parameter parsing, and then obtain the required data according to the output result combination, and return it to the user. The problem that the traditional data management system is not suitable for the intelligent query of the manufacturing execution system and the problem that there is no large language model in the prior art that meets the intelligent query in the production and manufacturing field is solved.

Description

Technical Field

[0001] The present invention relates to the field of intelligent query technology, and in particular to an intelligent query method and system based on a large language model. Background Art

[0002] Manufacturing Execution System (MES) is a software system used to manage and monitor the manufacturing process, which is used to implement and monitor the production plan. MES dashboard is an important part of the system, which is used to monitor key indicators and data in the production process in real time, helping production managers to quickly understand the production situation, make decisions in time, and improve production efficiency and quality. The content of MES dashboard includes information such as production progress, production quality and equipment status. Its content information is diverse and complex, and production managers cannot quickly query information. Traditional data management systems have high query efficiency when querying structured data, but they still face great challenges when facing unstructured data. The query methods of these systems often do not have the ability to process natural language, resulting in limitations in actual application scenarios. At present, information query in the Manufacturing Execution System (MES) is performed manually. Due to the complexity of the system, retrieval difficulty and query time have become difficulties in this field. Traditional information query systems require manual selection of query time period and content, which is extremely inconvenient.

[0003] With the development of large language models (LLMs), solutions using LLMs for data query are gradually emerging. Large language models have the capabilities of language understanding, question summarization, and code generation. Their general field question answering, text understanding, and text generation capabilities have been widely used. However, various vertical fields (such as the medical field, manufacturing field, etc.) are still exploring the application scenarios of large language models. Summary of the invention

[0004] The purpose of the present invention is to provide an intelligent query method and system based on a large language model, which is used to solve the problem that traditional data management systems in the prior art are not suitable for intelligent queries of manufacturing execution systems and that there is no large language model in the prior art that meets the requirements of intelligent queries in the production and manufacturing field.

[0005] The present invention solves the above problems through the following technical solutions:

[0006] An intelligent query method based on a large language model, comprising:

[0007] Step S1: According to the data query question and the response intention type, a data generation method is used to generate fine-tuning data sets suitable for the foreground interaction scenario, the parameter resolution interaction scenario and the time resolution interaction scenario respectively;

[0008] Step S2, fine-tuning the large language model using the fine-tuning data sets of the foreground interaction scenario, the parameter parsing interaction scenario, and the time parsing interaction scenario, respectively, to obtain a foreground model, a parameter parsing model, and a time parsing model;

[0009] Step S3: deploy and manage the foreground model, parameter parsing model and time parsing model to obtain a large language model service;

[0010] Step S4: upon receiving a user query input, the foreground model determines whether the query is available. If not, the user is prompted, or the user is prompted and the question-and-answer result is saved and recorded as historical information. If the query is available, the parameter parsing model is used to parse the query input. The dictionary output by the parameter parsing model contains the intent type and the key parameters of the query. If the key parameters of the query contain the time key parameter, the value corresponding to the time key parameter is input into the time parsing model, and the output results of the parameter parsing model and the time parsing model are combined to proceed to the next step. If the key parameters of the query contain the time key parameter or not, the next step is proceeded.

[0011] Step S5: Obtain required data according to the output result of the parameter analysis model or the combination of the output results of the parameter analysis model and the time analysis model, and return it to the user.

[0012] Furthermore, the step S1 specifically includes:

[0013] Step S11, dividing the interaction scenarios into foreground interaction scenarios, parameter analysis interaction scenarios and time analysis interaction scenarios;

[0014] Step S12: taking the historical information and the current query question as the first input question, summarizing the first input question and generating the first output text in a data generation manner, and setting the status symbols OK, RE, and NG at the beginning of the first output text, wherein OK indicates that the intent and parameters of the first output text are complete and can be queried; RE indicates that the first output text is within the queryable range, but the intent or parameters are incomplete; NG indicates that the query cannot be performed; the question-answering corpus consisting of the first input question and the first output text is added with irrelevant query negative samples to obtain the foreground model data;

[0015] The complete text that can be queried is used as the second input question. The second input question is learned by data generation and the unstructured data is converted into structured data. The second output text is output. The second output text is a dictionary containing the intent type and the key parameters of the query. The second input question and the second output text constitute parameter parsing data. The intent type needs to be consistent with the interface of the data management system, and the number of intents should be consistent with the number of interfaces of the data management system. Parameters should include all parameters required by the corresponding interface of the data management system, such as physical lines, device names, and time.

[0016] Taking the deterministic time expression and the non-deterministic time expression as the third input problem, using the data generation method to understand the time and extract key data of the third input problem, converting the unstructured time expression into computer-usable structured data, and outputting the third output text, which is a pre-designed executable code. The third input problem and the third output text constitute the time parsing data;

[0017] Step S13: pre-process the foreground model data, parameter analysis data and time analysis data respectively, divide them into a test set and a validation set, and obtain a foreground model data set, a parameter analysis data set and a time analysis data set.

[0018] Furthermore, the step S2 specifically includes:

[0019] Step S21, using the improved LoRA to adjust the structure of the basic model Qwen2-7B-Instruct to obtain a new model structure;

[0020] Step S22: fine-tune the new model using the foreground model data set, the parameter analysis data set and the time analysis data set respectively, to obtain a foreground model, a parameter analysis model and a time analysis model.

[0021] Furthermore, the step S21 specifically includes: the pre-training weight of the basic model Qwen2-7B-Instruct is Among them, b represents the dimension of the parameter matrix; increasing the low-rank matrix Low rank matrix Causal convolution conv1 and causal convolution conv2, to obtain a new model structure;

[0022] In step S22, the foreground model dataset, the parameter analysis dataset and the time analysis dataset are used to fine-tune the new model using the following method:

[0023] At the beginning of training, the matrix A is initialized using a normal distribution;

[0024] Matrix B is initialized with all zeros to ensure that the calculation results of the newly added structure are zero at the beginning of training;

[0025] The parameters of causal convolution onv1 and causal convolution conv2 are initialized using Kaiming to maintain the variance of the input data and accelerate the convergence of the model;

[0026] Fill the sequence of causal convolution conv1 and causal convolution conv2, and the filling conditions are satisfied:

[0027]

[0028] Among them, padding represents the padding size, ksize represents the size of the convolution kernel, and stride represents the convolution step size. When the condition is met, s * =s,s * is the sequence after filling, and s is the sequence before filling.

[0029] Therefore, the above improved LoRA structure calculation process is transformed into:

[0030] linear W (x)+conv2(linear B (conv1(linear A (x))));

[0031] Among them, linear represents linear operation.

[0032] Furthermore, the step S2 further includes:

[0033] Step S23: Evaluate the effects of the foreground model, parameter analysis model and time analysis model respectively to verify their effectiveness.

[0034] Furthermore, the step S3 specifically includes: using the vllm deployment framework to deploy the foreground model, parameter parsing model and time parsing model; then using Python to connect the data flow logic of the foreground model, parameter parsing model and time parsing model in series to ensure that they can interact and collaborate smoothly; and then using the Docker container to generate the environment deployment as a large language model service.

[0035] An intelligent query system based on a large language model, comprising:

[0036] The data generation module is used to generate foreground model data, parameter analysis data and time analysis data, and perform preprocessing, divide the training set and the validation set respectively to obtain the foreground model data set, parameter analysis data set and time analysis data set;

[0037] A model training module is used to use the foreground model data set, parameter parsing data set and time parsing data set to perform model training on the large language model LLM using an improved LoRA fine-tuning method, obtain a foreground model, a parameter parsing model and a time parsing model, and perform verification tests;

[0038] A large language model service, used to deploy and manage the foreground model, parameter parsing model and time parsing model;

[0039] The data management module is used to organize and store data, support data access and retrieval, and query and process the output of the large language model service;

[0040] The user interaction module is used to interact with users, provide user query entry and output user prompts; it is used to input and output user queries to the large language model service, and to receive the query and processing results of the data management module and display the data, and it is also used to save historical queries and collect user feedback.

[0041] Furthermore, the method for deploying and managing the foreground model, parameter parsing model and time parsing model is: use the vllm deployment framework to deploy the foreground model, parameter parsing model and time parsing model; then use Python to connect the data flow logic of the three models in series to ensure that they can interact and collaborate smoothly, and then use Docker containers to generate environment deployment.

[0042] Furthermore, the method for generating the foreground model data is:

[0043] Taking the historical information and the current query question as the first input question, summarizing the first input question to generate a first output text;

[0044] The starting position of the first output text is designed with status symbols: OK, RE, NG, where OK means that the intent and parameters of the problem summary output are complete and can be queried; RE means that the problem summary output is within the queryable range, but the intent or parameters are incomplete; NG means that the query cannot be performed.

[0045] Furthermore, irrelevant queries are added to the foreground model data set as negative samples, and the specific method is as follows:

[0046] Add questions from the general domain question answering dataset as negative query samples;

[0047] Use a random length of characters composed of numbers and letters as the query;

[0048] Generate fake text, city names, and person names as query content.

[0049] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0050] (1) The present invention divides the interaction scenarios and generates corresponding data sets using a data generation method. It fine-tunes the large language model through the improved LoRA to obtain three interaction models, which are then deployed and managed into a large language model service to ensure the efficiency and stability of the model operation. By flexibly switching between different models, the system can respond quickly in different scenarios, improving overall operating efficiency and user experience. It solves the problem that traditional data management systems are not suitable for intelligent queries in manufacturing execution systems and that there is no large language model in the prior art that meets the needs of intelligent queries in the production and manufacturing field.

[0051] (2) Compared with the traditional data query method in the manufacturing field, the present invention realizes fast data query through intelligent question and answer. Its advantage is that it simplifies the query process, reduces the query time cost, and significantly improves the user experience. Unlike the single-round question and answer of the traditional question and answer system, the present invention has the ability to integrate historical information and can support multiple rounds of question and answer. The present invention manages each interactive scene separately. This not only makes the maintenance process simpler and lowers the cost, but also ensures that the data processing process is concise and clear.

[0052] (3) In the field of manufacturing, it is difficult to achieve ideal results by only relying on large language models to convert unstructured data into structured data. In order to solve this problem, the present invention proposes a fine-tuning model based on interaction scenarios. The model is fine-tuned and trained according to different interaction scenarios so that the model can better adapt to and perform various specific tasks. At the same time, in order to improve the understanding ability of the fine-tuning model, this paper optimizes the model fine-tuning method, which includes prompt engineering and improved low-rank adaptation (LoRA). BRIEF DESCRIPTION OF THE DRAWINGS

[0053] Figure 1 is a system block diagram of the present invention;

[0054] Figure 2 is a flow chart of the present invention;

[0055] Figure 3 Schematic diagram of the improved LoRA structure;

[0056] Figure 4 Schematic diagram of the structure of causal convolution. DETAILED DESCRIPTION

[0057] The present invention is further described in detail below in conjunction with examples, but the embodiments of the present invention are not limited thereto.

[0058] Embodiment 1:

[0059] An intelligent query method based on a large language model, comprising:

[0060] Step S1: According to the data query question and the response intention type, a data generation method is used to generate fine-tuning data sets suitable for the foreground interaction scenario, the parameter resolution interaction scenario and the time resolution interaction scenario respectively;

[0061] In the field of manufacturing, it is difficult to achieve ideal results by relying solely on large language models to convert unstructured data into structured data. Since large language models have significant limitations in data query tasks in the field of manufacturing, it is difficult to directly use large language models for data query. To this end, the present invention breaks down the task into three interactive scenarios: foreground, parameter parsing, and time parsing, in order to improve the execution effect and accuracy of the task.

[0062] 1. Front-end interaction scenario

[0063] In the front-end interaction scenario, the user will communicate directly with the model. In this scenario, the model needs multiple capabilities to ensure smooth and effective interaction:

[0064] (1) Remembering historical conversations: The model needs to have a memory function that can save and use historical conversation content to make continuous queries smoother and more contextual.

[0065] (2) Summarizing query questions: Based on the current query and historical information, the model should be able to summarize and refine the user’s query questions to ensure the accuracy and relevance of the response.

[0066] (3) Guiding user queries: The model should be able to intelligently guide users to ask correct and effective queries, helping them to accurately express their needs and find the information they need.

[0067] (4) Determine data existence: The model needs to quickly and accurately determine whether the data queried by the user exists in the data management system in order to provide immediate feedback.

[0068] 2. Parameter parsing interaction scenario

[0069] In the parameter parsing interaction scenario, users do not communicate directly with the parameter parsing model, but interact with the system internally to obtain complete, correct, and valid unstructured data, namely, query text, from the system. In this scenario, the parameter parsing model needs to convert these queries into structured data. This requires the parameter parsing model to have the following capabilities:

[0070] (1) Intent recognition: The parameter parsing model needs to accurately understand and identify the intent of the user’s query in order to determine the type of information that needs to be processed.

[0071] (2) Key parameter extraction: The parameter parsing model needs to extract key parameters from the user's query, such as time, quantity, location, and device name, to ensure that all necessary information can be structured and used.

[0072] 3. Time-Resolved Interaction Scenarios

[0073] Time expressions can be roughly divided into deterministic time expressions and non-deterministic time expressions. Specifically, deterministic time expressions clearly point out a specific time point or time period, such as "January 2024". In contrast, non-deterministic time expressions lack a specific time point or time period, and usually require reasoning based on the current time. This type of time expression will change according to changes in the time base point and has a certain degree of uncertainty, such as "last week" or "yesterday". In order to meet query needs and improve user experience, the present invention requires that the parameter parsing model be able to obtain the ability to convert the above two unstructured time expressions into structured time data.

[0074] Based on the above scenario requirements and the characteristics of the large language model, the present invention finds that the large language model performs well in converting deterministic time representation into structured time. However, due to the lack of time base points, the large language model cannot infer non-deterministic time representation.

[0075] To solve the above problems, the present invention uses the code generation capability of the large language model to generate functions, and then obtains structured time data by executing the functions. Therefore, the parameter parsing model needs to have the following capabilities:

[0076] (1) Time comprehension ability: The parameter parsing model needs to accurately understand and parse various time expressions, including specific time points and relative time expressions.

[0077] (2) Key data extraction capability: The parameter parsing model needs to be able to extract key time information from the time representation and generate corresponding structured data.

[0078] Step S2, fine-tuning the large language model using the fine-tuning data sets of the foreground interaction scenario, the parameter parsing interaction scenario, and the time parsing interaction scenario, respectively, to obtain a foreground model, a parameter parsing model, and a time parsing model;

[0079] Fine-tuning refers to further training on an existing pre-trained model, using the existing knowledge of the pre-trained model to conduct targeted training on a smaller data set, thereby improving the performance of the model on a specific task. The fine-tuning step includes two parts: data preparation and training.

[0080] Data preparation: usually needs to cover data collection, data cleaning and data labeling. However, due to the particularity of the interaction scenario and the unique complexity of the data structure, these traditional methods were not adopted. Instead, data preparation was performed by generating data, which not only effectively avoided the tedious steps in the traditional data cleaning and data labeling process, but also greatly reduced the time cost. By generating high-quality synthetic data, the consistency and reliability of the data are ensured, thus providing a solid foundation for model training. This method not only improves efficiency, but also ensures the accuracy of the data, making the entire fine-tuning process more efficient and smooth. The generated data includes:

[0081] 1. Front-end model data

[0082] According to the requirements of the front-end interaction scenario, the model needs to have the ability to remember historical conversations and summarize query questions. Therefore, when designing data generation, historical information and current query questions are used as input, and these input questions are summarized to generate output text.

[0083] In order to guide users to query and judge the existence of data, three status symbols are designed at the beginning of the output text: "[OK]", "[RE]" and "[NG]". Among them, "[OK]" means that the intention and parameters of the problem summary output are complete and can be queried; "[RE]" means that the problem summary output is within the queryable range, but the intention or parameters are incomplete; "[NG]" means that the query cannot be performed.

[0084] In order to increase the stability of the model, this paper adds irrelevant queries as negative samples to the dataset. The specific methods of adding are as follows:

[0085] (1) General query questions: Add some questions from general domain question-answering datasets to the front-end model data as negative query samples.

[0086] (2) Random numbers and letters: Use numbers and letters to form a character of random length as a query.

[0087] (3) Fake data generation: Generate fake text, city names and person names as query content.

[0088] 2. Parameter parsing data

[0089] In the parameter parsing interaction scenario, the model needs to complete the intent recognition and key parameter extraction tasks. To this end, the input of the generated data is a complete text that can be queried, and the output is a dictionary containing intent types and parameters. Through this design, the model can learn from the data how to convert unstructured data into structured data.

[0090] The intent type needs to be consistent with the interface of the data management system, and the number of intents should be consistent with the number of interfaces of the data management system. The parameters should include all the parameters required by the corresponding interface of the data management system, such as physical line, device name and time, etc.

[0091] 3. Time-Resolved Data

[0092] Time parsing is to convert unstructured time representation into structured data that can be used by computers. To achieve this goal, the model needs to have the ability to understand time and extract key data. In the process of time parsing, the input can be a deterministic time representation or a non-deterministic time representation, and the output is a pre-designed executable code.

[0093] This data design effectively avoids the shortcoming of large models that cannot generate non-deterministic time expressions. Instead, it obtains the time base point through code, and uses algorithms to infer the time point or time period, and finally converts it into structured data.

[0094] Model training: Model training is the core step of fine-tuning, which mainly includes loading pre-trained models, adjusting model structure, setting training parameters and verification, etc. The present invention adjusts the model structure with an improved LoRA structure to improve the effect of the model.

[0095] In this invention, based on different interaction scenarios, the large model is fine-tuned using improved LoRA to obtain three fine-tuned models, namely the foreground model, the parameter parsing model and the time parsing model. All three models use the LoRA fine-tuning Decoder-only architecture, with the same parameter quantity and model structure, but they are fine-tuned in different interaction scenarios, and the feature distribution inside the model has shifted.

[0096] The model structure and training process of the three models are the same, but they are fine-tuned using data from three different tasks. The specific training steps are as follows:

[0097] (1) Select Qwen2-7B-Instruct as the base model.

[0098] (2) Use the improved LoRA to adjust the model structure and obtain a new model structure.

[0099] (3) The three data sets generated in the data preparation phase are used to fine-tune the model to obtain the foreground model, parameter analysis model, and time analysis model.

[0100] (4) Evaluate the above model to verify its effectiveness.

[0101] In step (2), the improved LoRA structure is as follows Figure 3 As shown:

[0102] The pre-trained weights of the original model are Where b represents the dimension of the parameter matrix; based on the original weights, two low-rank matrices are added and And two causal convolutions conv1 and conv2.

[0103] At the beginning of training, the matrix A is initialized using a normal distribution.

[0104] The matrix B is initialized with all zeros to ensure that the calculation results of the newly added structure are zero at the beginning of training.

[0105] The parameters of the two causal convolutions conv1 and conv2 are initialized using Kaiming to maintain the variance of the input data and accelerate the convergence of the model.

[0106] Among them, the structure of causal convolution is as follows Figure 4 shown.

[0107] In order to ensure that the sequence length does not change, the sequence needs to be padded. The padding conditions must meet the following conditions:

[0108]

[0109] Where padding represents the padding size, ksize represents the size of the convolution kernel, and stride represents the convolution step size. When the above condition is met, s * =s. The convolution kernel size of the causal convolution designed by the present invention is (1,3), the step length is 1, and a two-dimensional zero vector is filled at the beginning of the sequence. This design ensures that the length of the sequence will not change, and at the same time ensures that the newly added structure in the training process can only see the previous information, avoiding information leakage.

[0110] The above improved LoRA structure calculation process is transformed into:

[0111] linear W (x)+conv2(linear B (conv1(linear A (x))))

[0112] Through model fine-tuning, three types of models suitable for different scenarios are obtained: foreground model, parameter parsing model and time parsing model.

[0113] Step S3, deploy and manage the foreground model, parameter parsing model and time parsing model to obtain a large language model service; specifically: use the vllm deployment framework to deploy the foreground model, parameter parsing model and time parsing model; then use Python to connect the data flow logic of the foreground model, parameter parsing model and time parsing model in series to ensure that they can interact and collaborate smoothly; then use the Docker container to generate the environment deployment as a large language model service.

[0114] Step S4: upon receiving a user query input, the foreground model determines whether the query is available. If not, the user is prompted, or the user is prompted and the question-and-answer result is saved and recorded as historical information. If the query is available, the parameter parsing model is used to parse the query input. The dictionary output by the parameter parsing model contains the intent type and the key parameters of the query. If the key parameters of the query contain the time key parameter, the value corresponding to the time key parameter is input into the time parsing model, and the output results of the parameter parsing model and the time parsing model are combined to proceed to the next step. If the key parameters of the query contain the time key parameter or not, the next step is proceeded.

[0115] Step S5: Obtain required data according to the output result of the parameter analysis model or the combination of the output results of the parameter analysis model and the time analysis model, and return it to the user.

[0116] Embodiment 2:

[0117] Combined with Figure 1 As shown, an intelligent query system based on a large language model includes:

[0118] The data generation module is used to generate foreground model data, parameter analysis data and time analysis data, and perform preprocessing, divide the training set and the validation set respectively to obtain the foreground model data set, parameter analysis data set and time analysis data set;

[0119] A model training module is used to use the foreground model data set, parameter parsing data set and time parsing data set to respectively perform model training on the large language model LLM using an improved LoRA fine-tuning method to obtain models suitable for three interaction scenarios: a foreground model, a parameter parsing model and a time parsing model, and perform verification tests;

[0120] Large language model service, used to deploy and manage the foreground model, parameter parsing model and time parsing model to ensure the efficiency and stability of model operation. By flexibly switching different models, the system can respond quickly in different scenarios, improving overall operation efficiency and user experience;

[0121] The data management module is used to organize and store data, ensure data structuring and availability, support data access and retrieval, and efficiently query and process the output of the large language model service;

[0122] The user interaction module is used to interact with users, provide user query entry and output user prompts; it is used to input and output user queries to the large language model service, and to receive the query and processing results of the data management module and display the data, and it is also used to save historical queries and collect user feedback.

[0123] Taking the quality query as an example, the method of the present invention is demonstrated. First, a fine-tuning model based on three interaction scenarios is obtained. Model training includes the following:

[0124] Step 1: Get Qwen2-7B-Instruct as the basic large language model, which is trained on a variety of corpora and has the capabilities of language understanding, language generation, encoding, and reasoning.

[0125] Step 2: Generate the data required for quality query through the data generation module, including front-end data, parameter analysis data, and time analysis data. Next, pre-process these three types of data to generate corresponding question-answer corpora. Then, divide these question-answer corpora into training sets and validation sets, and finally obtain the front-end data set, parameter analysis data set, and time analysis data set.

[0126] Step 3: Use the large language model obtained in step 1 and the data set obtained in step 2 to train the model using the improved LoRA fine-tuning method to obtain the foreground model, parameter parsing model, and time parsing model.

[0127] Step 4: Test the foreground model, parameter parsing model, and time parsing model respectively. 1) Check whether the foreground model can correctly generate the status symbol and corresponding answer of the query question to ensure its accuracy and reliability. 2) Verify whether the key in the dictionary generated by the parameter parsing model is spelled correctly, and confirm whether the value of the dictionary is accurately extracted to ensure the correctness of the parsing result. 3) Check whether there are syntax errors in the code generated by the time parsing model. Execute the time parsing function to confirm whether the time can be parsed correctly to obtain the expected time point or time period.

[0128] Step 5: After testing, use the vllm deployment framework to deploy the front-end model, parameter parsing model, and time parsing model. Next, use Python to connect the data flow logic of the three models to ensure that they can interact and collaborate smoothly. Finally, use the Docker container to deploy the build environment as the system's large language model service.

[0129] The user interaction module, large language model service and data management module in the system work together to form the overall process of intelligent query, such as Figure 2 As shown, the specific process includes the following steps:

[0130] Step A: The user inputs the query to be queried in the user interaction module, and the user interaction module inputs the historical information and the current query question into the large language model service.

[0131] Step B: In the large language model service, the user's query will first be processed by the front-end model to determine whether it is a complete queryable question. The front-end model makes a judgment based on the output status symbol: "[OK]" means the query is complete and queryable, "[RE]" means the query is within the queryable range but lacks some parameters, and "[NG]" means the query is not within the query range.

[0132] Step C: If the status symbol is "[RE]" or "[NG]", the output result is directly returned to the user interaction module. When the status symbol is "[RE]", the user interaction module will save the question and answer result as a historical information record. When the status symbol is "[NG]", the question and answer result will not be saved.

[0133] Step D: If the status symbol is "[OK]", the parameter parsing model is used to convert a complete query into a dictionary. The dictionary contains two keys, intentType and param, which represent the query intent type and the key parameter of the query respectively. The value corresponding to intentType is a string representing the interface name, and the value corresponding to param is a dictionary containing the key of the key parameter of the query, such as device, line, or daterange. For example:

[0134] {

[0135] "intentType":"active / overview","param":{"daterange":"July","device":"die casting machine"}

[0136] }

[0137] Step E: Determine whether the output of the parameter parsing model contains the daterange keyword. If it does not contain daterange, the output of the parameter parsing model is directly transferred to the data management module for processing.

[0138] Step F: If the output contains the daterange keyword, the value corresponding to the keyword is input into the time resolution model. The time resolution model will output an executable code, and by executing this code, structured time data can be obtained. Then, the output results of the parameter resolution model and the time resolution model are combined, and the combined results are passed to the data management module.

[0139] Step G: After receiving the intent and parameters output by the large language model service, the data management module will execute the corresponding instructions to obtain the required data. Subsequently, the obtained data is returned to the user interaction module for further processing.

[0140] Step H: The user interaction module renders the received data to generate an easy-to-observe format, such as a pie chart, a bar chart, or a line chart, and displays it to the user. In addition, the results of this round of question and answer will be saved as historical information to facilitate prompt information for multiple rounds of question and answer.

[0141] Compared with the traditional data query method in the manufacturing field, the present invention realizes fast data query through intelligent question and answer. Its advantage is that it simplifies the query process, reduces the query time cost, and significantly improves the user experience. Unlike the single-round question and answer of the traditional question and answer system, the present invention has the ability to integrate historical information and can support multiple rounds of question and answer. The present invention manages each interactive scene separately. This not only makes the maintenance process simpler and less costly, but also ensures that the data processing process is concise and clear.

[0142] Although the present invention is described herein with reference to the illustrative embodiments of the present invention, the above embodiments are only preferred embodiments of the present invention, and the embodiments of the present invention are not limited to the above embodiments. It should be understood that those skilled in the art can design many other modifications and embodiments, which will fall within the scope and spirit of the principles disclosed in this application.

Claims

1. An intelligent query method based on a large language model, characterized in that: include: Step S1: According to the data query question and the response intention type, a data generation method is used to generate fine-tuning data sets suitable for the foreground interaction scenario, the parameter resolution interaction scenario and the time resolution interaction scenario respectively; Step S2, fine-tuning the large language model using the fine-tuning data sets of the foreground interaction scenario, the parameter parsing interaction scenario, and the time parsing interaction scenario, respectively, to obtain a foreground model, a parameter parsing model, and a time parsing model; Step S3: deploy and manage the foreground model, parameter parsing model and time parsing model to obtain a large language model service; Step S4: upon receiving the user's query input, the foreground model determines whether the query is available. If not, the user is prompted; If the query is possible, the parameter parsing model is used to parse the query input. The dictionary output by the parameter parsing model contains the intent type and the key parameters of the query. If the key parameters of the query contain the time key parameter, the value corresponding to the time key parameter is input into the time parsing model, and then the output results of the parameter parsing model and the time parsing model are combined to proceed to the next step. If the key parameters of the query contain the time key parameter or not, proceed to the next step. Step S5: Obtain required data according to the output result of the parameter analysis model or the combination of the output results of the parameter analysis model and the time analysis model, and return it to the user.

2. The intelligent query method based on a large language model according to claim 1, characterized in that: The step S1 specifically includes: Step S11, dividing the interaction scenarios into foreground interaction scenarios, parameter analysis interaction scenarios and time analysis interaction scenarios; Step S12: taking the historical information and the current query question as the first input question, summarizing the first input question and generating the first output text in a data generation manner, and setting the status symbols OK, RE, and NG at the beginning of the first output text, wherein OK indicates that the intent and parameters of the first output text are complete and can be queried; RE indicates that the first output text is within the queryable range, but the intent or parameters are incomplete; NG indicates that the query cannot be performed; the question-answering corpus consisting of the first input question and the first output text is added with irrelevant query negative samples to obtain the foreground model data; The complete text that can be queried is used as the second input question, and the second input question is learned by data generation, and the unstructured data is converted into structured data, and a second output text is output. The second output text is a dictionary containing the intent type and key parameters of the query. The second input question and the second output text constitute parameter parsing data; Taking the deterministic time expression and the non-deterministic time expression as the third input problem, using the data generation method to understand the time and extract key data of the third input problem, converting the unstructured time expression into computer-usable structured data, and outputting the third output text, which is a pre-designed executable code. The third input problem and the third output text constitute the time parsing data; Step S13: pre-process the foreground model data, parameter analysis data and time analysis data respectively, divide them into a test set and a validation set, and obtain a foreground model data set, a parameter analysis data set and a time analysis data set.

3. The intelligent query method based on a large language model according to claim 2, characterized in that: The step S2 specifically includes: Step S21, using the improved LoRA to adjust the structure of the basic model Qwen2-7B-Instruct to obtain a new model structure; Step S22: fine-tune the new model using the foreground model data set, the parameter analysis data set and the time analysis data set respectively, to obtain a foreground model, a parameter analysis model and a time analysis model.

4. The intelligent query method based on a large language model according to claim 3, characterized in that: The step S21 specifically includes: the pre-training weight of the basic model Qwen2-7B-Instruct is Among them, b represents the dimension of the parameter matrix; increasing the low-rank matrix Low rank matrix Causal convolution conv1 and causal convolution conv2, to obtain a new model structure; In step S22, the foreground model dataset, the parameter analysis dataset and the time analysis dataset are used to fine-tune the new model using the following method: At the beginning of training, the matrix A is initialized using a normal distribution; Matrix B is initialized with all zeros to ensure that the calculation results of the newly added structure are zero at the beginning of training; The parameters of causal convolution onv1 and causal convolution conv2 are initialized using Kaiming to maintain the variance of the input data and accelerate the convergence of the model; Fill the sequence of causal convolution conv1 and causal convolution conv2, and the filling conditions are satisfied: Among them, padding represents the padding size, ksize represents the size of the convolution kernel, and stride represents the convolution step size. When the condition is met, s * =s,s * is the sequence after filling, and s is the sequence before filling.

5. The intelligent query method based on a large language model according to claim 3, characterized in that: The step S2 further comprises: Step S23: Evaluate the effects of the foreground model, parameter analysis model and time analysis model respectively to verify their effectiveness.

6. The intelligent query method based on a large language model according to claim 1, characterized in that: The step S3 specifically includes: using the vllm deployment framework to deploy the foreground model, parameter parsing model and time parsing model; then using Python to connect the data flow logic of the foreground model, parameter parsing model and time parsing model in series to ensure that they can interact and collaborate smoothly; and then using the Docker container to generate the environment deployment as a large language model service.

7. An intelligent query system based on a large language model, characterized in that: include: The data generation module is used to generate foreground model data, parameter analysis data and time analysis data, and perform preprocessing, divide the training set and the validation set respectively to obtain the foreground model data set, parameter analysis data set and time analysis data set; A model training module is used to use the foreground model data set, parameter parsing data set and time parsing data set to perform model training on the large language model LLM using an improved LoRA fine-tuning method, obtain a foreground model, a parameter parsing model and a time parsing model, and perform verification tests; A large language model service, used to deploy and manage the foreground model, parameter parsing model and time parsing model; The data management module is used to organize and store data, support data access and retrieval, and query and process the output of the large language model service; User interaction module, used to interact with users, provide user query entry and output user prompts; It is used to input and output user queries to the large language model service, receive the query and processing results of the data management module and display the data, and also to save historical queries and collect user feedback.

8. The intelligent query system based on a large language model according to claim 7, characterized in that: The method for deploying and managing the foreground model, parameter parsing model and time parsing model is: use the vllm deployment framework to deploy the foreground model, parameter parsing model and time parsing model; then use Python to connect the data flow logic of the three models in series to ensure that they can interact and collaborate smoothly, and then use the Docker container to generate the environment deployment.

9. The intelligent query system based on a large language model according to claim 7, characterized in that: The method for generating foreground model data is: Taking the historical information and the current query question as the first input question, summarizing the first input question to generate a first output text; The starting position of the first output text is designed with status symbols: OK, RE, NG, where OK means that the intent and parameters of the problem summary output are complete and can be queried; RE means that the problem summary output is within the queryable range, but the intent or parameters are incomplete; NG means that the query cannot be performed.

10. The intelligent query system based on a large language model according to claim 9, characterized in that: The foreground model dataset also adds irrelevant queries as negative samples, and the specific method is as follows: Add questions from the general domain question answering dataset as negative query samples; Use a random length of characters composed of numbers and letters as the query; Generate fake text, city names, and person names as query content.

Images