Large language model method and device

Abstract

The invention provides a large language model method and device, and the method comprises the following steps: 1, obtaining and processing data of a specified industry, and constructing a professional knowledge base; the method comprises the following steps: constructing a professional knowledge base by acquiring specified industry data, vectorizing user question text and identity information, calculating cosine similarity with knowledge block vectors in the knowledge base to retrieve industry knowledge, generating identity background description by virtue of a large language model, and determining question intention types through an intention classification model. On the basis, RAG knowledge retrieval is conducted in the knowledge base, and finally answer content is generated through a large language model; the process overcomes the defects of dependence on artificial experience, insufficient generalization ability, high computing resource demand, lack of real-time interaction ability and the like in the prior art, more efficient, accurate and resource-friendly large language model interaction is realized, and the application performance of the large language model in different scenes is remarkably improved.

Description

Technical Field

[0001] The present invention relates to the technical field of natural language processing, and in particular to a method and device for a large language model. Background Art

[0002] In recent years, large language models (LLMs) have achieved remarkable results in the field of natural language processing (NLP). Models such as OpenAI's GPT series, Google's BERT and T5 have performed well in many tasks such as text generation, machine translation, question-answering systems, and sentiment analysis, thanks to their large number of parameters and excellent context understanding capabilities.

[0003] However, when interacting with large language models, the design and optimization of "prompt words" becomes a key problem. Currently, users interacting with large language models mostly rely on manually designed prompt words, a process full of subjectivity and trial and error. In complex tasks, users often need to adjust the content and structure of prompt words multiple times to get the ideal output. Existing prompt word enhancement technology also has many limitations: 1. Reliance on manual experience: Most of the prompt word design relies on manual experience, which is not only time-consuming but also inefficient when dealing with complex tasks; Second, insufficient generalization ability: Some automated prompt word generation and optimization methods perform poorly when faced with new tasks and lack sufficient generalization ability; 3. High computing resource requirements: The prompt word enhancement method based on reinforcement learning and multimodal input usually requires a lot of computing resources and is difficult to implement in some application scenarios; 4. Lack of real-time interaction capabilities: Current prompt word optimization is mostly based on offline training mode, which has response delay problems and cannot meet real-time interaction needs; To this end, a method and device for a large language model are proposed. Summary of the invention

[0004] In view of this, the embodiments of the present invention hope to provide a method and device for a large language model to solve or alleviate the technical problems existing in the prior art and at least provide a beneficial option.

[0005] To solve the above technical problems, a technical solution adopted in this application is: a method of a large language model, comprising the following steps: Step 1: Obtain and process the data of the designated industry to build a professional knowledge base; Step 2: Obtain the user's question text information and user identity information, and use the embedding model to vectorize the question text information and user identity information to generate a vectorized result; Step 3: Based on the vectorization result, the industry knowledge information related to the user's question text information is retrieved by calculating the cosine similarity with the knowledge block vector in the professional knowledge base; Step 4: Input the industry knowledge information and user identity information as prompt words into the large language model to generate a user identity background description; Step 5: Based on the intent classification model, vectorize the user identity background description and user question text information to generate output results; Step 6: Determine the intention type of the question from among the preset multiple question types according to the output results, and generate the user intention type; Step 7: Based on the user intention type, question text information, and user identity background description, perform RAG knowledge retrieval in the professional knowledge base to obtain knowledge information related to the question text; Step 8: Based on the context understanding capability of the large language model, generate answer content according to the user intent type and knowledge information related to the question text.

[0006] As a further preferred embodiment of the present technical solution, in step 1, the method for building a professional knowledge base comprises the following steps: Step 101: Based on the target industry scope, collect documents and forms related to the industry through multiple channels; Step 102: Convert the documents and table data into a unified text format and perform preprocessing to filter out professional knowledge data; Step 103, dividing the professional knowledge data into blocks according to the logical structure of the document or the rows and columns of the table to generate professional knowledge blocks; Step 104: Convert the professional knowledge block into a vector representation based on the embedding model to generate a knowledge block vector; Step 105: store the knowledge block vector in a professional knowledge base and associate it with corresponding metadata.

[0007] As a further preferred embodiment of the present technical solution, in step 3, the calculation formula of the cosine similarity is: ; in, is the problem vector in the vectorized result, is the knowledge block vector in the professional knowledge base; Dot product of vectors The calculation formula is: ; Where n is the dimension of the vector, and They are and The value of the vector in the i-th dimension; The magnitude of a vector The calculation formula is: ; The magnitude of a vector The calculation formula is: .

[0008] As a further preferred embodiment of the present technical solution, in step seven, the method for obtaining knowledge information related to the question text includes the following steps: Step 701: Integrate the user intention type, question text information, and user identity background description to form a comprehensive search condition text; Step 702: According to the characteristics of the professional knowledge base and the search requirements, a search strategy is selected to search the comprehensive search condition text to obtain knowledge information data; Step 703: Screen the retrieved knowledge information data to obtain knowledge information related to the question text.

[0009] As a further preferred embodiment of the present technical solution, in step 2, when the question text information and the user identity information are vectorized using the embedding model, the question text information and the user identity information are concatenated to form a fused text, and the fused text is input into an embedding model based on the Transformer architecture for vectorization.

[0010] As a further preferred embodiment of the present technical solution, in step 4, the large language model is an open source or commercial large language model based on the Transformer architecture, including GPT-4 and Llama.

[0011] As a further preferred embodiment of the present technical solution, in step five, the intent classification model is trained and constructed by a deep learning algorithm.

[0012] Provided as a further preferred embodiment of the present technical solution, in step six, the preset multiple question types include factual, explanatory, comparative, step-guiding, suggestive, inferential, hypothetical, opinion, emotional and subjective, confirmation, selection, rhetorical, humorous or sarcastic, speculative, moral, clarification, self-reflective, situational simulation, time and demand.

[0013] To solve the above technical problems, another technical solution adopted by the present application is: a device for a large language model, the device comprising: a professional knowledge base construction module, a user information processing module, an industry knowledge retrieval module, an identity background description generation module, an intention classification module, a RAG knowledge retrieval module and an answer content generation module; The professional knowledge base construction module is configured to obtain and process the data of the specified industry to construct a professional knowledge base; The user information processing module is configured to obtain user question text information and user identity information, and use the embedding model to vectorize the question text information and the user identity information to generate a vectorization result; The industry knowledge retrieval module is configured to retrieve industry knowledge information related to the industry knowledge in the professional knowledge base according to the vectorization result; The identity background description generating module is configured to input the industry knowledge information and the user identity information as prompt words into the large language model to generate the user identity background description; The intent classification module is configured to vectorize the user identity background description and the user question text information based on the intent classification model to generate an output result; The RAG knowledge retrieval module is configured to perform RAG knowledge retrieval in the professional knowledge base according to the user intention type, question text information and user identity background description, and obtain knowledge information related to the question text; The answer content generation module is configured to utilize the context understanding capability of the large language model, combined with the user's intention type and question text-related knowledge information, to generate answer content.

[0014] As a further preferred embodiment of the present technical solution, the device also includes a data cache module, and the data cache module is used to cache the retrieved knowledge information data and related retrieval conditions after the industry knowledge retrieval module and the RAG knowledge retrieval module perform the retrieval operation.

[0015] The embodiment of the present invention has the following advantages due to the adoption of the above technical solution: 1. The present invention greatly reduces the workload of manually designing prompt words through a series of automated steps such as building a professional knowledge base, using an embedding model for vectorization, and determining the question type using an intent classification model. Taking the medical industry as an example, when doctors ask professional questions, the system can quickly and accurately understand the intention and give answers, without the need for doctors to repeatedly adjust the prompt words, thereby improving the interaction efficiency. 2. The present invention can more comprehensively understand the problem and better cope with new tasks by comprehensively considering the user's question text information and identity information and combining the professional knowledge base to perform RAG knowledge retrieval. For example, in the emerging AI ethics consultation scenario, it can quickly adapt and provide targeted answers based on the existing knowledge and user identity; 3. The present invention avoids high resource consumption technology through optimized retrieval and intent analysis, and can run efficiently on ordinary hardware devices, thus broadening the application field, such as small enterprises with limited resources or mobile devices; 4. The present invention obtains user input in real time, quickly performs intent analysis and knowledge retrieval, and generates answer content in a timely manner to achieve real-time interaction. In the online customer service scenario, users can get replies quickly after asking questions, thereby improving user satisfaction.

[0016] The above summary is for illustrative purposes only and is not intended to be limiting in any way. In addition to the illustrative aspects, embodiments and features described above, further aspects, embodiments and features of the present invention will be readily apparent by reference to the accompanying drawings and the following detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or technical descriptions will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0018] Figure 1 A schematic diagram of a flow chart of a method for a large language model of the present invention; Figure 2 A schematic diagram of a process for constructing a professional knowledge base of the present invention; Figure 3 A schematic diagram of a process for obtaining knowledge information related to a question text according to the present invention; Figure 4 The figure is a schematic diagram of the functional modules of a large language model device of the present invention. DETAILED DESCRIPTION

[0019] The embodiments of the present disclosure are described in detail below with reference to the accompanying drawings.

[0020] It should be clear that the following embodiments of the present disclosure are described by specific specific examples, and those skilled in the art can easily understand other advantages and effects of the present disclosure from the contents disclosed in this specification. Obviously, the described embodiments are only a part of the embodiments of the present disclosure, rather than all of the embodiments. The present disclosure can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present disclosure. It should be noted that the following embodiments and features in the embodiments can be combined with each other in the absence of conflict. Based on the embodiments in the present disclosure, all other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of the present disclosure.

[0021] It should be noted that various aspects of the embodiments within the scope of the appended claims are described below. It should be apparent that the aspects described herein may be embodied in a wide variety of forms, and any specific structure and / or function described herein is merely illustrative. Based on the present disclosure, it should be understood by those skilled in the art that an aspect described herein may be implemented independently of any other aspect, and two or more of these aspects may be combined in various ways. For example, any number of aspects described herein may be used to implement the device and / or practice the method. In addition, other structures and / or functionalities other than one or more of the aspects described herein may be used to implement this device and / or practice this method.

[0022] It should also be noted that the illustrations provided in the following embodiments are only schematic illustrations of the basic concept of the present disclosure. The drawings only show components related to the present disclosure rather than being drawn according to the number, shape and size of components in actual implementation. In actual implementation, the type, quantity and proportion of each component may be changed arbitrarily, and the component layout may also be more complicated.

[0023] Additionally, in the following description, specific details are provided to facilitate a thorough understanding of the examples. However, it will be understood by those skilled in the art that the aspects described may be practiced without these specific details.

[0024] Figure 1 is a flow chart of a method for a large language model according to an embodiment of the present invention. It should be noted that if there are substantially the same results, the method of the present application is not limited to Figure 1 The process sequence shown is limited. Figure 1-Figure 3 As shown: A method for a large language model, comprising the following steps: Step 1: Obtain and process the data of the designated industry to build a professional knowledge base; Specifically, for designated industries, we will comprehensively sort out the channels for obtaining information. For example, in the medical industry, the websites of government health departments (such as the official website of the National Health Commission) will publish disease prevention and control policies and statistical data; professional medical databases (such as PubMed and Wanfang Medical Network) include a large number of academic research papers; hospitals store case records, medical treatment records and other information, and use multiple channels to complement each other to ensure the comprehensiveness and authority of the information; at the same time, we will use web crawler technology to automatically collect structured web page data, and use Python's Scrapy framework to capture articles, reports and other content on industry websites according to the set rules. For data in the database, if there is an open interface, use the corresponding API to extract data to improve information collection efficiency.

[0025] Step 2: Obtain the user's question text information and user identity information, and use the embedding model to vectorize the question text information and user identity information to generate a vectorized result; Specifically, first, when the user interacts with the system, the system will capture the question text information entered by the user in real time, such as the content entered by the user in the search box. At the same time, with the help of the identity information registered and stored in the background when the user logs into the system, such as in the e-commerce system, the user is registered as an ordinary consumer, merchant or platform administrator; in the medical consultation system, the user identity is a patient, doctor or medical researcher, etc. These identity information will be combined with the question text information to provide a richer context for subsequent analysis; in order to more effectively use the embedding model to process information, the question text information and the user identity information are usually spliced, for example, the identity information is pre-placed to form a fused text format of "[user identity information]: [question text information]"; taking the medical consultation scenario as an example, if the user identity is "diabetic patient" and the question is "What should I do if my blood sugar is always unstable recently", the spliced ​​fused text is "diabetic patient: What should I do if my blood sugar is always unstable recently". This splicing method allows the embedding model to better capture the correlation information between identity and question during the vectorization process; Then, select embedding models based on the Transformer architecture, such as Sentence-BERT and BERT. These models perform well in natural language processing tasks and can effectively capture the semantic features of text. Sentence-BERT is optimized based on BERT and is better at processing sentence-level semantic representations. It has high computational efficiency and is suitable for scenarios with high response speed requirements. BERT has advantages in processing complex semantic understanding and context association, and is more suitable for tasks that require deep semantic analysis. You can choose a suitable model based on the needs of the actual application scenario and system performance requirements. Next, the concatenated fused text (or directly the question text information and user identity information separately) is input into the selected embedding model. The model will perform a series of operations such as word segmentation and encoding on the text to convert the text into a vector representation in the vector space. Taking the Sentence-BERT model as an example, it will extract features from the text through pre-trained weights, map the text to a vector space of fixed dimensions, and generate a vectorized result. This vectorized result contains the semantic information of the text. Each dimension in the vector represents the numerical representation of the text on a certain semantic feature, which can be used to calculate the similarity between texts and other operations later. Finally, the generated vectorized results are stored in the temporary storage area of ​​the system for use in subsequent steps. When storing, they can be associated with the user ID or session ID for easy and quick retrieval. At the same time, the vectorized results are transmitted to the subsequent industry knowledge retrieval module to provide a data basis for retrieving industry knowledge information related to user questions.

[0026] Step 3: Based on the vectorization result, the industry knowledge information related to the user's question text information is retrieved by calculating the cosine similarity with the knowledge block vector in the professional knowledge base; Specifically, in actual calculations, numerical computing libraries (such as the NumPy library in Python) are used to efficiently perform vector operations; For example, using NumPy's dot() function to calculate the vector dot product, and the linalg.norm() function to calculate the vector modulus, and then get the value of cosine similarity, this can greatly improve the calculation efficiency and meet the system's performance requirements for large-scale vector calculations; Then, according to the industry characteristics and application scenario requirements, a cosine similarity threshold is pre-set ; In the medical field with high professional requirements, the threshold can be set higher, such as 0.8, to ensure the high relevance of the retrieved knowledge; while in general knowledge question-and-answer scenarios, the threshold can be appropriately lowered, such as 0.6, to obtain richer information; As the system is used and data accumulates, the threshold is dynamically adjusted by analyzing the accuracy and relevance of the search results. If it is found that there are too many search results but the relevance is not strong, the threshold can be appropriately increased; if there are too few search results or key information is omitted, the threshold can be lowered. Traverse all knowledge block vectors in the professional knowledge base and convert each knowledge block vector Vector with question Perform cosine similarity calculation; The calculated cosine similarity is compared with the set threshold Compare and set the cosine similarity to be greater than or equal to the threshold The industry knowledge information corresponding to the knowledge block vector is screened out to determine the industry knowledge information related to the user's question text information; If there are a large number of relevant knowledge information screened out, they will be sorted in descending order according to the cosine similarity, and the knowledge information with higher similarity will be output first; for example, the system retrieves 10 relevant knowledge information, sorts them according to similarity, and displays the top 5 pieces of information as key recommended content to users or for subsequent processing, so that users can quickly obtain the most relevant knowledge.

[0027] Step 4: Input the industry knowledge information and user identity information as prompt words into the large language model to generate a user identity background description; Specifically, first, the industry knowledge information retrieved in the previous step and the obtained user identity information are integrated to construct prompt words suitable for input into the large language model. When constructing, the information is organized according to a certain logic and format to highlight the key content. For example, in a medical scenario, if the user is a diabetic patient, the industry knowledge information includes common symptoms and treatment methods of diabetes, etc., and the following prompt words can be constructed: "The user's identity is a diabetic patient. Currently, the common symptoms of diabetes are [list symptoms], and the treatment methods are [list methods]. Please describe the user's identity background based on this information"; Choose open source or commercial large language models based on the Transformer architecture, such as GPT-4 and Llama. These models have been trained with a large amount of text and have strong language understanding and generation capabilities. When choosing, you need to consider factors such as the model's performance, applicable scenarios, and cost. If you have extremely high requirements for the accuracy and professionalism of the generated content and have sufficient budget, commercial models such as GPT-4 are a good choice; if you focus on cost-effectiveness and customization, you can choose open source models such as Llama and fine-tune them according to your own data; Then, the constructed prompt words are input into the selected large language model. After receiving the prompt words, the model will use the language knowledge and patterns learned in pre-training to understand and analyze the prompt words, and then generate a description of the user's identity background; Finally, the user identity background description generated by the large language model is verified to check whether the content is reasonable and accurate and whether it is consistent with known user information and industry knowledge. If the generated description has logical errors, information bias and other problems, it can be optimized by adjusting the content and structure of the prompt words or reselecting the large language model parameters (such as temperature parameters, controlling the randomness of the generated content), and the description can be generated again until a satisfactory result is obtained.

[0028] Step 5: Based on the intent classification model, vectorize the user identity background description and user question text information to generate output results; Specifically, first, build intent classification models based on deep learning algorithms, such as the common convolutional neural network (CNN), recurrent neural network (RNN) and its variants (such as LSTM, GRU), etc.; these models can automatically learn the feature representation in the text and classify the intent of the text; for example, when processing short text intent classification tasks, convolutional neural networks can quickly extract local features in the text; and for texts that need to consider the context order, recurrent neural networks and their variants can better capture the sequence information of the text; when selecting a model, it is necessary to comprehensively consider factors such as the complexity of the task, the size of the data, and computing resources; Then, use a vectorization method similar to that in step 2 to vectorize the user identity background description and the user question text information. You can continue to use embedding models based on the Transformer architecture, such as BERT, Sentence-BERT, etc., to convert the text into vector form. The purpose of this is to convert the text information into a numerical vector so that the intent classification model can calculate and analyze it. For example, the BERT model can map each word or phrase in the text to a high-dimensional vector space. Each dimension of the vector represents the numerical representation of the text at different semantic features, thereby retaining the semantic information of the text. Next, the vectorized user identity background description vector and user question text information vector are input into the intent classification model. The model will perform a series of operations and processing on these vectors. Through the neural network structure inside the model, the key features in the text vector are automatically extracted, and the intent classification is calculated based on these features. Taking a simple convolutional neural network intent classification model as an example, the model first performs a convolution operation on the input vector through the convolution layer to extract local features, then compresses and filters the features through the pooling layer, and finally maps the features to different intent categories through the fully connected layer, and outputs the prediction score for each intent category. Finally, after calculation, the intent classification model will generate an output result, which is usually a vector containing the prediction scores of each preset intent category. Each score represents the possibility that the input text belongs to the intent category. For example, assuming that 10 intent categories are preset, the model output result is a 10-dimensional vector, and each element in the vector corresponds to the prediction score of an intent category. The subsequent steps will determine the intent type of the question based on these scores.

[0029] Step 6: Determine the intention type of the question from among the preset multiple question types according to the output results, and generate the user intention type; Specifically, during the system development phase, multiple types of questions are pre-set based on business needs and common problem patterns. For example, in the medical consultation system, question types include symptom consultation (asking about the causes and diagnosis of specific symptoms), treatment plan (asking about the treatment methods and drug use of a certain disease), and preventive health care (asking about disease prevention measures, healthy lifestyles, etc.); in the financial field, there are investment advice (asking about investment product selection, investment strategies, etc.), account operation (asking about account registration, transfers, and other operational issues), and market conditions (asking about financial market dynamics, stock price trends, etc.); The output of the intent classification model is a series of scores representing the possibility of different intent categories. These scores are analyzed to find the intent category corresponding to the score with the largest value. For example, the output of the intent classification model is [0.1, 0.3, 0.8, 0.2, 0.1], which correspond to the scores of the preset question types of symptom consultation, treatment plan, preventive health care, investment advice, and account operation. Then the preventive health care type corresponding to the score of 0.8 is the current question intent type. To ensure the reliability of the classification results, a threshold is set. When the maximum score exceeds the threshold, the corresponding intent type is determined to be the user intent type. For example, if the threshold is set to 0.6, if the maximum score is 0.8, which is greater than the threshold 0.6, the user intent type can be determined. If the maximum score is 0.5, which is less than the threshold, the current result is considered unreliable, and it is necessary to recheck the input data, adjust the intent classification model, or supplement more training data to improve the accuracy of intent classification. Once the qualified intent type is determined, it will be recorded and marked as the user intent type. This type will serve as an important basis for subsequent operations, such as conducting more targeted RAG knowledge retrieval in the professional knowledge base to provide directional guidance for generating accurate answers.

[0030] Step 7: Based on the user intention type, question text information, and user identity background description, perform RAG knowledge retrieval in the professional knowledge base to obtain knowledge information related to the question text; Specifically, the user intent type, question text information, and user identity background description are organically integrated to form a comprehensive and targeted comprehensive search condition text. For example, in the e-commerce industry, if the user intent type is "product recommendation", the question text is "I want to buy a smart watch suitable for outdoor sports", and the user identity background description is "a sports enthusiast who often participates in marathon competitions", then the integrated search condition text is "recommend smart watches suitable for outdoor sports for sports enthusiasts who often participate in marathon competitions"; In the integration process, extract key information and organize it reasonably so that the search conditions can more accurately match the content in the knowledge base. For example, in the above example, highlight key information such as "marathon", "outdoor sports", and "smart watches" to more accurately filter out knowledge that meets the needs; Extract keywords based on the comprehensive search condition text, and use the keywords to search in the professional knowledge base. For example, extract keywords such as "marathon", "outdoor sports", and "smart watch" from "recommending smart watches suitable for outdoor sports for sports enthusiasts who often participate in marathon competitions", and search for knowledge blocks containing these keywords in the knowledge base. This method is simple and direct, and can quickly locate relevant information for clearly stated questions; With the help of natural language processing technology, the semantics of the search condition text is understood and matched with the semantics of the knowledge blocks in the knowledge base. For example, using semantic retrieval based on the embedding model, both the search condition text and the knowledge blocks in the knowledge base are converted into vector representations, and the relevance is determined by calculating the similarity between vectors (such as cosine similarity). This method can handle problems with ambiguous semantics or diverse expressions, and improve the accuracy and recall rate of retrieval; In practical applications, keyword search and semantic search are often used in combination. First, keyword search is used for preliminary screening to narrow the search scope, and then semantic search is used to accurately match the screened results, so as to obtain accurate knowledge information more efficiently.

[0031] According to the selected search strategy, search in the professional knowledge base to obtain a series of knowledge information data. This data contains various knowledge related to the problem, such as product introduction, usage method, case analysis, etc. The retrieved knowledge information data is screened to remove content that is irrelevant or less relevant to the question. Evaluation and screening can be performed based on pre-set rules, such as the degree of match between knowledge information and search conditions, the timeliness of knowledge, the reliability of sources, and other factors. For example, when screening recommended information for smart watches, priority is given to knowledge content that has been recently released and comes from authoritative evaluation agencies to ensure that the acquired knowledge information is actually helpful in answering user questions.

[0032] Integrate the filtered knowledge information and organize it in a certain logical order to make it more organized. For example, arrange it in order of importance, relevance, etc. to facilitate subsequent use; If the retrieved knowledge information is found to be missing or incomplete, try to supplement the relevant knowledge based on the structure and association of the knowledge base. For example, when recommending a smartwatch, if the retrieved knowledge only mentions some of the functions, you can supplement the battery life, price and other information of the watch by associating other relevant knowledge blocks to provide a more comprehensive answer.

[0033] Step 8: Based on the context understanding ability of the large language model, generate answer content according to the user's intention type and the knowledge information related to the question text; Specifically, first, the user intention types and the screened and integrated knowledge information related to the question text are reasonably organized to construct input content that is easy for the large language model to understand and process.

[0034] Then, select a suitable large language model, such as GPT-4, Llama, etc. (can be the same or different from the model used in step 4, depending on actual needs), and input the constructed input content into the model. When calling the model, you can set corresponding parameters according to the model characteristics and task requirements, such as the maximum length of the generated text, the temperature coefficient (used to control the randomness of the generated content, the higher the temperature, the stronger the randomness of the generated content; the lower the temperature, the more deterministic and conservative the generated content), etc. For example, for professional questions that require rigorous and standardized answers, the temperature coefficient can be appropriately lowered to make the generated answers more accurate and standardized; for some open and creative questions, the temperature coefficient can be appropriately increased to increase the diversity of answers; After receiving the input content, the large language model uses its powerful contextual understanding ability to analyze and understand the user's intentions and the connection between relevant knowledge information, and then generates corresponding answers based on the language patterns and knowledge learned in pre-training. For example, in the above legal consultation scenario, the large language model will combine legal clauses and cases to explain in detail the scope of application, specific meaning, and application methods of legal clauses in similar cases, and generate a complete and organized interpretation of legal clauses; Next, the answers generated by the large language model are optimized and verified. On the one hand, the answers are checked to see if they are complete and accurate, if they cover key information, and if there are any logical errors or unclear expressions. For example, check if all the key points of the legal clauses are explained in the answers, and if the case analysis is appropriate. On the other hand, the answers are formatted and optimized in style according to the requirements of the actual application scenarios.

[0035] Finally, the optimized and verified answer content is output to the user, completing the entire interactive process of the intention analysis based on the large language model and enhanced prompt words. The output method can be selected according to the specific application scenario, such as directly displaying the text answer on the web page interface, converting the answer content into voice and playing it to the user through speech synthesis technology, etc.

[0036] In one embodiment, specifically, in step 1, the method for building a professional knowledge base includes the following steps: Step 101: Based on the target industry scope, collect documents and forms related to the industry through multiple channels; Among them, multiple channels cover public data sources, such as industry websites, academic databases, etc., from which industry reports, research papers, etc. are obtained; there are also internal industry resources, such as internal business documents, operational data tables, etc.; as well as professional forums, industry conference materials, etc., to ensure the comprehensiveness and professionalism of the collected information. For example, in the medical industry, medical journal literature, hospital case data, drug instructions and other materials will be collected.

[0037] Step 102: Convert the documents and table data into a unified text format and perform preprocessing to filter out professional knowledge data; Specifically, first, the collected documents and table data in different formats are uniformly converted into a text format that is easy to process; Next, preprocessing is carried out to remove noise information, such as irrelevant headers and footers, and advertising content; spelling and grammatical errors are corrected; the data is also deduplicated to ensure the accuracy and purity of the information and to screen out truly valuable professional knowledge information.

[0038] Step 103, dividing the professional knowledge data into blocks according to the logical structure of the document or the rows and columns of the table to generate professional knowledge blocks; Specifically, the professional knowledge materials are divided into blocks according to the logical structure of the document itself, such as the chapters and paragraphs of an article, or according to the rows and columns of a table; this is done to make each knowledge block contain relatively independent and complete information units, and to generate professional knowledge blocks of appropriate size that are easy to manage and retrieve; for example, a medical textbook will be split into multiple knowledge blocks according to chapters and paragraphs, and each knowledge block focuses on a specific medical knowledge point.

[0039] Step 104: Convert the professional knowledge block into a vector representation based on the embedding model to generate a knowledge block vector; Among them, the embedding model can map text information into a low-dimensional vector space and retain the semantic features of the text. Commonly used models include the Sentence-BERT model based on the Transformer architecture, which can efficiently realize text vectorization and facilitate subsequent similarity calculation and knowledge retrieval.

[0040] Step 105, storing the knowledge block vector in a professional knowledge base and associating corresponding metadata; Specifically, the generated knowledge block vector is stored in a professional knowledge base. Generally, a vector database such as Milvus, Faiss, etc. is selected. When storing, corresponding metadata is associated with each knowledge block vector. These metadata contain key information such as the source of the knowledge block, subject category, and update time. By associating metadata, it is convenient to manage, update and filter the knowledge in the knowledge base, thereby improving the efficiency and accuracy of knowledge retrieval. At the same time, a maintenance and update mechanism for the professional knowledge base is established, new industry information is collected regularly, and the above steps are repeated to update the knowledge base to ensure the timeliness and accuracy of the knowledge.

[0041] In one embodiment, specifically, in step three, the calculation formula of cosine similarity is: ; in, is the problem vector in the vectorized result, is the knowledge block vector in the professional knowledge base; Dot product of vectors The calculation formula is: ; Where n is the dimension of the vector, and They are and The value of the vector in the i-th dimension; The magnitude of a vector The calculation formula is: ; The magnitude of a vector The calculation formula is: ; In practical applications, the cosine similarity calculated using these formulas is compared with a pre-set threshold. The industry knowledge information corresponding to the knowledge block vector that is greater than or equal to the threshold is considered to be relevant to the user's question, thereby achieving the purpose of accurately retrieving relevant knowledge from the professional knowledge base.

[0042] In one embodiment, specifically, in step seven, the method for obtaining knowledge information related to the question text includes the following steps: Step 701: Integrate the user intention type, question text information, and user identity background description to form a comprehensive search condition text; Specifically, combining the user intent type, question text information, and user identity background description can construct search conditions that more comprehensively and accurately reflect user needs. The user intent type indicates the core direction of the problem. For example, the "explanatory" intent indicates that the user expects to obtain an explanation of a concept or phenomenon; the question text information contains the specific content that the user is concerned about; and the user identity background description provides additional contextual information to help filter out knowledge that better meets the user's specific needs. For example, in a medical consultation scenario, the user intent type is "diagnosis advice", the question text is "I have been having headaches and nausea recently, what's going on", and the user identity background description is "an office worker who has been working at a desk for a long time." When integrated, this information will be organized into a comprehensive search condition text such as "explain the reasons for the recent headaches and nausea for office workers who have been working at a desk for a long time and give diagnosis advice"; In the integration process, key information is extracted and reorganized reasonably. From the above example, keywords such as "headache", "nausea", "long-term desk work", and "diagnosis suggestions" are extracted and reorganized according to logical relationships, so that the search condition text can accurately cover user needs and facilitate subsequent search operations. This step improves the pertinence of the search, avoids blind searches in the knowledge base, and improves search efficiency.

[0043] Step 702: According to the characteristics of the professional knowledge base and the search requirements, a search strategy is selected to search the comprehensive search condition text to obtain knowledge information data; Specifically, select the appropriate search strategy based on the structure of the professional knowledge base, data characteristics, and search requirements. If the professional knowledge base is built based on keyword indexes and has high requirements for search speed, then a keyword-based search strategy is more appropriate; if the knowledge base stores data in vector form and focuses on semantic understanding, a semantic search strategy (such as vector search based on cosine similarity) will be more effective. For example, in a legal professional knowledge base, if its data is stored in the form of legal texts and a keyword index is established, for issues that clearly involve specific legal provisions, keyword-based searches can quickly locate relevant provisions; After determining the search strategy, search the comprehensive search condition text. Taking keyword search as an example, the keywords in the comprehensive search condition text are matched and searched in the knowledge base; if semantic search is used, the text is converted into a vector and the similarity is calculated with the knowledge vector in the knowledge base. Through the search, a series of knowledge information data related to the problem is obtained from the professional knowledge base. These data contain redundant or irrelevant content and need to be further screened.

[0044] Step 703: Screen the retrieved knowledge information data to obtain knowledge information related to the question text; Specifically, formulate screening rules to remove irrelevant information in the search results and retain knowledge that is closely related to the question text. The screening criteria are usually based on factors such as relevance, accuracy, and timeliness. In terms of relevance, check the degree of match between the knowledge information and the comprehensive search condition text. For example, in the search of the medical knowledge base, determine whether the acquired knowledge is targeted at the problem of "headaches and nausea caused by long-term desk work"; in terms of accuracy, ensure that the source of knowledge information is reliable and the content is correct; timeliness is particularly important for some fields that are greatly affected by time (such as technology and finance), and give priority to the latest knowledge information; According to the screening criteria, the retrieved knowledge information data is evaluated and screened one by one. The evaluation results can be quantified by setting thresholds. For example, for relevance, the text similarity score is calculated, and the knowledge information with a score higher than a certain threshold is retained. After screening, the knowledge information that is highly relevant to the question text is finally obtained. This information will serve as an important basis for generating answer content, improving the accuracy and usefulness of the answer.

[0045] In one embodiment, specifically, in step 2, when the question text information and the user identity information are vectorized using the embedding model, the question text information and the user identity information are concatenated to form a fused text, and the fused text is input into the embedding model based on the Transformer architecture for vectorization; Among them, the user identity information includes the user's background characteristics, such as occupation, field of interest, professional level, etc., while the question text information reflects the user's current specific needs. Splicing the two allows the embedding model to obtain richer contextual information. For example, a user with a doctor's identity asks about "the treatment plan for the recent flu." After splicing the identity information of "doctor" with the question text, the model can understand that the user expects a more professional and in-depth treatment plan, not just basic treatment suggestions for the general public; Through splicing, the model can learn the semantic connection between identity information and questions. Taking the financial field as an example, when ordinary investors and senior financial analysts ask questions, combined with their different identities, the model can better understand that ordinary investors are more concerned with basic market trend analysis, while senior analysts need more in-depth industry data interpretation and trend prediction model-related content, thereby generating more targeted vector representations; By converting the fused text into vector form, subsequent calculations and processing are more efficient. When calculating the cosine similarity of the knowledge block vector in the professional knowledge base in step 3, the data in vector form can be directly operated mathematically, which can conveniently and quickly measure the similarity between texts, thereby retrieving relevant industry knowledge information. For example, in the e-commerce field, after the user identity and product consultation questions are concatenated and vectorized, it is possible to quickly find product introductions and answer information with high matching in the product knowledge base; Vectorization provides a unified data format for subsequent intent classification models, RAG knowledge retrieval, etc. Different text information is converted into vectors of the same dimension through vectorization, so that each module can work together more conveniently and improve the operating efficiency and accuracy of the entire system.

[0046] In one embodiment, specifically, in step 4, the large language model is an open source or commercial large language model based on the Transformer architecture, including GPT-4 and Llama; Among them, the Transformer architecture, with its unique multi-head attention mechanism, can efficiently process long sequences of text, accurately capture the complex dependencies between words in the text, and deeply understand the semantics of the text. In step four, the large language model needs to generate a user identity background description based on industry knowledge information and user identity information. The Transformer architecture enables the model to fully explore the key content and semantic associations in the input information, and then generate a logically coherent and semantically accurate description. Taking the financial field as an example, if the input industry knowledge contains current market volatility information, and the user identity is a senior investor, the model based on the Transformer architecture can understand the relationship between the two and generate identity background descriptions that fit the actual situation, such as "As an experienced senior investor, in the current market environment of frequent fluctuations, with his keen market insight and rich investment experience, he continues to pay attention to market trends and actively adjusts the investment portfolio." As an advanced commercial large language model, GPT-4 has a huge parameter scale and a massive amount of training data, and has powerful language understanding and generation capabilities. When generating user identity background descriptions, it can generate high-quality, detailed and realistic content. In medical scenarios, after inputting the patient's symptom-related industry knowledge and patient identity information, GPT-4 can generate a detailed patient identity background description, providing rich background information for subsequent diagnostic recommendations; Llama is an open source large language model with the advantages of being customizable and scalable. Researchers and developers can fine-tune it using their own data according to specific industry or task requirements, making the model perform better in specific areas. In a professional question-and-answer system for a specific industry, such as the machinery manufacturing industry, developers can use professional information in the industry to fine-tune Llama. After inputting industry knowledge and user identity information, it can generate a user identity background description that conforms to the context of the machinery manufacturing industry, providing a more targeted background basis for solving professional problems.

[0047] In one embodiment, specifically, in step five, the intent classification model is trained and constructed by a deep learning algorithm; Specifically, common deep learning algorithms used to build intent classification models include convolutional neural networks (CNN), recurrent neural networks (RNN) and their variants (such as long short-term memory networks (LSTMs) and gated recurrent units (GRUs). CNNs can automatically extract local features in texts through convolutional layers, and can quickly capture key information for short text intent classification tasks. LSTMs or GRUs can remember the previously mentioned information such as computer freezes and invalid operations, and accurately determine that the user's intention is to seek further solutions; In order to train the intent classification model, a large amount of text data labeled with intent types needs to be collected. These data come from a wide range of sources, such as historical user question records, artificially simulated question samples, etc. In the labeling process, it is necessary to ensure that the intent type classification is accurate and clear, and consistent with the preset question types in step six. Taking the e-commerce field as an example, the collected training data includes various questions and corresponding intent labels such as "I want a refund, how do I do it" (intent type: refund process consultation), "What discounts are there for this product" (intent type: product discount query), etc. The larger the amount of data and the richer the intent types covered, the stronger the generalization ability of the trained model, and the more accurately it can respond to problems in various actual scenarios; Divide the prepared training data into training set, validation set and test set in a certain ratio. The training set is used to train the model so that the model can learn the mapping relationship between text features and intent types; the validation set is used to evaluate the performance of the model during the training process, adjust the model parameters and prevent overfitting; the test set is used to finally evaluate the performance of the trained model on unseen data. During training, the user's identity background description and question text information are input into the model. The model extracts features, transforms and classifies the input data according to the calculation logic of the deep learning algorithm. By continuously adjusting the parameters of the model (such as the weights and biases of the neural network), the difference between the model's prediction results and the labeled intent types (such as cross entropy loss) is gradually reduced. This process will undergo multiple rounds of iterative training until the performance of the model on the validation set reaches the optimal level or no longer improves; After training is completed, the model is evaluated using the test set. The main evaluation indicators include accuracy, recall, F1 value, etc. Accuracy reflects the proportion of correct classification by the model; recall measures the proportion of a certain type of intent samples correctly identified by the model to all samples of that type; F1 value comprehensively considers accuracy and recall to more comprehensively evaluate model performance. If the model evaluation result is not ideal, the model can be further optimized. Optimization methods include adjusting the model structure (such as increasing or decreasing the number of network layers, changing the size of the convolution kernel, etc.), changing the deep learning algorithm, increasing the amount of training data, or enhancing the training data (such as randomly deleting or replacing words in the text), etc., to continuously improve the accuracy and stability of the intent classification model.

[0048] In one embodiment, specifically, in step 6, the preset multiple question types include factual, explanatory, comparative, step-guiding, suggestion, inference, hypothesis, opinion, emotion, confirmation, selection, rhetorical, humorous or sarcastic, speculative, moral, clarifying, self-reflective, situation simulation, time, and demand; Factual questions: These questions mainly ask about objective factual information, and the answers are certain. For example, "What is the altitude of Mount Everest?" "When was Apple founded?" The system needs to extract accurate factual data from the knowledge base to answer; Explanatory: Aims to help users understand a concept, phenomenon or principle. For example, “What is quantum mechanics?” or “Why is the sky blue?” When answering, the relevant knowledge needs to be explained in detail. Comparative: requires comparison of the characteristics and attributes of two or more objects. For example, "What is the difference between an iPhone and an Android phone?" "Which is more convenient, a high-speed train or an airplane?" The system needs to analyze and explain the similarities and differences of the objects of comparison based on the knowledge base; Step-by-step instructions: Seeking specific steps or processes to complete a task. Common examples include "How to make a cake?" and "How to install a computer operating system?" The answer needs to clearly list the steps; Suggestive: users want to get advice on a certain problem or situation. For example, "I want to lose weight, do you have any suggestions?" The system needs to give reasonable advice based on knowledge and experience. Reasoning: Draw conclusions based on given information through logical reasoning. For example, "If it rains today, the sports meeting will be postponed. Now that the sports meeting has been postponed, what can be concluded?" The answer requires the use of logical reasoning knowledge; Hypothetical: Questions are based on hypothetical situations. For example, "What would the world be like if humans could live forever?" These questions are relatively open-ended and require systematic imagination and analysis. Opinion type: Asking others about their views or opinions on something. For example, "What do you think of this movie?" "What impact do you think artificial intelligence will have on employment?" The answers are somewhat subjective; Emotional Subjective: Focuses on expressing the user's emotions, feelings or personal feelings. For example, "I am in a bad mood today, what should I do?" "I like this product very much, how about you?" The system needs to recognize emotions and respond accordingly; Confirmation: used to confirm whether a piece of information is correct. For example, "The Earth is a planet in the solar system, right?" "Will this meeting be held tomorrow morning?" The answer is usually "yes" or "no", and the basis is provided; Selection: Give users multiple options to choose from, or ask about the pros and cons of two or more options. For example, "Should I buy red or blue clothes?" "Which is better, a trip to the seaside or a trip to the mountains?" The system needs to compare and analyze the options; Rhetorical question: On the surface, it is a question, but the actual meaning is the opposite of the literal meaning, with an emphasis or questioning tone. For example, "Is this the right thing to do?" The system needs to understand the real intention behind the rhetorical question; Humorous or sarcastic: Use humorous or sarcastic expressions to make the language more interesting or critical. For example, "My wallet is so thin, what should I do?" When answering, you should consider the language style and give an appropriate response. Speculative: Make guesses and inferences about unknown situations based on existing information. Answers must be based on knowledge and experience. Moral: Questions involving morals and ethics. For example, “Is lying moral in this situation?” The answer should be analyzed based on moral principles and values. Clarification: To clarify vague or ambiguous information. For example, "I don't quite understand the solution you mentioned. Can you explain it more clearly?" The system needs to reorganize the information and express it clearly; Self-reflection: Users reflect on their own behaviors and thoughts and ask questions. For example, “I’ve been procrastinating recently, how can I change this?” The answer should provide methods for self-improvement and improvement; Situation simulation: Set a virtual situation and ask questions. For example, "If you are my secretary, please help me arrange tomorrow's schedule." The system should respond according to the situation requirements; Time type: Asking for information related to time. For example, "When will the sports meet be held?" "How long will this event last?" The answer needs to specify a time point or time period; Demand-based: Directly express a certain need. For example, "I need a recommendation for a book about history" or "I want to find a nearby restaurant." The system should provide services based on the needs.

[0049] Figure 4 is a functional module diagram of a large language model device according to an embodiment of the present application, such as Figure 4 As shown, a device for a large language model includes: a professional knowledge base construction module, a user information processing module, an industry knowledge retrieval module, an identity background description generation module, an intention classification module, a RAG knowledge retrieval module and an answer content generation module; A professional knowledge base construction module is configured to obtain and process information data of a specified industry to construct a professional knowledge base; A user information processing module is configured to obtain user question text information and user identity information, and use an embedding model to vectorize the question text information and user identity information to generate a vectorized result; An industry knowledge retrieval module is configured to retrieve industry knowledge information related to the industry knowledge in the professional knowledge base according to the vectorization result; An identity background description generation module is configured to input industry knowledge information and user identity information as prompt words into a large language model to generate a user identity background description; The intent classification module is configured to vectorize the user identity background description and the user question text information based on the intent classification model to generate an output result; The RAG knowledge retrieval module is configured to perform RAG knowledge retrieval in the professional knowledge base according to the user intention type, question text information, and user identity background description to obtain knowledge information related to the question text; The answer content generation module is configured to utilize the context understanding capability of the large language model, combine the user intent type and the knowledge information related to the question text, and generate the answer content.

[0050] In one embodiment, specifically, the device further includes a data cache module, and the data cache module is used to cache the retrieved knowledge information data and related search conditions after the industry knowledge retrieval module and the RAG knowledge retrieval module perform the search operation.

[0051] For other details about the technical solutions implemented by each module in the device of the above embodiment, please refer to the description of a method of a large language model in the above embodiment, which will not be repeated here.

[0052] It should be noted that each embodiment in this specification is described in a progressive manner, and each embodiment focuses on the differences from other embodiments, and the same or similar parts between the embodiments can be referred to each other. For the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0053] The basic principles of the present disclosure are described above in conjunction with specific embodiments. However, it should be noted that the advantages, strengths, effects, etc. mentioned in the present disclosure are only examples and not limitations, and it cannot be considered that these advantages, strengths, effects, etc. are required by each embodiment of the present disclosure. In addition, the specific details disclosed above are only for the purpose of illustration and ease of understanding, and are not limitations. The above details do not limit the present disclosure to the necessity of adopting the above specific details to be implemented.

[0054] In the present disclosure, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply that there is any such actual relationship or order between these entities or operations. The block diagrams of the devices, devices, equipment, and systems involved in the present disclosure are only illustrative examples and are not intended to require or imply that they must be connected, arranged, and configured in the manner shown in the block diagram. As will be appreciated by those skilled in the art, these devices, devices, equipment, and systems can be connected, arranged, and configured in any manner. Words such as "including", "comprising", "having", etc. are open words, referring to "including but not limited to", and can be used interchangeably with them. The words "or" and "and" used here refer to the words "and / or" and can be used interchangeably with them, unless the context clearly indicates otherwise. The words "such as" used here refer to the phrase "such as but not limited to", and can be used interchangeably with them.

[0055] Additionally, as used herein, "or" used in a list of items beginning with "at least one" indicates a separate list, so that, for example, a list of "at least one of A, B, or C" means A or B or C, or AB or AC or BC, or ABC (i.e., A and B and C). Furthermore, the word "exemplary" does not mean that the example described is preferred or better than other examples.

[0056] It should also be noted that in the system and method of the present disclosure, each component or each step can be decomposed and / or recombined. Such decomposition and / or recombination should be regarded as equivalent solutions of the present disclosure.

[0057] Various changes, substitutions, and modifications of the techniques described herein may be made without departing from the teachings defined by the appended claims. Furthermore, the scope of the claims of the present disclosure is not limited to the specific aspects of the processes, machines, manufactures, compositions of events, means, methods, and actions described above. Currently existing or later to be developed processes, machines, manufactures, compositions of events, means, methods, or actions that perform substantially the same functions or achieve substantially the same results as the corresponding aspects described herein may be utilized. Thus, the appended claims include such processes, machines, manufactures, compositions of events, means, methods, or actions within their scope.

[0058] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use the present disclosure. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other aspects without departing from the scope of the present disclosure. Therefore, the present disclosure is not intended to be limited to the aspects shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

[0059] The above description has been given for the purpose of illustration and description. In addition, this description is not intended to limit the embodiments of the present disclosure to the forms disclosed herein. Although multiple example aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, changes, additions and sub-combinations thereof.

Claims

1. A method for a large language model, characterized in that: The following steps are involved: Obtain and process data from designated industries to build a professional knowledge base; Obtain the user's question text information and user identity information, and use the embedding model to vectorize the question text information and user identity information to generate vectorized results; According to the vectorization results, the industry knowledge information related to the user's question text information is retrieved by calculating the cosine similarity with the knowledge block vector in the professional knowledge base; Input industry knowledge information and user identity information as prompt words into the large language model to generate a user identity background description; Based on the intent classification model, the user's identity background description and user question text information are vectorized to generate output results; Determine the intention type of the question from among the preset multiple question types based on the output results, and generate the user intention type; According to the user's intention type, question text information, and user identity background description, RAG knowledge retrieval is performed in the professional knowledge base to obtain knowledge information related to the question text; Based on the contextual understanding ability of the large language model, the answer content is generated according to the user's intention type and the knowledge information related to the question text.

2. The method of a large language model according to claim 1, characterized in that: The method for building a professional knowledge base comprises the following steps: According to the scope of the target industry, collect documents and forms related to the industry through multiple channels; Convert documents and table data into a unified text format and pre-process them to filter out professional knowledge data; Divide professional knowledge materials into blocks according to the logical structure of the document or the rows and columns of the table to generate professional knowledge blocks; Based on the embedding model, professional knowledge blocks are converted into vector representations to generate knowledge block vectors; The knowledge block vectors are stored in the professional knowledge base and associated with the corresponding metadata.

3. The method of a large language model according to claim 1, characterized in that: The calculation formula of the cosine similarity is: ； in, is the problem vector in the vectorized result, is the knowledge block vector in the professional knowledge base; Dot product of vectors The calculation formula is: ； Where n is the dimension of the vector, and They are and The value of the vector in the i-th dimension; The magnitude of a vector The calculation formula is: ； The magnitude of a vector The calculation formula is: 。 4. The method of a large language model according to claim 1, characterized in that: The method for obtaining knowledge information related to the question text comprises the following steps: Integrate the user's intention type, question text information, and user identity background description to form a comprehensive search condition text; According to the characteristics of the professional knowledge base and the retrieval requirements, select the retrieval strategy, search the comprehensive retrieval condition text, and obtain the knowledge information data; The retrieved knowledge information data is screened to obtain knowledge information related to the question text.

5. The method of a large language model according to claim 1, characterized in that: When the question text information and the user identity information are vectorized using the embedding model, the question text information and the user identity information are concatenated to form a fused text, and the fused text is input into the embedding model based on the Transformer architecture for vectorization.

6. The method of a large language model according to claim 1, characterized in that: The large language model is an open source or commercial large language model based on the Transformer architecture, including GPT-4 and Llama.

7. The method of a large language model according to claim 1, characterized in that: The intent classification model is constructed through training via a deep learning algorithm.

8. The method of a large language model according to claim 1, characterized in that: The preset multiple question types include factual, explanatory, comparative, step-by-step guidance, suggestion, reasoning, hypothesis, opinion, emotional and subjective, confirmation, choice, rhetorical, humorous or sarcastic, speculative, moral, clarification, self-reflection, situation simulation, time and demand.

9. A large language model device, applied to a large language model method according to any one of claims 1 to 7, characterized in that: The device comprises: a professional knowledge base construction module, a user information processing module, an industry knowledge retrieval module, an identity background description generation module, an intention classification module, a RAG knowledge retrieval module and an answer content generation module; The professional knowledge base construction module is configured to obtain and process the data of the specified industry to construct a professional knowledge base; The user information processing module is configured to obtain user question text information and user identity information, and use the embedding model to vectorize the question text information and the user identity information to generate a vectorization result; The industry knowledge retrieval module is configured to retrieve industry knowledge information related to the industry knowledge in the professional knowledge base according to the vectorization result; The identity background description generating module is configured to input the industry knowledge information and the user identity information as prompt words into the large language model to generate the user identity background description; The intent classification module is configured to vectorize the user identity background description and the user question text information based on the intent classification model to generate an output result; The RAG knowledge retrieval module is configured to perform RAG knowledge retrieval in the professional knowledge base according to the user intention type, question text information and user identity background description, and obtain knowledge information related to the question text; The answer content generation module is configured to utilize the context understanding capability of the large language model, combined with the user's intention type and question text-related knowledge information, to generate answer content.

10. The device for a large language model according to claim 9, characterized in that: The device also includes a data cache module, which is used to cache the retrieved knowledge information data and related retrieval conditions after the industry knowledge retrieval module and the RAG knowledge retrieval module perform the retrieval operation.

Images