Method, system and device for constructing enterprise question and answer large model based on ES and L0 large model and medium

Abstract

The invention discloses a method, a system, equipment and a medium for constructing an enterprise question and answer large model based on an ES and an L0 large model, belongs to the crossing field of artificial intelligence and natural language processing, and aims to solve the technical problem of how to construct an efficient and accurate enterprise question and answer large model by utilizing the search function of Elasticsearch and the deep understanding ability of the L0 large model. According to the technical scheme, the method comprises the following steps: data preparation and preprocessing: collecting documents, product description, common question answers and various data in an enterprise fed back by a user, carrying out preprocessing operations of cleaning, denoising, word segmentation and index establishment on the various collected data in the enterprise, and converting original data into a format suitable for model training and retrieval; the method comprises the following steps of: performing data indexing and efficient retrieval by utilizing Elasticsearch: creating a new epitome in the Elasticsearch, designing a reasonable mapping for each document, and defining a structure of the document in an index by the mapping; training and optimizing an L0 large model; and testing and deploying.

Description

Technical Field

[0001] The present invention relates to the intersection of artificial intelligence and natural language processing (NLP), and specifically to a method, system, device and medium for building an enterprise question-answering big model based on ES and L0 big models. Background Art

[0002] The method of building an enterprise question-answering big model based on the basic capabilities of ES (Elasticsearch) and the L0 big model. Its technical background mainly involves Elasticsearch's search technology, the deep learning capabilities of the L0 big model, and the needs and development trends of enterprise question-answering systems.

[0003] Elasticsearch (ES) is an open source, distributed, RESTful search engine built on Lucene. It is designed for cloud computing and can achieve real-time search. It is stable, reliable, fast, and easy to install and use. Its distributed architecture enables it to handle large-scale data and provide efficient search and analysis capabilities. These features of ES make it an ideal choice for enterprise data storage and retrieval.

[0004] The L0 large model represents the latest progress in the current field of deep learning. A large model refers to a technology that processes and analyzes massive amounts of data based on distributed computing, machine learning and other technologies. The L0 large model has acquired the ability to deeply understand and generate natural language by pre-training on a large-scale corpus. This ability enables the model to understand complex semantic relationships and perform precise semantic matching and reasoning.

[0005] As an important tool for information exchange between enterprises, customers and employees, the enterprise question-answering system is of great significance for improving customer satisfaction and employee work efficiency. With the continuous growth of enterprise data and the increasing requirements of users for information acquisition efficiency, traditional question-answering systems can no longer meet the needs of enterprises.

[0006] Therefore, how to use the search function of Elasticsearch and the deep understanding ability of the L0 large model to build an efficient and accurate enterprise question-answering large model is a technical problem that needs to be solved urgently. Summary of the invention

[0007] The technical task of the present invention is to provide a method, system, device and medium for building an enterprise question and answer big model based on ES and L0 big models, so as to solve the problem of how to use the search function of Elasticsearch and the deep understanding ability of the L0 big model to build an efficient and accurate enterprise question and answer big model.

[0008] The technical task of the present invention is achieved in the following way: a method for building an enterprise question-answering big model based on ES and L0 big models, the method is as follows:

[0009] Data preparation and preprocessing: Collect various internal data such as documents, product descriptions, FAQs, and user feedback, and perform preprocessing operations such as cleaning, denoising, word segmentation, and indexing on the collected internal data to convert the raw data into a format suitable for model training and retrieval;

[0010] Use Elasticsearch (ES) for data indexing and efficient retrieval: Create a new microcosm in Elasticsearch and design a reasonable mapping for each document. The mapping defines the structure of the document in the index, including which fields are strings, numbers, and dates, and whether they should be indexed or stored.

[0011] L0 large model training and optimization: Combine L0 large model for deep language understanding and generation;

[0012] Model integration and question-answering system construction: Integrate ES's search results with the output of the L0 large model to build an enterprise question-answering large model;

[0013] Testing and deployment: Conduct functional testing, performance testing, and stability testing on the constructed enterprise question-and-answer big model to ensure stability and availability; design test plans and effect evaluation strategies to conduct effect evaluation, and deploy the enterprise question-and-answer big model that passes the test to the enterprise production environment to provide users with real-time and efficient question-and-answer services.

[0014] As a preferred method, Elasticsearch is used for data indexing and efficient retrieval as follows:

[0015] Based on the various types of data collected within the enterprise, the content of the materials is saved in the Elasticsearch index using word vectors;

[0016] Store each document content (original text or processed text) and the corresponding document vector as a document in Elasticsearch;

[0017] The document vector is stored as a special field (document_vector) and is stored and retrieved using vector search, reverse indexing, and N-Gram technology;

[0018] When a query is initiated, the search engine converts the query into an embedding of digital representations of the data and related context and stores it in a vector; it then uses the kNN (k nearest neighbor) algorithm to match the vectors of existing documents with the query vector and rank the search results based on concept relevance.

[0019] As a preferred method, the L0 large model training and optimization are as follows:

[0020] Pre-training: Use the corpus of the large model to pre-train the L0 large model, and use self-supervised learning to enable the large model to have basic language understanding and generation capabilities;

[0021] Domain adaptation: To meet the specific needs of the enterprise field, the L0 large model is trained for domain adaptation, and the performance of the model in enterprise question-answering tasks is improved by introducing data and knowledge from the enterprise field. At the same time, optimizing Prompt to improve the performance of the large model in enterprise question-answering is a complex but effective strategy. Enterprise question-answering usually involves professional knowledge, business logic, and terminology in specific fields, so optimizing Prompt is crucial to ensure that the large model can accurately understand and answer relevant questions.

[0022] Preferably, LoRA fine-tuning is used for pre-training. LoRA fine-tuning is an efficient fine-tuning method that greatly reduces the number of parameters of the L0 model by applying a decomposed low-rank matrix on the weight matrix of the LLM (large language model), thereby reducing computational complexity and memory requirements.

[0023] In LoRA fine-tuning, the weight matrix W of each layer is split into the product of the base matrix B and two low-rank matrices E and F, that is, W≈B+EF^T; during the fine-tuning process, only E and F need to be updated, while the base model B remains unchanged, which greatly reduces storage and computing costs.

[0024] Preferably, the specific steps of LoRA fine-tuning are as follows:

[0025] ① Freeze the weights of the pre-trained model: During fine-tuning, the weights of the pre-trained L0 model remain unchanged;

[0026] ②Inject trainable layers in each Transformer block: Fine-tune the L0 model by adding trainable layers in each Transformer block;

[0027] ③Use rank-decomposition matrix: During fine-tuning, use the rank-decomposition matrix instead of the original weight matrix to reduce the number of parameters that need to be trained;

[0028] ④ Focus on the Transformer attention block: During the fine-tuning process, focus on the Transformer attention block to improve the quality and efficiency of fine-tuning;

[0029] Among them, when choosing a suitable LoRA fine-tuning method, the fine-tuning objectives, data sets and evaluation indicators should be determined according to the specific task; at the same time, the size of the bypass matrix and the hyperparameters of the learning rate need to be adjusted to optimize the fine-tuning effect; finally, the L0 model needs to be evaluated and the best model should be selected for deployment and application.

[0030] Preferably, the domain adaptation is as follows:

[0031] Use business-related terms: Based on the industry and business field of the enterprise, use relevant professional terms and concepts to build prompts, which helps improve the big model's ability to understand specific fields and enable it to answer business-related questions more accurately;

[0032] Introducing examples and cases: By providing specific examples and cases, we can help the big model better learn and understand the structure and pattern of the problem, so that the model can quickly give accurate answers when faced with similar problems.

[0033] Optimize the length and structure of the prompt: A prompt that is too long may contain redundant information, making it difficult for the model to grasp the key points; while a prompt that is too short may lack sufficient information for the model to understand. Therefore, it is necessary to adjust the length and structure of the prompt according to the actual situation to make the length and structure of the prompt concise and clear;

[0034] Iteration and optimization: By continuously iterating and optimizing Prompt, its effect can be gradually improved. Iteration and optimization include adjusting the design of Prompt according to the feedback of the model and users, and trying different Prompt styles and expressions.

[0035] As a preferred method, the model integration and question-answering system construction are as follows:

[0036] API interface development: Develop API interfaces for Elasticsearch and L0 big model to realize data interaction and function calling between the two. Combine the content retrieved from ElasticSearch with the big model prompt project, use the trained and optimized big model to organize it into logical answers and return them to the inquirer.

[0037] Question-answering logic design: Design the core logic of the question-answering system, which includes parsing user questions, identifying question types, and selecting search strategies. Based on industry scenarios, use the classification capabilities of large models to conduct preliminary analysis and pre-classification of user questions, further narrow the query scope, and find content with more similar semantics.

[0038] Integration: Integrate the search function of Elasticsearch with the deep understanding ability of the L0 large model; when a user asks a question, first use Elasticsearch to quickly search, then use the L0 large model to deeply understand and process the search results, remove unreasonable answers, and finally give correct, accurate and useful answers.

[0039] A system for constructing an enterprise question-answering big model based on ES and L0 big models, the system is used to implement the method for constructing an enterprise question-answering big model based on ES and L0 big models as described above; the system comprises:

[0040] The data preparation and preprocessing module is used to collect various types of internal enterprise data such as documents, product descriptions, FAQs, and user feedback, and to perform preprocessing operations such as cleaning, denoising, word segmentation, and indexing on the collected internal enterprise data, converting the raw data into a format suitable for model training and retrieval;

[0041] The indexing and retrieval module is used to use Elasticsearch (ES) for data indexing and efficient retrieval. Specifically, it creates a new microcosm in Elasticsearch and designs a reasonable mapping for each document. The mapping defines the structure of the document in the index. The structure of the document in the index includes which fields are strings, numbers, and dates, and whether they should be indexed or stored.

[0042] L0 large model training and optimization module, used to combine L0 large model for deep language understanding and generation;

[0043] Model integration and question-answering system construction module, which is used to integrate the ES search results with the output of the L0 large model to build an enterprise question-answering large model;

[0044] The testing and deployment module is used to perform functional testing, performance testing, and stability testing on the constructed enterprise question-and-answer big model to ensure stability and availability; and to design test plans and effect evaluation strategies for effect evaluation. At the same time, the enterprise question-and-answer big model that passes the test is deployed to the enterprise production environment to provide users with real-time and efficient question-and-answer services.

[0045] An electronic device comprising: a memory and at least one processor;

[0046] Wherein, the memory stores computer-executable instructions;

[0047] The at least one processor executes the computer-executable instructions stored in the memory, so that the at least one processor executes the method of building an enterprise question and answer big model based on the ES and L0 big models as described above.

[0048] A computer-readable storage medium having computer-executable instructions stored therein, which, when executed by a processor, implements the method of constructing an enterprise question-and-answer big model based on ES and L0 big models as described above.

[0049] The method, system, device and medium for constructing an enterprise question-answering big model based on the ES and L0 big models of the present invention have the following advantages:

[0050] (i) The present invention improves the efficiency and accuracy of information retrieval: by integrating the efficient search function of Elasticsearch and the deep understanding ability of the L0 large model, the present invention can quickly locate information related to the user's question and perform accurate matching and analysis; compared with the traditional information retrieval method based on keyword matching, the present invention can more accurately understand the user's intention, thereby providing information that better meets the user's needs;

[0051] (ii) The present invention enhances the intelligence of the enterprise question-answering system: the L0 large model has powerful language understanding and generation capabilities, enabling the enterprise question-answering system to understand and answer users' questions like humans, not only providing a more natural and smooth interactive experience, but also being able to handle complex questions and semantic relationships, further improving the intelligence and usability of the system;

[0052] (III) The present invention reduces the operating costs of enterprises: by building an efficient and accurate enterprise question-and-answer system, enterprises can reduce their reliance on manual customer service and reduce labor costs; at the same time, automated and intelligent processing can also reduce errors and repetitive work and improve work efficiency;

[0053] (IV) The present invention improves user experience and satisfaction: By providing fast, accurate and intelligent question-and-answer services, the method can improve user experience and satisfaction; users can interact with enterprises through natural language to obtain the required information and help, thereby enhancing their trust and loyalty to the enterprise;

[0054] (V) The present invention promotes knowledge sharing and inheritance: it can organize and summarize the knowledge and experience of the enterprise to form a searchable and shareable knowledge base, which not only facilitates employees to acquire and learn the knowledge of the enterprise, but also promotes the sharing and inheritance of knowledge within the enterprise, thereby enhancing the overall competitiveness of the enterprise;

[0055] (VI) The method of constructing an enterprise question-answering model based on the basic capabilities of the ES and L0 large models has multiple beneficial effects, such as improving information retrieval efficiency and accuracy, enhancing the intelligence of the enterprise question-answering system, reducing enterprise operating costs, improving user experience and satisfaction, and promoting knowledge sharing and inheritance. It helps to improve the service quality and efficiency of enterprises and enhance their competitiveness and market position;

[0056] (VII) Compared with traditional knowledge base solutions, the present invention focuses more on accuracy in professional situations. It not only retains the original semantic information of the material, but also can perform accurate matching, thus improving retrieval efficiency and accuracy;

[0057] (VIII) The present invention aims to improve the efficiency and accuracy of enterprise information retrieval. Traditional enterprise information retrieval methods are often based on keyword matching and cannot effectively understand and process complex semantic information, resulting in inaccurate retrieval results and low efficiency. By combining the search function of Elasticsearch and the deep understanding ability of the L0 large model, the present invention can achieve intelligent analysis and precise matching of enterprise data, improve the efficiency and accuracy of retrieval, and thus meet the needs of enterprises for efficient and accurate information acquisition;

[0058] (IX) The present invention is committed to building a widely applicable enterprise question-answering system. Different enterprises have different data types and business needs, so a question-answering system that can flexibly adapt to various scenarios is needed; at the same time, the method based on the ES and L0 large models can process various types of data, including text, pictures, videos, etc., and can adapt to different business needs by adjusting model parameters and optimizing algorithms. It has wide applicability and can meet the needs of various types of enterprises;

[0059] (10) The present invention also aims to improve the service quality and customer satisfaction of enterprises. By building an efficient and accurate enterprise question-and-answer system, enterprises can quickly respond to users' questions and provide accurate and useful answers, which can not only improve the work efficiency and satisfaction of employees, but also improve the experience and loyalty of customers. At the same time, by continuously optimizing the model and improving the performance of the system, enterprises can also continuously improve service quality and win more customers' trust and support.

[0060] (XI) Based on the powerful search capabilities of Elasticsearch, the deep learning capabilities of the L0 large model, and the needs of enterprises for efficient and accurate question-answering systems, the present invention combines the latest search technology and deep learning technology to provide enterprises with a new and efficient information retrieval and question-answering solution. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] The present invention is further described below in conjunction with the accompanying drawings.

[0062] Attached Figure 1 A flowchart of the method for building an enterprise question-answering big model based on the ES and L0 big models. DETAILED DESCRIPTION

[0063] The method, system, device and medium for constructing an enterprise question and answer big model based on the ES and L0 big models of the present invention are described in detail below with reference to the drawings and specific embodiments of the specification.

[0064] Embodiment 1:

[0065] As attached Figure 1 As shown, this embodiment provides a method for building an enterprise question-answering big model based on the ES and L0 big models. The method is specifically as follows:

[0066] S1. Data preparation and preprocessing: Collect various internal data of the enterprise, such as documents, product descriptions, FAQs and user feedback, and perform preprocessing operations such as cleaning, denoising, word segmentation and indexing on the collected internal data, so as to convert the raw data into a format suitable for model training and retrieval;

[0067] S2. Use Elasticsearch (ES) for data indexing and efficient retrieval: Create a new microcosm in Elasticsearch and design a reasonable mapping for each document. The mapping defines the structure of the document in the index. The structure of the document in the index includes which fields are strings, numbers, and dates, and whether they should be indexed or stored.

[0068] S3, L0 large model training and optimization: Combine L0 large model for deep language understanding and generation;

[0069] S4, model integration and question-answering system construction: Integrate the ES search results with the output of the L0 large model to build an enterprise question-answering large model;

[0070] S5. Testing and deployment: Conduct functional testing, performance testing, and stability testing on the constructed enterprise question-and-answer big model to ensure stability and availability; design test plans and effect evaluation strategies to conduct effect evaluation, and deploy the enterprise question-and-answer big model that passes the test to the enterprise production environment to provide users with real-time and efficient question-and-answer services.

[0071] The specific details of using Elasticsearch to index and efficiently retrieve data in step S2 of this embodiment are as follows:

[0072] S201. Based on the collected data from the enterprise, the content of the material is saved in the Elasticsearch index in the form of word vectors.

[0073] S202, storing each document content (original text or processed text) and the corresponding document vector as a document in Elasticsearch;

[0074] S203, the document vector is stored as a special field (document_vector), and is stored and retrieved using vector search, reverse indexing and N-Gram technology;

[0075] S204. When a query is initiated, the search engine converts the query into an embedding of digital representations of data and related context and stores it in a vector; then uses the kNN (k nearest neighbor) algorithm to match the vectors of existing documents with the query vector and rank the search results according to concept relevance.

[0076] The L0 large model training and optimization in step S3 of this embodiment are specifically as follows:

[0077] S301, pre-training: Use the corpus of the large model to pre-train the L0 large model, and use self-supervised learning to enable the large model to have basic language understanding and generation capabilities;

[0078] S302. Domain Adaptation: Based on the specific needs of the enterprise field, the L0 large model is trained for domain adaptation, and the performance of the model in enterprise question-answering tasks is improved by introducing data and knowledge from the enterprise field. At the same time, optimizing Prompt to improve the performance of the large model in enterprise question-answering is a complex but effective strategy. Enterprise question-answering usually involves professional knowledge, business logic, and terminology in specific fields, so optimizing Prompt is crucial to ensure that the large model can accurately understand and answer relevant questions.

[0079] The pre-training in step S301 of this embodiment adopts LoRA fine-tuning, which is an efficient fine-tuning method. By applying a decomposed low-rank matrix to the weight matrix of the LLM (large language model), the number of parameters of the L0 model is greatly reduced, thereby reducing the computational complexity and memory requirements;

[0080] In LoRA fine-tuning, the weight matrix W of each layer is split into the product of the base matrix B and two low-rank matrices E and F, that is, W≈B+EF^T; during the fine-tuning process, only E and F need to be updated, while the base model B remains unchanged, which greatly reduces storage and computing costs.

[0081] The specific steps of LoRA fine-tuning in this embodiment are as follows:

[0082] ① Freeze the weights of the pre-trained model: During fine-tuning, the weights of the pre-trained L0 model remain unchanged;

[0083] ②Inject trainable layers in each Transformer block: Fine-tune the L0 model by adding trainable layers in each Transformer block;

[0084] ③Use rank-decomposition matrix: During fine-tuning, use the rank-decomposition matrix instead of the original weight matrix to reduce the number of parameters that need to be trained;

[0085] ④ Focus on the Transformer attention block: During the fine-tuning process, focus on the Transformer attention block to improve the quality and efficiency of fine-tuning;

[0086] Among them, when choosing a suitable LoRA fine-tuning method, the fine-tuning objectives, data sets and evaluation indicators should be determined according to the specific task; at the same time, the size of the bypass matrix and the hyperparameters of the learning rate need to be adjusted to optimize the fine-tuning effect; finally, the L0 model needs to be evaluated and the best model should be selected for deployment and application.

[0087] The domain adaptation in step S302 of this embodiment is specifically as follows:

[0088] S30201. Use business-related terms: Use relevant professional terms and concepts to build prompts based on the industry and business field of the enterprise, which helps improve the big model's ability to understand specific fields and enable it to answer business-related questions more accurately.

[0089] S30202. Introduce examples and cases: By providing specific examples and cases, we can help the big model better learn and understand the structure and pattern of the problem, so that the model can quickly give accurate answers when faced with similar problems.

[0090] S30203. Optimize the length and structure of prompts: A prompt that is too long may contain redundant information, making it difficult for the model to grasp the key points; while a prompt that is too short may lack sufficient information for the model to understand. Therefore, it is necessary to adjust the length and structure of prompts according to the actual situation to make them concise and clear.

[0091] S30204, Iteration and optimization: gradually improve the effect of Prompt by continuously iterating and optimizing it; iteration and optimization include adjusting the design of Prompt according to the feedback of the model and users, and trying different Prompt styles and expressions.

[0092] The model integration and question-answering system construction in step S4 of this embodiment are specifically as follows:

[0093] S401, API interface development: Develop API interfaces for Elasticsearch and L0 big model to realize data interaction and function calling between the two. Combine the content retrieved from ElasticSearch with the big model prompt project, use the trained and optimized big model to organize it into logical answers and return them to the inquirer.

[0094] S402, Question and Answer Logic Design: Design the core logic of the question and answer system, which includes parsing user questions, identifying question types, and selecting search strategies. Based on industry scenarios, use the classification capabilities of the big model to conduct preliminary analysis and pre-classification of user questions, further narrow the query scope, and find content with more similar semantics.

[0095] S403, Integration: Integrate the search function of Elasticsearch with the deep understanding ability of the L0 large model; when a user asks a question, first perform a quick search through Elasticsearch, then use the L0 large model to deeply understand and process the search results, remove unreasonable answers, and finally give correct, accurate and useful answers.

[0096] Embodiment 2:

[0097] This embodiment provides a system for constructing an enterprise question-answering big model based on the ES and L0 big models. The system is used to implement the method for constructing an enterprise question-answering big model based on the ES and L0 big models in Embodiment 1; the system includes:

[0098] The data preparation and preprocessing module is used to collect various types of internal enterprise data such as documents, product descriptions, FAQs, and user feedback, and to perform preprocessing operations such as cleaning, denoising, word segmentation, and indexing on the collected internal enterprise data, converting the raw data into a format suitable for model training and retrieval;

[0099] The indexing and retrieval module is used to use Elasticsearch (ES) for data indexing and efficient retrieval. Specifically, it creates a new microcosm in Elasticsearch and designs a reasonable mapping for each document. The mapping defines the structure of the document in the index. The structure of the document in the index includes which fields are strings, numbers, and dates, and whether they should be indexed or stored.

[0100] L0 large model training and optimization module, used to combine L0 large model for deep language understanding and generation;

[0101] Model integration and question-answering system construction module, which is used to integrate the ES search results with the output of the L0 large model to build an enterprise question-answering large model;

[0102] The testing and deployment module is used to perform functional testing, performance testing, and stability testing on the constructed enterprise question-and-answer big model to ensure stability and availability; and to design test plans and effect evaluation strategies for effect evaluation. At the same time, the enterprise question-and-answer big model that passes the test is deployed to the enterprise production environment to provide users with real-time and efficient question-and-answer services.

[0103] Embodiment 3:

[0104] This embodiment also provides an electronic device, including: a memory and at least one processor;

[0105] Wherein, the memory stores computer-executable instructions;

[0106] The at least one processor executes the computer-executable instructions stored in the memory, so that the at least one processor executes the method for building an enterprise question and answer big model based on the ES and L0 big models as described in any one of the present inventions.

[0107] The processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The processor may be a microprocessor or any conventional processor, etc.

[0108] The memory can be used to store computer programs and / or modules. The processor realizes various functions of the electronic device by running or executing the computer programs and / or modules stored in the memory, and calling the data stored in the memory. The memory can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, at least one application required for a function, etc.; the data storage area can store data created according to the use of the terminal, etc. In addition, the memory can also include a high-speed random access memory, and can also include a non-volatile memory, such as a hard disk, a memory, a plug-in hard disk, a smart memory card (SMC), a secure digital (SD) card, a flash memory card, at least one disk storage period, a flash memory device, or other volatile solid-state storage devices.

[0109] Embodiment 4:

[0110] This embodiment also provides a computer-readable storage medium, which stores a plurality of instructions, which are loaded by a processor to enable the processor to execute the method of building an enterprise question-answering model based on the ES and L0 large models in any embodiment of the present invention. Specifically, a system or device equipped with a storage medium can be provided, on which a software program code that implements the functions of any of the above embodiments is stored, and a computer (or CPU or MPU) of the system or device reads and executes the program code stored in the storage medium.

[0111] In this case, the program code itself read from the storage medium can realize the function of any one of the above-mentioned embodiments, and thus the program code and the storage medium storing the program code constitute a part of the present invention.

[0112] The storage medium embodiments for providing the program code include a floppy disk, a hard disk, a magneto-optical disk, an optical disk (such as CD-ROM, CD-R, CD-RW, DVD-ROM, DVD-RAM, DVD-RW, DVD+RW), a magnetic tape, a non-volatile memory card, and a ROM. Alternatively, the program code can be downloaded from a server computer by a communication network.

[0113] In addition, it should be clear that the functions of any of the above embodiments can be implemented not only by executing the program code read by the computer, but also by enabling an operating system operating on the computer to complete part or all of the actual operations based on instructions from the program code.

[0114] In addition, it can be understood that the program code read from the storage medium is written to a memory provided in an expansion board inserted into the computer or written to a memory provided in an expansion unit connected to the computer, and then based on the instructions of the program code, a CPU installed on the expansion board or the expansion unit is enabled to perform part or all of the actual operations, thereby realizing the functions of any of the above-mentioned embodiments.

[0115] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for constructing an enterprise question-answering model based on ES and L0 large models, characterized in that: The method is as follows: Data preparation and preprocessing: Collect various internal data such as documents, product descriptions, FAQs, and user feedback, and perform preprocessing operations such as cleaning, denoising, word segmentation, and indexing on the collected internal data to convert the raw data into a format suitable for model training and retrieval; Use Elasticsearch for data indexing and efficient retrieval: Create a new microcosm in Elasticsearch and design a reasonable mapping for each document. The mapping defines the structure of the documents in the index; the structure of the documents in the index includes which fields are strings, numbers, and dates, and whether they should be indexed or stored; L0 large model training and optimization: Combine L0 large model for deep language understanding and generation; Model integration and question-answering system construction: Integrate ES's search results with the output of the L0 large model to build an enterprise question-answering large model; Testing and deployment: Conduct functional testing, performance testing, and stability testing on the constructed enterprise question-and-answer big model to ensure stability and availability; design test plans and effect evaluation strategies to conduct effect evaluation, and deploy the enterprise question-and-answer big model that passes the test to the enterprise production environment to provide users with real-time and efficient question-and-answer services.

2. The method for constructing an enterprise question-answering big model based on ES and L0 big models according to claim 1 is characterized in that: The details of using Elasticsearch for data indexing and efficient retrieval are as follows: Based on the various types of data collected within the enterprise, the content of the materials is saved in the Elasticsearch index using word vectors; Store each document content and the corresponding document vector as a document in Elasticsearch; Document vectors are stored as a special field and stored and retrieved using vector search, reverse indexing, and N-Gram technology; When a query is initiated, the search engine converts the query into an embedding of digital representations of the data and related context and stores it in a vector; it then uses the kNN algorithm to match the vectors of existing documents with the query vector and rank the search results based on concept relevance.

3. The method for constructing an enterprise question-answering big model based on ES and L0 big models according to claim 1 is characterized in that: The specific L0 large model training and optimization are as follows: Pre-training: Use the corpus of the large model to pre-train the L0 large model, and use self-supervised learning to enable the large model to have basic language understanding and generation capabilities; Domain adaptation: To meet the specific needs of the enterprise field, the L0 large model is trained for domain adaptation, and the performance of the model in enterprise question-answering tasks is improved by introducing data and knowledge from the enterprise field.

4. The method for constructing an enterprise question-answering big model based on ES and L0 big models according to claim 3 is characterized in that: Pre-training uses LoRA fine-tuning, which is an efficient fine-tuning method that applies a decomposed low-rank matrix to the weight matrix of LLM; In LoRA fine-tuning, the weight matrix W of each layer is split into the product of the base matrix B and two low-rank matrices E and F, that is, W≈B+EF^T; during the fine-tuning process, only E and F need to be updated, while the base model B remains unchanged, which greatly reduces storage and computing costs.

5. The method for constructing an enterprise question-answering big model based on ES and L0 big models according to claim 4 is characterized in that: The specific steps of LoRA fine-tuning are as follows: ① Freeze the weights of the pre-trained model: During fine-tuning, the weights of the pre-trained L0 model remain unchanged; ②Inject trainable layers in each Transformer block: Fine-tune the L0 model by adding trainable layers in each Transformer block; ③Use rank-decomposition matrix: During fine-tuning, use the rank-decomposition matrix to replace the original weight matrix; ④ Focus on the Transformer attention block: During the fine-tuning process, focus on the Transformer attention block to improve the quality and efficiency of fine-tuning; Among them, when choosing a suitable LoRA fine-tuning method, the fine-tuning objectives, data sets and evaluation indicators should be determined according to the specific task; at the same time, the size of the bypass matrix and the hyperparameters of the learning rate need to be adjusted to optimize the fine-tuning effect; finally, the L0 model needs to be evaluated and the best model should be selected for deployment and application.

6. The method for constructing an enterprise question-answering big model based on ES and L0 big models according to claim 3 is characterized in that: The specific field adaptation is as follows: Use business-related terms: Use relevant professional terms and concepts to build prompts based on the industry and business field of the company; Introduce examples and cases: Help the big model better learn and understand the structure and pattern of the problem by providing specific examples and cases; Optimize the length and structure of the prompt: adjust the length and structure of the prompt according to the actual situation to make it concise and clear; Iteration and optimization: By continuously iterating and optimizing Prompt, its effect can be gradually improved. Iteration and optimization include adjusting the design of Prompt according to the feedback of the model and users, and trying different Prompt styles and expressions.

7. The method for constructing an enterprise question-answering big model based on ES and L0 big models according to claim 1 is characterized in that: The model integration and question-answering system construction are as follows: API interface development: Develop API interfaces for Elasticsearch and L0 big model to realize data interaction and function calling between the two. Combine the content retrieved from ElasticSearch with the big model prompt project, use the trained and optimized big model to organize it into logical answers and return them to the inquirer. Question-answering logic design: Design the core logic of the question-answering system, which includes parsing user questions, identifying question types, and selecting search strategies; Based on industry scenarios, we use the classification capabilities of large models to conduct preliminary analysis and pre-classification of questions raised by users, further narrow the scope of queries, and find content with more similar semantics. Integration: Integrate the search function of Elasticsearch with the deep understanding ability of the L0 large model; when a user asks a question, first use Elasticsearch to quickly search, then use the L0 large model to deeply understand and process the search results, remove unreasonable answers, and finally give correct, accurate and useful answers.

8. A system for building an enterprise question-answering model based on ES and L0 big models, characterized in that: The system is used to implement the method for building an enterprise question-answering big model based on the ES and L0 big models as described in any one of claims 1 to 6; the system comprises: The data preparation and preprocessing module is used to collect various types of internal enterprise data such as documents, product descriptions, FAQs, and user feedback, and to perform preprocessing operations such as cleaning, denoising, word segmentation, and indexing on the collected internal enterprise data, converting the raw data into a format suitable for model training and retrieval; The indexing and retrieval module is used to use Elasticsearch for data indexing and efficient retrieval. Specifically, it creates a new microcosm in Elasticsearch and designs a reasonable mapping for each document. The mapping defines the structure of the document in the index. The structure of the document in the index includes which fields are strings, numbers, and dates, and whether they should be indexed or stored. L0 large model training and optimization module, used to combine L0 large model for deep language understanding and generation; Model integration and question-answering system construction module, which is used to integrate the ES search results with the output of the L0 large model to build an enterprise question-answering large model; The testing and deployment module is used to perform functional testing, performance testing, and stability testing on the constructed enterprise question-and-answer big model to ensure stability and availability; and to design test plans and effect evaluation strategies for effect evaluation. At the same time, the enterprise question-and-answer big model that passes the test is deployed to the enterprise production environment to provide users with real-time and efficient question-and-answer services.

9. An electronic device, characterized in that: include: memory and at least one processor; Wherein, the memory stores computer-executable instructions; The at least one processor executes the computer-executable instructions stored in the memory, so that the at least one processor executes the method for building an enterprise question and answer big model based on the ES and L0 big models as described in any one of claims 1 to 6.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer execution instructions. When the processor executes the computer execution, the method for building an enterprise question and answer big model based on the ES and L0 big models as described in any one of claims 1 to 6 is implemented.

Images