A local construction method and device for a vertical large model in the field of environmental monitoring and evaluation

Abstract

The present invention discloses a local construction method and device of a vertical large model in the field of environmental monitoring and evaluation, belonging to the field of environmental monitoring and evaluation, large language model and retrieval enhancement technology. The method selects a general large model suitable for the field of environmental monitoring and supports local deployment, and uses the special knowledge data in the field of environmental monitoring and evaluation to fine-tune the general large model; constructs environmental monitoring and evaluation knowledge base documents according to industry classification, and performs text segmentation processing on the environmental monitoring and evaluation knowledge base documents to obtain text fragments; constructs a vector database suitable for the field of environmental monitoring and evaluation, and converts knowledge fragments into real-valued vectors and stores them in the vector database; adopts retrieval enhancement generation technology to pre-retrieve user input and context information, and generate knowledge fragments for assisting the reasoning of the general large model. The present invention solves the problems of low efficiency, easy error, and difficulty in meeting the growing intelligent demand for environmental protection work in traditional technologies.

Description

Technical Field

[0001] The present invention belongs to the field of environmental monitoring and evaluation, large language models and retrieval enhancement technology, and specifically relates to a local construction method and device for a vertical large model in the field of environmental monitoring and evaluation. Background Art

[0002] At present, staff members face many challenges in environmental monitoring and evaluation. On the one hand, environmental pollution data are complicated. Depending on the type of project, pollutant emissions are in various forms, and pollutant emission standards are constantly adjusted. This requires staff members to have high professional qualities and be able to verify a large amount of environmental data according to specific standards. For example, an environmental monitoring and evaluation impact report often contains hundreds of pages of content, including tens of thousands of environmental data. Manual verification work is not only a huge workload, but also prone to errors, resulting in low work efficiency and affecting the project approval timeliness and implementation progress.

[0003] On the other hand, with the development of artificial intelligence technology, technologies such as deep learning, neural networks, and large models are constantly iterating. However, the existing environmental monitoring and evaluation work model has failed to make full use of these advanced technologies, and the degree of intelligence in data processing and analysis is low. Although some enterprises and institutions have realized the electronicization and structured storage of data, there are still deficiencies in the use of large models for efficient monitoring and evaluation, which makes it difficult to meet the growing intelligent needs of environmental protection work. Therefore, it is of great practical significance to develop a local construction method and device for vertical large models in the field of environmental monitoring and evaluation. Summary of the invention

[0004] To this end, the present invention provides a local construction method and device for a vertical large model in the field of environmental monitoring and evaluation, so as to solve the problems of low efficiency, easy errors and difficulty in meeting the growing intelligent needs of environmental protection work in traditional technologies.

[0005] In order to achieve the above object, the present invention provides the following technical solution: a local construction method of a vertical large model in the field of environmental monitoring and evaluation, comprising: Model selection and adjustment: Select a general large model that is suitable for the field of environmental monitoring and supports local deployment, and use specialized knowledge data in the field of environmental monitoring and evaluation to fine-tune the general large model; Hierarchical construction of the knowledge base: construct environmental monitoring and evaluation knowledge base documents by industry classification, and perform text segmentation processing on the environmental monitoring and evaluation knowledge base documents to obtain text fragments; Vector database construction: Build a vector database suitable for the field of environmental monitoring and evaluation, convert knowledge fragments into real-valued vectors and store them in the vector database; Reasoning assistance: Retrieval-enhanced generation technology is used to pre-retrieve user input and context information to generate knowledge fragments to assist general large model reasoning.

[0006] As a preferred solution for local construction of vertical large models in the field of environmental monitoring and evaluation, in the process of model selection and adjustment: Let x be a local input with a dimension of d; the parameter W of the original weight of the setting part of the general large model is a d×d high-dimensional space; Introduce two low-dimensional matrix spaces A and B with dimension r, defined as update matrix A and update matrix B; Update matrix A and update matrix B first reduce the input d-dimensional data to r dimensions, where r is the rank of the matrix, from d×d to d×r+r×d; The update matrix A and update matrix B map the r-dimensional parameter matrix back to the d-dimensional matrix to keep consistent with the original matrix dimension of the base model. The original weight matrix of the base model remains frozen during training, and only the update matrix is ​​trained to generate a fine-tuning weight file.

[0007] As a preferred solution for local construction of vertical large models in the field of environmental monitoring and evaluation, the model selection and adjustment process also includes: The basic model weights and fine-tuning weights are loaded separately to verify the effect of the general large model, and the basic weight file and the fine-tuning weight file of the specified industry are merged separately to form a vertical large model in the environmental protection field for reasoning and deployment of different industry types.

[0008] As a preferred solution for the local construction method of the vertical large model in the field of environmental monitoring and evaluation, during the hierarchical construction of the knowledge base, the step of performing text segmentation processing on the environmental monitoring and evaluation knowledge base document includes: First, recursively split the document into paragraphs or titles, set the slice length L and the sliding window size M; for paragraphs longer than L, split them modulo L-2M; For titles and paragraphs whose length is less than L, they are merged and filled based on semantics and contextual relationships, so that the size of the first and last fragments is fixed to LM. The first M characters of the next fragment are added to the end of the first fragment, the last M characters of the previous fragment are added to the beginning of the last fragment, and M characters of the previous and next fragments are added to the beginning and end of the middle fragments respectively, forming text fragments of a fixed size of L.

[0009] As a preferred solution for local construction of vertical large models in the field of environmental monitoring and evaluation, during the construction of the vector database: The generated text segments are vectorized through an embedding model, the data are converted into numerical representations, and stored in a vector database; Evaluate the generated vector database in terms of relevance, grammaticality, and conciseness of the search results; the relevance is the degree of relevance of the generated text to the input and context, the grammaticality is whether the generated grammar is correct, and the conciseness is the degree of conciseness of the generated text; The vector database is iteratively updated. During the iterative update, new evaluation standards in the industry, new enterprise information and new verified evaluation result documents are updated to the vector database; or outdated and unreliable data are cleared.

[0010] As a preferred solution for local construction of vertical large models in the field of environmental monitoring and evaluation, in the reasoning assistance process: Pre-searching relevant knowledge fragments in the vector database according to user input and context information, and providing the relevant knowledge fragments obtained in the pre-search and the user questions to the fine-tuned general large model for auxiliary reasoning; The history of each conversation is recorded, and when the user asks a new question, the retrieval and provision process is repeated, and the historical conversations are superimposed for use by the general large model.

[0011] The present invention also provides a local construction device for a vertical large model in the field of environmental monitoring and evaluation, comprising: The model selection and adjustment module is used to select a general large model that is suitable for the field of environmental monitoring and supports local deployment, and to fine-tune the general large model using specialized knowledge data in the field of environmental monitoring and evaluation; A knowledge base construction processing module is used to construct environmental monitoring and evaluation knowledge base documents according to industry classification, and perform text segmentation processing on the environmental monitoring and evaluation knowledge base documents to obtain text fragments; The vector database building module is used to build a vector database suitable for the field of environmental monitoring and evaluation, and convert knowledge fragments into real-valued vectors and store them in the vector database; The reasoning assistance module is used to use retrieval enhancement generation technology to pre-retrieve user input and context information and generate knowledge fragments for assisting general large model reasoning.

[0012] As a preferred solution for local construction of vertical large models in the field of environmental monitoring and evaluation, the model selection and adjustment module includes: The parameter configuration submodule is used to set x as a local input with a dimension of d; the parameter W of the original weight of the setting part of the general large model is a d×d high-dimensional space; The update matrix definition submodule is used to introduce two low-dimensional matrix spaces A and B with a dimension of r, which are defined as update matrix A and update matrix B; The data dimension reduction submodule is used to update the matrix A and update the matrix B. First, the input d-dimensional data is reduced to r dimensions, where r is the rank of the matrix, from d×d to d×r+r×d; The fine-tuning weight file generation submodule is used to update the matrix A and update the matrix B to map the r-dimensional parameter matrix back to the d-dimensional matrix to keep consistent with the original matrix dimension of the basic model. The original weight matrix of the basic model remains frozen during training, and only the update matrix is ​​trained to finally generate a fine-tuning weight file; A model effect verification submodule is used to load the basic model weights and fine-tuning weights respectively to verify the effect of the general large model; The vertical large model generation submodule merges the basic weight file and the fine-tuning weight file of the specified industry respectively to form a vertical large model in the environmental protection field for reasoning and deployment of different industry types.

[0013] As a preferred solution for local construction of vertical large models in the field of environmental monitoring and evaluation, the knowledge base construction processing module includes: The text paragraph segmentation submodule is used to recursively segment the document into paragraphs by paragraph or title, and set the segment length L and the sliding window size M; for paragraphs longer than L, segment them with L-2M as the modulus; The text segmentation submodule is used to merge and fill titles and paragraphs whose length is less than L according to semantics and contextual relationships, so that the size of the first and last segments is fixed to LM. The first M characters of the next segment are added to the end of the first segment, the last M characters of the previous segment are added to the beginning of the last segment, and M characters of the previous and next segments are added to the beginning and end of the middle segment respectively, to form text segments of a fixed size of L.

[0014] As a preferred solution for local construction of vertical large models in the field of environmental monitoring and evaluation, the vector database construction module includes: A vector database generation submodule, used to vectorize the generated text segments through an embedding model, convert the data into numerical representation, and store them in a vector database; A vector database evaluation submodule is used to evaluate the generated vector database in terms of relevance, grammaticality, and conciseness of the search results; the relevance is the degree of relevance between the generated text and the input and context, the grammaticality is whether the generated grammar is correct, and the conciseness is the degree of conciseness of the generated text; The vector database update submodule is used to iteratively update the vector database. During the iterative update, new evaluation standards in the industry, new enterprise information and new verified evaluation result documents are updated to the vector database; or outdated and unreliable data are cleared.

[0015] As a preferred solution for local construction of vertical large models in the field of environmental monitoring and evaluation, the reasoning auxiliary module includes: A knowledge fragment pre-retrieval submodule, used to pre-retrieve relevant knowledge fragments in the vector database according to user input and context information, and provide the relevant knowledge fragments obtained in the pre-retrieval and the user questions to the fine-tuned general large model for auxiliary reasoning; The historical conversation superposition submodule is used to record the history of each conversation. When the user asks a new question, the retrieval and provision process is repeated, and the historical conversations are superimposed for use by the general large model.

[0016] The present invention has the following advantages: it can ensure that data is completely privatized, reduce the risk of data loss and leakage, and protect the data security and privacy of users; it is based on a general large language model and fine-tunes the model through professional knowledge in the field of environmental protection, and through a large amount of pre-labeled data, the large model is equipped with basic knowledge in the field of environmental protection evaluation, so that the model is more suitable for use in the field of environmental protection monitoring and evaluation; a knowledge base in the field of environmental protection monitoring and evaluation is established, and the retrieval enhancement generation technology is used to make the system more scalable, and the knowledge base is continuously updated to expand and adapt to new environments without adjusting the model. On the other hand, the retrieval enhancement generation uses structured and interrelated data to enhance the reasoning ability of the model and reduce the illusion phenomenon; the workload of environmental protection staff is reduced and work efficiency is improved; it avoids dependence on cloud service providers, and reduces the use cost through distributed deployment in a local area network, and improves concurrency and user response performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the implementation methods of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for the implementation methods or the description of the prior art. Obviously, the drawings in the following description are only exemplary, and for ordinary technicians in this field, other implementation drawings can be derived from the provided drawings without creative work.

[0018] Figure 1 A schematic diagram of a method for constructing a vertical large model in the field of environmental monitoring and evaluation provided in an embodiment of the present invention; Figure 2 A schematic diagram of constructing a vector database in a local construction method of a vertical large model in the field of environmental monitoring and evaluation provided in an embodiment of the present invention; Figure 3 A schematic diagram of local deployment of a vertical large model in the field of environmental monitoring and evaluation provided in an embodiment of the present invention; Figure 4 A schematic diagram of the reasoning process in the local construction method of the vertical large model in the field of environmental monitoring and evaluation provided in an embodiment of the present invention; Figure 5 Schematic diagram of a local construction device for a vertical large model in the field of environmental monitoring and evaluation provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0019] The following is a description of the implementation of the present invention by specific embodiments. People familiar with the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0020] Example 1

[0021] See also Figure 1 The embodiment of the present invention provides a local construction method of a vertical large model in the field of environmental monitoring and evaluation, comprising the following steps: S1. Model selection and adjustment: Select a general large model that is suitable for the field of environmental monitoring and supports local deployment, and use the specialized knowledge data in the field of environmental monitoring and evaluation to fine-tune the general large model; S2. Hierarchical construction of knowledge base: construct environmental monitoring and evaluation knowledge base documents by industry classification, and perform text segmentation processing on the environmental monitoring and evaluation knowledge base documents to obtain text fragments; S3. Vector database construction: Build a vector database suitable for the field of environmental monitoring and evaluation, convert knowledge fragments into real-valued vectors and store them in the vector database; S4. Reasoning assistance: Retrieval-enhanced generation technology is used to pre-retrieve user input and context information to generate knowledge fragments to assist general large model reasoning.

[0022] In this embodiment, in step S1, an environmental protection data set is constructed by utilizing the specialized knowledge data in the field of environmental monitoring and evaluation, and the environmental protection data is preprocessed.

[0023] Specifically, environmental data with clear evaluation results are obtained from the existing environmental monitoring and evaluation database, and preprocessed according to the specified data format based on the industry type. The JSON Lines format is used, which is optimized for performance and usability based on the JSON format. Each line of data is a complete JSON object, that is, each line contains at least one complete data. This format allows the program to read and process line by line when processing data without loading the entire file into memory. For large data sets such as training and fine-tuning large models, data can be loaded and processed on demand, saving memory resources and reducing the memory requirements of training computing devices.

[0024] In addition, this JSON Lines format makes it easy to process data in parallel. Data can be distributed and processed according to the number of processor cores and graphics processing units of the training computing device, thereby speeding up data processing and model training. The basic structure of each line of data is {"message":[{"role":"user","content":"Industry information and questions"},{"role":"assistant","content":"answer"}]}.

[0025] Among them, we use industry as the classification standard, select a batch of high-quality environmental protection data from the environmental protection knowledge base of different industries, build a custom fine-tuning dataset, pre-process the dataset, and convert the data into a multi-round dialogue format dataset required for fine-tuning, that is, a question-and-answer structure, for example: "{ “message”: [ { "role": "user", “content”:”This is the production emission data of XXXXX Chemical Co., Ltd. in XX month of XX year. Please help me check whether there are any excessive emissions and energy consumption in the specific indicators after I input the data.” }, { "role":"assistant", “content”:”OK, this company is a XXX type company, please enter the specific emission data.” } ] [ { "role": "user", “content”:”Product name: XXX, product output 100 tons; production unit name: XXX production line 1”; raw material consumption: 90 tons; auxiliary material consumption: 30 tons; electricity consumption: 10000kwh; steam consumption 10m³; natural gas consumption 500m³; industrial fresh water 50m³; normal production time: 720h; shutdown time: 0h; abnormal operation time: 0h; production load: 83%; waste gas emissions: 50000m³, sulfur dioxide concentration: 5mg / m³, nitrogen oxide concentration 78mg / m³; wastewater discharge: 40m³, COD concentration 196mg / L, ammonia nitrogen concentration 20mg / L; solid waste generation: 3kg.

[0026] } { "role":"assistant", "content": "The overall pollutant emissions of this enterprise are qualified; it should be noted that the COD concentration in wastewater discharge is close to the limit. The COD concentration limit of such enterprises should be below 200mg / L. The nitrogen oxide emissions in waste gas emissions are relatively high. According to relevant industry standards, the emission limit of such enterprises is 100mg / m³; the energy consumption of this enterprise meets the requirements of relevant industry standards."

[0027] } ] }".

[0028] Finally, the data set is divided into a training data set and an evaluation data set. According to the complexity of the actual data set, training resources, and model complexity, the maximum number of training steps, learning rate, matrix size of model degradation, and scaling factor are determined, and then model fine-tuning training is started. After the training is completed, a model weight file is generated, and finally the appropriate weight file is merged into the basic model to form a basic large model with environmental monitoring domain knowledge weights that can be deployed locally.

[0029] Among them, the data set is used to fine-tune the large model and form the fine-tuning weight file: First, let x be a local input with a dimension of d; the parameter W of the original weight of the setting part of the general large model is a d×d high-dimensional space; Next, two low-dimensional matrix spaces A and B are introduced with dimension r, which are defined as update matrix A and update matrix B; Then, update matrix A and update matrix B first reduce the input d-dimensional data to r dimensions, where r is the rank of the matrix, from d×d to d×r+r×d. Because r is much smaller than d, and the specific value of r is 1% to 1 / 1000 of d, the amount of matrix calculation will be greatly reduced during the fine-tuning process. Finally, update matrix A and update matrix B map the r-dimensional parameter matrix back to d-dimensional, consistent with the original matrix dimension of the base model. The original weight matrix of the base model remains frozen during training, and only the update matrix is ​​trained to finally generate a fine-tuning weight file.

[0030] Among them, the fine-tuning weight file finally formed after the fine-tuning is completed cannot be used alone. During verification, the basic model weights and fine-tuning weights are loaded separately to verify the effect of the general large model, and the basic weight file and the fine-tuning weight files of the specified industry are merged separately to form a vertical large model in the environmental protection field for reasoning and deployment of different industry types.

[0031] In this embodiment, in step S2, industry-related knowledge base documents are sorted according to environmental protection industry classification, including basic environmental protection knowledge, relevant standards, enterprise data, electronic documents of frequently referenced professional books, and internal evaluation documents that have been sorted out.

[0032] Specifically, for each document, in the process of hierarchical construction of the knowledge base, the steps of text segmentation processing for the environmental protection monitoring and evaluation knowledge base document include: First, recursively divide the document by paragraphs or headings to obtain paragraphs, and set the slice length L (the actual value used is 510) and the sliding window size M ((M < L / 20)); for paragraphs with a length greater than L, divide them modulo L-2M; For headings and paragraphs with a length less than L, merge and fill them according to semantic and context relationships, so that the head and tail slice sizes are fixed at L-M, add the first M characters of the next slice to the end of the first slice, add the last M characters of the previous slice to the head of the last slice, and add the first M characters of the previous slice and the last M characters of the next slice to the head and tail of the middle slices respectively, forming text slices with a fixed size of L.

[0033] See Figure 2 , in this embodiment, in step S3, in the process of building the vector database: Convert the generated text slices into vectorized form through an embedding model, convert the data into numerical representation, and store it in the vector database; Evaluate the generated vector database in terms of the relevance, grammar, and conciseness of the retrieval results; the relevance is the degree of relevance between the generated text and the input and context, the grammar is whether the generated grammar is correct, and the conciseness is the conciseness of the generated text; Iteratively update the vector database. When iteratively updating, update the new evaluation criteria, new enterprise information, and new verified evaluation result documents in the industry to the vector database; or remove obsolete and unreliable data.

[0034] Specifically, text slices are semi-structured or unstructured text data, which are difficult for computers to directly understand and process. The embedding model is based on the principle of deep learning word vector models (such as Word2Vec, BERT, etc.), and regards each text slice as a sequence of words or tokens. Taking BERT as an example, through multiple layers of Transformer encoders, in-depth semantic understanding of the text is carried out to capture the complex semantic relationships between words. In this process, the text slices are mapped into a low-dimensional vector space, and each text slice is transformed into a fixed-length real-valued vector. The distance between these vectors in the space reflects the similarity degree of the text content, and similar text slices are closer in the vector space. Subsequently, these numerical vectors are stored in the vector database, and the vector database usually adopts special data structures and algorithms (such as KD tree, HNSW, etc.) to efficiently store and retrieve these vectors for subsequent quick search of relevant knowledge fragments.

[0035] Among them, when searching for relevant knowledge fragments in the vector database, it is necessary to determine the degree of relevance between the search results and the user input and context. The relevance evaluation is based on the principle of the vector space model, and calculates the similarity between the retrieved text vector and the user input vector (also generated by the embedding model). Commonly used similarity measurement methods include cosine similarity and Euclidean distance. Cosine similarity measures the similarity by calculating the cosine value of the angle between two vectors. The closer the cosine value is to 1, the more similar the text content represented by the two vectors is; the Euclidean distance measures the geometric distance between the two vectors in space. The smaller the distance, the more similar the text. Through these similarity calculation methods, the relevance of the search results to the input and context is evaluated to ensure that the retrieved knowledge fragments are closely related to current needs.

[0036] Among them, grammatical evaluation uses grammatical analysis technology in natural language processing, such as dependency syntactic analysis and part-of-speech tagging. Dependency syntactic analysis can analyze the grammatical structure relationship between words in a sentence and determine whether the sentence conforms to grammatical rules. Part-of-speech tagging marks the part of speech (such as noun, verb, adjective, etc.) for each word, and detects whether there are grammatical errors in the generated text by analyzing the collocation and grammatical structure of the part of speech. For example, if the collocation of the verb and the object in a sentence does not conform to the grammatical rules, or the sentence lacks necessary components, the grammatical analysis technology can detect it, thereby evaluating the grammatical correctness of the vector database retrieval results.

[0037] Among them, the conciseness evaluation is based on the principles of text length and information redundancy. First, the length of the retrieved text is counted. Too long text may contain more redundant information. Then, the information density of the text is measured by indicators such as information entropy. The higher the information entropy, the richer and less redundant the information in the text. If there are a large number of repeated words or expressions in the text, the information entropy will be low, indicating that the text is not concise enough. By comprehensively considering factors such as text length and information entropy, the conciseness of the search results is evaluated to avoid returning lengthy and redundant texts.

[0038] As the environmental protection industry continues to develop, new evaluation standards, corporate information, and verified evaluation result documents continue to emerge, and there are also outdated and unreliable data. In order to ensure the timeliness and accuracy of the vector database, iterative updates are required. When there is new data, the new data is first preprocessed, such as cleaning, labeling, etc., and then converted into vector form through the embedding model and inserted into the vector database. For outdated and unreliable data, it is identified and deleted according to pre-set rules (such as the timestamp of the data, the reliability of the source, etc.). During the update process, the vector database will recalculate the relevant indexes and similarity metrics to ensure that the new data can be correctly integrated into the database without affecting the retrieval and use of the original data, so that the vector database always maintains high quality and provides accurate and up-to-date knowledge support for the vertical large model in the field of environmental monitoring and evaluation.

[0039] In this embodiment, in step S4, in the reasoning assistance process: Pre-searching relevant knowledge fragments in the vector database according to user input and context information, and providing the relevant knowledge fragments obtained in the pre-search and the user questions to the fine-tuned general large model for auxiliary reasoning; The history of each conversation is recorded, and when the user asks a new question, the retrieval and provision process is repeated, and the historical conversations are superimposed for use by the general large model.

[0040] Specifically, each interaction with the user is recorded, including the user's questions, the system's answers, and related context information. This conversation history information is stored in a structured manner, for example, the details of each round of conversation can be recorded in a time series manner. This recording method can fully preserve the conversation process and information, providing rich background information for subsequent interactions.

[0041] When the user asks a new question, the search operation is performed again to obtain knowledge fragments related to the new question. At the same time, the previous conversation history information is extracted and provided to the large model together with the new question and the newly retrieved knowledge fragment. The large model can use this historical information to better understand the user's intentions and the background of the question. For example, the user previously asked a question about the emission of specific pollutants from a certain company, and then asked about the company's overall environmental protection situation. The large model can more accurately grasp the user's needs by combining the previous conversation history about the company, use the previous information as a context reference, and perform more comprehensive reasoning, thereby generating answers that better meet the user's expectations. This process utilizes the large model's ability to process long-sequence information. By continuously superimposing historical conversations, the model learns more contextual information and user preferences, and improves its reasoning and answering capabilities in multi-round conversation scenarios.

[0042] In a possible embodiment, the fine-tuned large model and vector database are deployed to the inference computing device. According to the actual usage, the fine-tuned large model generated by the training computing device is deployed to the inference computing device, and the generated knowledge base is deployed to the database device, that is, the production environment is deployed to the application environment. The user accesses the interface and then calls the fine-tuned large model deployed on the inference computing device. After the deployment is completed, the production environment can continue to fine-tune with the updated data set, and the knowledge base can continue to be dynamically updated and tested. Figure 3The figure shows the deployment of training computing equipment. The training computing equipment generates fine-tuned large models for different industries based on the environmental protection industry classification. After training and verification, they are distributed to the corresponding reasoning devices. Users interact with the reasoning devices through access interfaces. The reasoning devices can call vector knowledge bases to assist reasoning, and the knowledge bases can be updated dynamically. The training computing equipment generates fine-tuned large models for different industries based on the environmental protection industry classification. After training and verification, they can be directly pushed to the corresponding reasoning devices to provide services for users. The reasoning devices can call knowledge fragments in the knowledge base at any time to assist reasoning. At the same time, new knowledge fragments can be continuously added to the knowledge base, and outdated knowledge fragments can be deleted at any time.

[0043] In one possible embodiment, see Figure 4 , which shows the inference processing flow. When the user uses it, the access interface first selects and directs the user to the appropriate inference computing device according to the type of environmental protection industry to which the user belongs. When the user asks a question, the inference computing device first converts the input content into a vector, and then retrieves several knowledge fragments related to the industry and the question from the vector knowledge base according to the industry type. The retrieved knowledge fragments and the original question are combined and submitted to the fine-tuned large model, and the fine-tuned large model gives a reply after reasoning operation. When the user needs to continue the interaction, the above steps are repeated, and the inference computing device passes the historical session context information to the next interaction.

[0044] In summary, the present invention screens out a model suitable for the environmental monitoring field and capable of local deployment from a general large model. Special knowledge data in the field of environmental monitoring and evaluation are collected, and the selected model is fine-tuned using these data. During the fine-tuning process, by introducing the low-rank matrix technology, the high-dimensional data is dimensionally reduced and then mapped back to the original dimension, and only the low-rank matrix is trained, while the original weight matrix of the basic model is frozen, thereby generating a fine-tuned weight file. Then, the weights of the basic model and the fine-tuned weights are respectively loaded for effect verification, and then the basic weight file and the fine-tuned weight files of different industries are merged to form a vertical large model in the environmental protection field suitable for inference and deployment in different industries. According to the categories of the environmental protection industry, various relevant knowledge documents are collected, such as basic environmental protection knowledge, relevant standards, enterprise data, and electronic documents of professional books, etc. These documents are first recursively segmented by paragraph or title, and appropriate slice length L (such as 510) and sliding window size M (M < L / 20) are set. For paragraphs longer than L, they are segmented with a modulus of L - 2M; for titles and paragraphs shorter than L, they are merged and filled according to semantic and context relationships to generate text slices of a fixed size of L, enhancing the coherence and retrievability of knowledge. The generated text slices are transformed into real-valued vectors through an embedding model and stored in a vector database. Algorithms such as cosine similarity and Euclidean distance are used to evaluate the vector database in terms of the relevance, grammar, and conciseness of the retrieval results. If problems are found in the evaluation, iterative updates are made in a timely manner, adding new evaluation standards, enterprise information, and verified evaluation result documents in the industry, and at the same time clearing outdated and unreliable data to ensure the high quality of the vector database. After the user inputs a question, the system converts it into a vector form, retrieves relevant knowledge fragments in the vector database, and provides these fragments and the user's question to the fine-tuned large model for inference. The large model generates an answer based on the received information and records this conversation. When the user asks a new question, the above retrieval and inference processes are repeated, and the historical conversation information is superimposed and provided to the large model to help the model better understand the user's intention, so as to give a more accurate and user-demand-compliant answer, continuously optimizing the user interaction experience.

[0045] Example 2

[0046] See Figure 5 , the present invention in Example 2 also provides a local construction device for a vertical large model in the field of environmental monitoring and evaluation, including: A model selection and adjustment module 100, configured to select a general large model suitable for the environmental monitoring field and supporting local deployment, and fine-tune the general large model using special knowledge data in the field of environmental monitoring and evaluation; A knowledge base construction and processing module 200, configured to construct an environmental monitoring and evaluation knowledge base document by industry classification, perform text segmentation processing on the environmental monitoring and evaluation knowledge base document to obtain text slices; The vector database building module 300 is used to build a vector database suitable for the field of environmental monitoring and evaluation, and convert knowledge fragments into real-valued vectors and store them in the vector database; The reasoning assistance module 400 is used to adopt the retrieval enhancement generation technology to pre-retrieve the user input and context information, and generate knowledge fragments for assisting the general large model reasoning.

[0047] In this embodiment, the model selection and adjustment module 100 includes: The parameter configuration submodule 101 is used to set x as a local input with a dimension of d; the parameter W of the original weight of the setting part of the general large model is a d×d high-dimensional space; An update matrix definition submodule 102 is used to introduce two low-dimensional matrix spaces A and B, with a dimension of r, which are defined as update matrix A and update matrix B; The data dimension reduction submodule 103 is used to update the matrix A and update the matrix B. First, the input d-dimensional data is reduced to r dimensions, where r is the rank of the matrix, from d×d to d×r+r×d; A fine-tuning weight file generation submodule 104 is used to update the matrix A and the update matrix B to map the r-dimensional parameter matrix back to the d-dimensional matrix to keep consistent with the original matrix dimension of the basic model. The original weight matrix of the basic model remains frozen during training, and only the update matrix is ​​trained to finally generate a fine-tuning weight file; The model effect verification submodule 105 is used to load the basic model weights and the fine-tuning weights respectively to verify the effect of the general large model; The vertical large model generation submodule 106 merges the basic weight file and the fine-tuning weight file of the specified industry respectively to form a vertical large model in the environmental protection field for reasoning and deployment of different industry types.

[0048] In this embodiment, the knowledge base construction processing module 200 includes: The text paragraph segmentation submodule 201 is used to first recursively segment the document by paragraph or title to obtain paragraphs, set the segment length L and the sliding window size M; for paragraphs with a length greater than L, segment them with L-2M as the modulus; The text segmentation submodule 202 is used to merge and fill titles and paragraphs whose length is less than L according to semantics and contextual relationships, so that the size of the first and last segments is fixed to LM, the first M characters of the next segment are added to the tail of the first segment, the last M characters of the previous segment are added to the head of the last segment, and the M characters of the previous segment and the next segment are added to the head and tail of the middle segment respectively, to form text segments of a fixed size of L.

[0049] In this embodiment, the vector database building module 300 includes:

[0050] A vector database generation submodule 301 is used to vectorize the generated text segments through an embedding model, convert the data into numerical representation, and store them in a vector database; The vector database evaluation submodule 302 is used to evaluate the generated vector database in terms of relevance, grammaticality, and conciseness of the search results; the relevance is the degree of relevance between the generated text and the input and context, the grammaticality is whether the generated grammar is correct, and the conciseness is the degree of conciseness of the generated text; The vector database update submodule 303 is used to iteratively update the vector database. During iterative update, new evaluation standards in the industry, new enterprise information and new verified evaluation result documents are updated into the vector database; or outdated and unreliable data are cleared.

[0051] In this embodiment, the reasoning auxiliary module 400 includes: The knowledge fragment pre-retrieval submodule 401 is used to pre-retrieve relevant knowledge fragments in the vector database according to user input and context information, and provide the relevant knowledge fragments obtained in the pre-retrieval and the user questions to the fine-tuned general large model for auxiliary reasoning; The historical conversation superposition submodule 402 is used to record each conversation history, and when the user asks a new question, the retrieval and provision process is repeated, and the historical conversations are superimposed for use by the general large model.

[0052] It should be noted that the information interaction, execution process and other contents between the modules of the above-mentioned device are based on the same concept as the method embodiment in Example 1 of the present application, and the technical effects they bring are the same as those of the method embodiment of the present application. For specific contents, please refer to the description in the method embodiment shown in the previous part of the present application, and will not be repeated here.

[0053] Example 3

[0054] Embodiment 3 of the present invention provides a non-transitory computer-readable storage medium, in which a program code for a local construction method of a vertical large model in the field of environmental monitoring and evaluation is stored. The program code includes instructions for executing embodiment 1 or any possible implementation method of the local construction method of a vertical large model in the field of environmental monitoring and evaluation.

[0055] Computer-readable storage media can be any available medium that can be accessed by a computer or a data storage device such as a server or data center that includes one or more available media. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive (SSD)).

[0056] Example 4

[0057] Embodiment 4 of the present invention provides an electronic device, including: a memory and a processor; The processor and the memory communicate with each other via a bus; the memory stores program instructions that can be executed by the processor, and the processor calls the program instructions to execute the local construction method of the vertical large model in the field of environmental monitoring and evaluation of Example 1 or any possible implementation method thereof.

[0058] Specifically, the processor can be implemented by hardware or by software. When implemented by hardware, the processor can be a logic circuit, an integrated circuit, etc.; when implemented by software, the processor can be a general-purpose processor implemented by reading software codes stored in a memory. The memory can be integrated in the processor or can be located outside the processor and exist independently.

[0059] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present invention is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions can be transmitted from a website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server or data center.

[0060] Obviously, those skilled in the art should understand that the above modules or steps of the present invention can be implemented by a general computing device, they can be concentrated on a single computing device, or distributed on a network composed of multiple computing devices, and optionally, they can be implemented by a program code executable by a computing device, so that they can be stored in a storage device and executed by the computing device, and in some cases, the steps shown or described can be executed in a different order than here, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. Thus, the present invention is not limited to any specific combination of hardware and software.

[0061] Although the present invention has been described in detail above by general description and specific embodiments, it is obvious to those skilled in the art that some modifications or improvements can be made to the present invention. Therefore, these modifications or improvements made without departing from the spirit of the present invention all belong to the scope of protection claimed by the present invention.

[0062] Although the present invention has been described in detail above by general description and specific embodiments, it is obvious to those skilled in the art that some modifications or improvements can be made to the present invention. Therefore, these modifications or improvements made without departing from the spirit of the present invention all belong to the scope of protection claimed by the present invention.

Claims

1. A local construction method for a vertical large model in the field of environmental monitoring and evaluation, characterized in that: include: Model selection and adjustment: Select a general large model that is suitable for the field of environmental monitoring and supports local deployment, and use knowledge data in the field of environmental monitoring and evaluation to fine-tune the general large model; Hierarchical construction of the knowledge base: construct environmental monitoring and evaluation knowledge base documents by industry classification, and perform text segmentation processing on the environmental monitoring and evaluation knowledge base documents to obtain text fragments; Vector database construction: Build a vector database suitable for the field of environmental monitoring and evaluation, convert knowledge fragments into real-valued vectors and store them in the vector database; Reasoning assistance: Retrieval-enhanced generation technology is used to pre-retrieve user input and context information to generate knowledge fragments to assist general large model reasoning.

2. According to claim 1, a local construction method of a vertical large model in the field of environmental monitoring and evaluation is characterized in that: During the model selection and adjustment process: set up x is a local input with dimension d ; The parameters of the original weights of the setting part of the general large model W For one d×d High-dimensional space; Introducing two low-dimensional matrix spaces A and B , the dimension is r , defined as the update matrix A and update the matrix B ; Update Matrix A and update the matrix B First, enter d Dimensionality reduction of data to r dimension, r is the rank of the matrix, from d×d becomes d ×r+r×d ; Update Matrix A and update the matrix B Will r The parameter matrix of dimension is then mapped back to d The dimension is consistent with the original matrix dimension of the base model. The original weight matrix of the base model remains frozen during training, and only the matrix is ​​updated during training to finally generate a fine-tuning weight file.

3. According to claim 2, a local construction method of a vertical large model in the field of environmental monitoring and evaluation is characterized in that: The model selection and adjustment process also includes: The basic model weights and fine-tuning weights are loaded separately to verify the effect of the general large model, and the basic weight file and the fine-tuning weight file of the specified industry are merged separately to form a vertical large model in the environmental protection field for reasoning and deployment of different industry types.

4. According to claim 1, a local construction method of a vertical large model in the field of environmental monitoring and evaluation is characterized in that: In the process of constructing the knowledge base hierarchically, the step of performing text segmentation processing on the environmental monitoring and evaluation knowledge base document includes: First, recursively split the document into paragraphs or titles, set the slice length L and the sliding window size M; for paragraphs longer than L, split them modulo L-2M; For titles and paragraphs whose length is less than L, they are merged and filled based on semantics and contextual relationships, so that the size of the first and last fragments is fixed to LM. The first M characters of the next fragment are added to the end of the first fragment, the last M characters of the previous fragment are added to the beginning of the last fragment, and M characters of the previous and next fragments are added to the beginning and end of the middle fragments respectively, forming text fragments of a fixed size of L.

5. The local construction method of a vertical large model in the field of environmental monitoring and evaluation according to claim 1 is characterized in that: During the construction of the vector database: The generated text segments are vectorized through an embedding model, the data are converted into numerical representations, and stored in a vector database; Evaluate the generated vector database in terms of relevance, grammaticality, and conciseness of the search results; the relevance is the degree of relevance of the generated text to the input and context, the grammaticality is whether the generated grammar is correct, and the conciseness is the degree of conciseness of the generated text; The vector database is iteratively updated. During the iterative update, new evaluation standards in the industry, new enterprise information and new verified evaluation result documents are updated to the vector database; or outdated and unreliable data are cleared.

6. The local construction method of a vertical large model in the field of environmental monitoring and evaluation according to claim 1 is characterized in that: Inference assistance process: Pre-searching relevant knowledge fragments in the vector database according to user input and context information, and providing the relevant knowledge fragments obtained in the pre-search and the user questions to the fine-tuned general large model for auxiliary reasoning; The history of each conversation is recorded, and when the user asks a new question, the retrieval and provision process is repeated, and the historical conversations are superimposed for use by the general large model.

7. A local construction device for a vertical large model in the field of environmental monitoring and evaluation, characterized in that: include: The model selection and adjustment module is used to select a general large model that is suitable for the environmental monitoring field and supports local deployment, and to fine-tune the general large model using knowledge data in the field of environmental monitoring and evaluation; A knowledge base construction processing module is used to construct environmental monitoring and evaluation knowledge base documents according to industry classification, and perform text segmentation processing on the environmental monitoring and evaluation knowledge base documents to obtain text fragments; The vector database building module is used to build a vector database suitable for the field of environmental monitoring and evaluation, and convert knowledge fragments into real-valued vectors and store them in the vector database; The reasoning assistance module is used to use retrieval enhancement generation technology to pre-retrieve user input and context information and generate knowledge fragments for assisting general large model reasoning.

8. The local construction device of a vertical large model in the field of environmental monitoring and evaluation according to claim 7 is characterized in that: The model selection and adjustment module includes: Parameter configuration submodule, used to set x is a local input with dimension d ; The parameters of the original weights of the setting part of the general large model W For one d×d High-dimensional space; Update the matrix definition submodule to introduce two low-dimensional matrix spaces A and B , the dimension is r , defined as the update matrix A and update the matrix B ; Data dimensionality reduction submodule, used to update the matrix A and update the matrix B First, enter d Dimensionality reduction of data to r dimension, r is the rank of the matrix, from d×d becomes d×r+r×d ; Fine-tuning weight file generation submodule for updating the matrix A and update the matrix B Will r The parameter matrix of dimension is then mapped back to d The dimension is consistent with the original matrix dimension of the base model. The original weight matrix of the base model remains frozen during training, and only the matrix is ​​updated during training to generate a fine-tuning weight file. A model effect verification submodule is used to load the basic model weights and fine-tuning weights respectively to verify the effect of the general large model; The vertical large model generation submodule merges the basic weight file and the fine-tuning weight file of the specified industry respectively to form a vertical large model in the environmental protection field for reasoning and deployment of different industry types.

9. The local construction device of a vertical large model in the field of environmental monitoring and evaluation according to claim 7 is characterized in that: The knowledge base construction processing module includes: The text paragraph segmentation submodule is used to recursively segment the document into paragraphs by paragraph or title, and set the segment length L and the sliding window size M; for paragraphs longer than L, segment them with L-2M as the modulus; The text segmentation submodule is used to merge and fill titles and paragraphs whose length is less than L according to semantics and contextual relationships, so that the size of the first and last segments is fixed to LM. The first M characters of the next segment are added to the end of the first segment, the last M characters of the previous segment are added to the beginning of the last segment, and M characters of the previous and next segments are added to the beginning and end of the middle segment respectively, to form text segments of a fixed size of L.

10. The local construction device of a vertical large model in the field of environmental monitoring and evaluation according to claim 7 is characterized in that: The vector database building module includes: A vector database generation submodule, used to vectorize the generated text segments through an embedding model, convert the data into numerical representation, and store them in a vector database; A vector database evaluation submodule is used to evaluate the generated vector database in terms of relevance, grammaticality, and conciseness of the search results; the relevance is the degree of relevance between the generated text and the input and context, the grammaticality is whether the generated grammar is correct, and the conciseness is the degree of conciseness of the generated text; A vector database update submodule is used to iteratively update the vector database. During iterative update, new evaluation standards, new enterprise information and new verified evaluation result documents in the industry are updated into the vector database; or outdated and unreliable data are cleared; The reasoning auxiliary module comprises: A knowledge fragment pre-retrieval submodule, used to pre-retrieve relevant knowledge fragments in the vector database according to user input and context information, and provide the relevant knowledge fragments obtained in the pre-retrieval and the user questions to the fine-tuned general large model for auxiliary reasoning; The historical conversation superposition submodule is used to record the history of each conversation. When the user asks a new question, the retrieval and provision process is repeated, and the historical conversations are superimposed for use by the general large model.

Images