Green large model construction method based on behavior prompt retrieval enhancement technology

Abstract

The invention relates to the technical field of artificial intelligence crossing, in particular to a green large model construction method based on a behavior prompt retrieval enhancement technology, which comprises the following steps: acquiring text data related to green life and green behavior through an automatic crawler technology; extracting a structured text data set through a large model, and storing the structured text data set into a database to obtain a green knowledge base; performing data annotation through a large model to obtain a sentence pair training set, and performing fine tuning training on the sentence pair training data set through a deep learning model; obtaining a green knowledge retrieval model; performing text pre-coding on green behavior texts in the green knowledge base to obtain coding vectors, and storing the coding vectors in an index database to obtain a green behavior KV vector base; a green large model application process of RAG enhanced retrieval based on a large model is constructed based on a green knowledge retrieval model, and the technical framework combines a core technology of the large model with green knowledge to realize cross enabling of an artificial intelligence technology to the green low-carbon field.

Description

TECHNICAL FIELD

[0001] The application relates to the field of artificial intelligence cross technology, and particularly relates to a green large model construction method based on behavior prompt retrieval enhancement technology. BACKGROUND

[0002] The application of artificial intelligence technology in the field of green low carbon is mainly focused on traditional machine learning methods, and the deep combination of large model technology and green scenarios is relatively less. At present, the large model lacks directional retrieval and reasoning ability for green knowledge, resulting in problems such as knowledge fragmentation and inaccurate response when guiding public green behavior.

[0003] Therefore, the application provides a green large model construction method based on behavior prompt retrieval enhancement technology. SUMMARY

[0004] The application aims to provide a green large model construction method based on behavior prompt retrieval enhancement technology, and relates to the field of artificial intelligence cross technology, and particularly relates to a green large model construction method based on behavior prompt retrieval enhancement technology, which comprises the following steps: obtaining text data related to green life and green behavior through automatic crawler technology; extracting structured green behavior and description through a Deepseek-R1-671B large model and storing them in a MYSQL database to obtain a green knowledge base; performing data labeling through the Deepseek-R1-671B large model, labeling as a sentence pair training set, fine-tuning the sentence pair training data set through a Sentence-Transformer model, obtaining a green knowledge retrieval model; performing text pre-encoding on the green behavior text in the green knowledge base to obtain an encoding vector and storing the encoding vector in a Cassandra index database to obtain a green behavior KV vector base; and constructing a green large model application process based on RAG enhanced retrieval based on the large model based on the green knowledge retrieval model. The technical framework combines the core technology of the large model with green knowledge, adopts the technology of the large model RAG and Agent, and realizes the cross empowerment of artificial intelligence technology in the field of green low carbon.

[0005] To achieve the above-mentioned purpose, the application provides the following technical scheme: A green large model construction method based on behavior prompt retrieval enhancement technology comprises the following steps: S1. Unstructured text data related to green life and behavior is obtained from various Internet platforms through a distributed crawler framework, structured green behavior and description are extracted through a DeepSeek-R1-671B large model, structured text data sets are formed and input into a database, and a green knowledge base is obtained; S2. The sentence pair training data set is labeled by the DeepSeek-R1-671B large model; the sentence pair training data set is fine-tuned by the Sentence-Transformer model; and a green knowledge retrieval model is obtained; S3. The structured description of the green knowledge base is batch-encoded to generate an Embedding vector, which is stored in a Cassandra index database; the generated text description and the corresponding Embedding vector are stored in the form of a key-value pair, generating a green behavior KV vector library; S4. When a user question query is received, the green knowledge retrieval model is first used to encode the user question query to generate a query vector, and a context is obtained based on the query vector and the green KV vector library; the user query and the context are spliced to form an enhanced prompt word, which is input into the DeepSeek-R1-671B large model for reasoning to generate a final answer, forming a closed-loop decision chain, and a green large model application process based on RAG enhanced retrieval based on a large model is constructed based on the green knowledge retrieval model.

[0006] Preferably, the specific steps of S1 are: first, unstructured text is scraped from government websites, microblogs, Douyin, Xiaohongshu, and public accounts based on the Scrapy framework of Python; then, the DeepSeek-R1-671B large model is called to perform knowledge extraction on the scraped text to form a structured text data set; finally, the structured text data set is stored in a predetermined MYSQL database to constitute the green knowledge base.

[0007] Preferably, the generation process of the sentence pair training data set in S2 includes: calling the DeepSeek-R1-671B large model to rewrite the sentence in the green knowledge base to generate a positive sample of a sentence pair; and by random sampling technology, matching multiple semantically unrelated sentences as negative samples for each positive sample, and the number of negative samples is more than that of positive samples, a training data set with a predetermined positive and negative sample ratio is constructed.

[0008] Preferably, the fine-tuning process is: inputting the sentence pair training data set into a Sentence-Transformer model based on SIMCSE for a predetermined number of iterations of training; during the training process, an InfoNCE-based contrastive learning loss function is applied to optimize the model parameters, and the Sentence-Transformer model generates a vector space through an encoder, and the vector space includes positive sample pair vectors and negative sample pair vectors.

[0009] Preferably, the loss function is used to maximize the cosine similarity between the positive sample pair vectors, and at the same time minimize the cosine similarity between the negative sample pair vectors.

[0010] Preferably, the context acquisition process comprises: performing similarity retrieval on the query vector and the set of encoded green vectors in the green KV vector library, retrieving the previous sequence vectors with similarity greater than a preset threshold, and combining the text corresponding to the previous sequence vectors to form the context.

[0011] Preferably, the green large model application process is an AI Agent application process based on RAG enhanced retrieval constructed in combination with the green knowledge retrieval model; first, based on the RAG enhanced retrieval process, the text related to the user input Query is retrieved from the green KV vector library, and the text is sorted to splice the prompt word with the Context background into the Deepseek-R1-671B large model for reasoning; and then the final Answer is generated and returned to the user, completing the closed loop of the entire application process.

[0012] Compared with the prior art, the application has the following advantages: 1. The application automatically collects massive unstructured text data through the Scrapy automatic crawler framework; it innovatively uses the powerful reasoning and generation capability of the DeepSeek-R1-671B large model to automatically complete knowledge extraction, data structuring, and sample generation and labeling, and connects the whole link from raw web data to high-quality training data set, greatly improving the construction efficiency of the domain knowledge base, so as to further realize the cross empowerment of artificial intelligence technology in the field of green and low carbon.

[0013] 2. Based on the green knowledge base, the application generates positive and negative sample pairs through the large model, fine-tunes on the SIMCSE model base, trains the green knowledge retrieval model, batch encodes the structured description of the green knowledge base to generate Embedding vectors, stores the Embedding vectors in the Cassandra index database; stores the generated text description and the corresponding Embedding vector in the form of key-value pair to generate the green behavior KV vector library; when receiving a user problem query, first encode the user problem query using the green knowledge retrieval model to generate a query vector, and obtain a context based on the query vector and the green KV vector library; splice the user query and the context to form an enhanced prompt word, and input it into the DeepSeek-R1-671B large model for reasoning to generate a final answer, forming a closed loop decision chain, so as to further realize the cross empowerment of artificial intelligence technology in the field of green and low carbon.

[0014] 3、The green large model reasoning technical process designed according to the characteristics of green knowledge fills the blank of the current large model in the application of the green low-carbon field, so that the cross empowerment of artificial intelligence technology to the green low-carbon field can be further realized. BRIEF DESCRIPTION OF DRAWINGS

[0015] Figure 1 A green large model construction method based on behavior prompt retrieval enhancement technology is provided for the embodiment of the application. Figure 2 A flowchart of the SIMCSE model is provided for the embodiment of the application. DETAILED DESCRIPTION

[0016] The technical solutions in the embodiments of the application will be clearly and completely described below with reference to the drawings in the embodiments of the application. Obviously, the described embodiments are only part of the embodiments of the application, rather than all the embodiments of the application. Based on the embodiments in the application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of the application.

[0017] Embodiment one The application of artificial intelligence technology in the green low-carbon field is mostly focused on traditional machine learning methods, and the deep combination of large model technology and green scenarios is relatively less. The current large model lacks directional retrieval and reasoning ability for green knowledge, resulting in problems such as knowledge fragmentation and inaccurate response when guiding public green behavior.

[0018] In order to realize the cross empowerment of artificial intelligence technology to the green low-carbon field, a green large model construction method based on behavior prompt retrieval enhancement technology is applied; For example Figure 1 A flowchart of the green large model construction method based on behavior prompt retrieval enhancement technology is provided for the embodiment of the application, which specifically includes: S1. Through a distributed crawler framework, unstructured text data related to green life and behavior is grabbed from various Internet platforms, structured green behavior and description are extracted by DeepSeek-R1-671B large model, structured text data set is formed and input into the database, and then green knowledge base is obtained; Further, the specific steps of S1 are as follows: first, unstructured text is scraped from government websites, microblogs, TikTok, Xiaohongshu and public accounts based on the Python-based Scrapy framework; then, the DeepSeek-R1-671B large model is called to extract knowledge from the scraped text to form a structured text dataset; finally, the structured text dataset is stored in a preset MYSQL database to form the green knowledge base.

[0019] Further, the extraction method uses large model prompting engineering technology, which requires the large model to extract green and low-carbon related terms and sentences from unstructured data to form a structured text dataset. This embodiment automatically collects massive amounts of unstructured text data through the Scrapy automated crawler framework. It innovatively uses the powerful reasoning and generation capabilities of the DeepSeek-R1-671B large model to automatically complete knowledge extraction, data structuring, and sample generation and labeling, breaking through the full link from raw web data to high-quality training datasets, greatly improving the efficiency of building domain knowledge bases, and further enabling the cross-empowerment of artificial intelligence technology in the green and low-carbon field.

[0020] S2. Labeling sentence pair training dataset through DeepSeek-R1-671B large model; fine-tuning sentence pair training dataset through Sentence-Transformer model; obtaining green knowledge retrieval model; Further, the generation process of the sentence pair training dataset in S2 includes: calling the DeepSeek-R1-671B large model to rewrite the sentence in the green knowledge base to generate positive samples; and through random sampling technology, matching multiple semantically unrelated sentences as negative samples for each positive sample, and the number of negative samples is more than that of positive samples, to construct a training dataset with a preset positive-negative sample ratio; the preset positive-negative sample ratio is 1:5; the main labeling method of the sentence pair training dataset is large model prompting engineering technology; Further, the fine-tuning epoch is 50; Further, the fine-tuning training process is: inputting the sentence pair training data set into a Sentence-Transformer model based on SIMCSE, and performing a predetermined number of iterations of training; during the training process, an InfoNCE-based contrastive learning loss function is applied to optimize the model parameters, and the model generates a vector space through an encoder, and the vector space includes positive sample pair vectors and negative sample pair vectors. The Sentence-Transformer model is a Python library based on PyTorch and Transformer, and is mainly used to generate embedding vectors of sentences and texts; The loss function formula is: ; Wherein, represents the loss function; represents an exponential function with as the base; represents the cosine similarity of sample and sample ; represents a temperature coefficient for controlling the distribution of similarity; represents the number of samples; represents the index; Further, the loss function is used to maximize the cosine similarity between positive sample pair vectors, and at the same time minimize the cosine similarity between negative sample pair vectors.

[0021] S3. Batch encoding of structured descriptions of green knowledge bases to generate Embedding vectors, and storing the Embedding vectors in a Cassandra index database; storing the generated text description and the corresponding Embedding vector in the format of key-value pair to generate a green behavior KV vector library; Further, the process of obtaining the context includes: performing similarity retrieval on the query vector and the set of encoded green vectors in the green KV vector library, retrieving the pre-sequence vectors with similarity greater than a preset threshold, and associating the text corresponding to the pre-sequence vectors to form the context.

[0022] Further, the preset threshold is 0.8; the pre-sequence vector represents the top 5 vectors in the ranking; Further, the green large model application process is a set of AI Agent application processes based on RAG enhanced retrieval constructed in combination with the green knowledge retrieval model. First, based on the RAG enhanced retrieval process, text sorting related to the user input Query is retrieved from the green KV vector library, so as to splice the prompt word with the Context background into the Deepseek-R1-671B large model for reasoning; and then the final Answer is returned to the user, and the closed loop of the entire application process is completed.

[0023] S4. When receiving the user problem query, first, the green knowledge retrieval model is utilized to encode the user problem query to generate a query vector, and a context is obtained based on the query vector and the green KV vector library; the user query and the context are spliced to form an enhanced prompt word, and are input into the DeepSeek-R1-671B large model for reasoning to generate a final answer, form a closed loop decision chain, and based on the green knowledge retrieval model, a green large model application process based on RAG enhanced retrieval of the large model is constructed.

[0024] In the embodiment, the automatic crawler technology is used to obtain text data related to green life and green behavior; the Deepseek-R1-671B large model is used to extract structured green behavior and description and store them in a MYSQL database to obtain a green knowledge base; the Deepseek-R1-671B large model is used for data labeling to label as a sentence pair training set, the Sentence-Transformer model is used to fine-tune the training of the sentence pair training data set; the green knowledge retrieval model is obtained; the green behavior text in the green knowledge base is pre-encoded to obtain an encoding vector and is stored in a Cassandra index database to obtain a green behavior KV vector library; based on the green knowledge retrieval model, a green large model application process based on RAG enhanced retrieval of the large model is constructed, the technical framework combines the core technology of the large model with green knowledge, adopts the technology of the large model RAG and Agent, and realizes the cross empowerment of artificial intelligence technology to the green low-carbon field.

[0025] Embodiment two In order to realize the cross empowerment of artificial intelligence technology to the green low-carbon field, a green large model construction method based on behavior prompt retrieval enhancement technology is applied. As Figure 1 A flowchart of a green large model construction method based on behavior prompt retrieval enhancement technology provided by the embodiment of the application, specifically comprising: S1. Extracting unstructured text data related to green life and behavior from various Internet platforms through a distributed crawler framework, extracting structured green behavior and description through DeepSeek-R1-671B large model, forming structured text data set and inputting into database, and then obtaining green knowledge base; S2. Annotate sentence pair training data set through DeepSeek-R1-671B large model; fine-tune training of sentence pair training data set through Sentence-Transformer model; obtain green knowledge retrieval model; S3. Batch coding of structured description of green knowledge base to generate Embedding vector, store Embedding vector in Cassandra index database; store generated text description and corresponding Embedding vector in key-value pair format to generate green behavior KV vector library; S4. When receiving a user problem query, first encode the user problem query using the green knowledge retrieval model to generate a query vector, and obtain a context based on the query vector and the green KV vector library; concatenate the user query and the context to form an enhanced prompt word, and input it into the DeepSeek-R1-671B large model for reasoning to generate a final answer, form a closed-loop decision chain, and build a green large model application process based on RAG enhanced retrieval based on the large model based on the green knowledge retrieval model.

[0026] Further, the specific steps of S1 are: first, using the Scrapy framework based on Python to extract unstructured text from government websites, microblogs, Douyin, Xiaohongshu and public accounts; then, calling the DeepSeek-R1-671B large model to extract knowledge from the extracted text to form a structured text data set; finally, storing the structured text data set in a predetermined MYSQL database to form the green knowledge base.

[0027] Further, the generation process of the sentence pair training data set in S2 includes: calling the DeepSeek-R1-671B large model to rewrite the sentence in the green knowledge base to generate positive samples of sentence pairs; and through random sampling technology, matching multiple semantically unrelated sentences as negative samples for each positive sample, and the number of negative samples is more than that of positive samples, to build a training data set with a predetermined positive and negative sample ratio.

[0028] Further, the fine-tuning training process is: inputting the sentence pair training data set into a Sentence-Transformer model based on SIMCSE, and performing a predetermined number of iterations of training; during the training process, an InfoNCE-based contrastive learning loss function is applied to optimize the model parameters, and the Sentence-Transformer model generates a vector space through an encoder, and the vector space includes positive sample pair vectors and negative sample pair vectors. Figure 2 A flowchart of a SIMCSE model provided for an embodiment of the application is shown in FIG. 1. 、 represents input; represents an encoder; represents an encoder; represents a positive sample; represents a negative sample; is used to generate a vector; represents a cosine similarity; 0.02 and the like represent respective cosine similarity values; 、 represents an output vector; Further, the loss function is used to maximize the cosine similarity between the positive sample pair vectors, and simultaneously minimize the cosine similarity between the negative sample pair vectors.

[0029] Further, the context acquisition process includes: performing similarity retrieval on the query vector and the set of encoded green vectors in the green KV vector library, retrieving the pre-sequence vectors with similarity greater than a preset threshold, and associating the text corresponding to the pre-sequence vectors to form a context.

[0030] The embodiment is based on a green knowledge base, generates positive and negative sample pairs through a large model, fine-tunes on a SIMCSE model base, trains a green knowledge retrieval model, batch-encodes the structured description of the green knowledge base to generate Embedding vectors, stores the Embedding vectors in a Cassandra index database; stores the generated text description and the corresponding Embedding vectors in the format of key-value pairs to generate a green behavior KV vector library; when receiving a user problem query, first encodes the user problem query using the green knowledge retrieval model to generate a query vector, and obtains a context based on the query vector and the green KV vector library; concatenates the user query and the context to form an enhanced prompt word, and inputs it into a DeepSeek-R1-671B large model for reasoning to generate a final answer, forming a closed-loop decision chain, thereby further realizing the cross-empowerment of artificial intelligence technology in the field of green and low-carbon.

[0031] Further, the green large model application process is a set of AI Agent application processes based on RAG enhanced retrieval constructed in combination with the green knowledge retrieval model. First, based on the RAG enhanced retrieval process, the text related to the user input Query is retrieved from the green KV vector library in order, so as to be spliced into the prompt word with the Context background and input into the Deepseek-R1-671B large model for reasoning; and then the final Answer is generated and returned to the user, thereby completing the closed loop of the entire application process.

[0032] The green large model reasoning technology process designed according to the characteristics of green knowledge fills the blank of the current large model in the application of the green low-carbon field, thereby further realizing the cross empowerment of artificial intelligence technology to the green low-carbon field.

[0033] In the embodiment, the text data related to green life and green behavior is obtained through automatic crawler technology; the structured green behavior and description are extracted through the Deepseek-R1-671B large model and stored in the MYSQL database to obtain the green knowledge base; the data is labeled through the Deepseek-R1-671B large model and labeled as a sentence pair training set; the sentence pair training data set is fine-tuned through the Sentence-Transformer model; the green knowledge retrieval model is obtained; the green behavior text in the green knowledge base is pre-encoded to obtain the encoding vector and stored in the Cassandra index database to obtain the green behavior KV vector library; the green large model application process based on the RAG enhanced retrieval of the large model is constructed based on the green knowledge retrieval model, the technical framework combines the core technology of the large model with green knowledge, adopts the technology of large model RAG and Agent, and realizes the cross empowerment of artificial intelligence technology to the green low-carbon field.

[0034] Although the embodiments of the present application have been shown and described, it can be understood by those of ordinary skill in the art that various changes, modifications, replacements and variations can be made to the embodiments without departing from the principles and spirits of the present application, and the scope of the present application is defined by the appended claims and their equivalents.

Claims

1. A method for constructing a green large model based on behavior prompt retrieval enhanced technology, characterized in that, Comprise: S1. Unstructured text data related to green life and behavior is crawled from various Internet platforms through a distributed crawler framework, structured green behavior and description are extracted through DeepSeek-R1-671B large model, structured text data set is formed and input into database, and then green knowledge base is obtained; S2. The sentence pair training data set is labeled by the DeepSeek-R1-671B large model; the sentence pair training data set is fine-tuned by the Sentence-Transformer model; and the green knowledge retrieval model is obtained; S3. The structured description of the green knowledge base is batch encoded to generate Embedding vectors, which are stored in the Cassandra index database; the generated text description and the corresponding Embedding vectors are stored in the form of key-value pairs to generate the green behavior KV vector library; S4. When receiving a user query, first encode the user query using the green knowledge retrieval model to generate a query vector, and obtain the context based on the query vector and the green KV vector library; The user query and the context are spliced to form an enhanced prompt word, which is input into the DeepSeek-R1-671B large model for reasoning to generate a final answer, forming a closed-loop decision chain, and constructing a green large model application process based on RAG enhanced retrieval based on the large model based on the green knowledge retrieval model.

2. The green large model construction method based on behavior prompt retrieval enhancement technology according to claim 1, characterized in that: The specific steps of S1 are: first, unstructured text is crawled from government websites, microblogs, Douyin, Xiaohongshu and public accounts based on the Scrapy framework of Python; then, the DeepSeek-R1-671B large model is called to extract knowledge from the crawled text to form a structured text data set; finally, the structured text data set is stored in a predetermined MYSQL database to constitute the green knowledge base.

3. The green large model construction method based on behavior prompt retrieval enhancement technology according to claim 1, characterized in that: The generation process of the sentence pair training data set in S2 includes: calling the DeepSeek-R1-671B large model to rewrite the sentence in the green knowledge base to generate positive samples of sentence pairs; and through random sampling technology, multiple semantically unrelated sentences are matched for each positive sample as negative samples, and the number of negative samples is more than that of positive samples, to build a training data set with a predetermined positive and negative sample ratio.

4. The green large model construction method based on behavior prompt retrieval enhancement technology according to claim 1, characterized in that: The process of fine-tuning training is: inputting the sentence pair training data set into a Sentence-Transformer model based on SIMCSE for a predetermined number of iterations of training; During training, the InfoNCE-based contrastive learning loss function is applied to optimize the model parameters, and the Sentence-Transformer model generates a vector space through an encoder, which includes positive sample pair vectors and negative sample pair vectors.

5. The green large model construction method based on behavior prompt retrieval enhancement technology according to claim 4, characterized in that: The loss function is used to maximize the cosine similarity between positive sample pair vectors and minimize the cosine similarity between negative sample pair vectors.

6. The green large model construction method based on behavior prompt retrieval enhancement technology according to claim 1, characterized in that: The acquisition process of the context includes: performing similarity retrieval on the query vector and the set of encoded green vectors in the green KV vector library, retrieving a previous sequence vector with similarity greater than a preset threshold, and combining a text corresponding to the previous sequence vector to form a context.

7. The green large model construction method based on behavior prompt retrieval enhancement technology according to claim 1, characterized in that: The green large model application process is an AI Agent application process based on RAG enhanced retrieval constructed in combination with the green knowledge retrieval model. First, based on the RAG enhanced retrieval process, a text sequence related to the user input Query is retrieved from the green KV vector library, thereby splicing a prompt word with a Context background into the Deepseek-R1-671B large model for reasoning; and then generating a final Answer answer returned to the user, completing the closed loop of the entire application process.

Images