Retrieval enhancement generation improvement method based on Embedded-FineTuning

CN119621896AActive Publication Date: 2025-03-14STATE GRID FUJIAN ELECTRIC POWER RES INST +1
View PDF 3 Cites 4 Cited by

Patent Information

Application Number
CN202411689335.9
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-11-25
Publication Date
2025-03-14
Estimated Expiration
2044-11-25

AI Technical Summary

Technical Problem

In the field of power grids, existing large language models may experience ‘illusion’ problems when generating responses based on lack of information, and traditional fixed-length text division methods cannot adapt to the complex structure of different types of documents, resulting in information loss or redundancy, and reducing processing efficiency and accuracy.

Method used

The search enhancement generation improvement method based on Embedding-FineTuning is adopted. By pre-processing and cleaning large-scale multi-source heterogeneous data in the power grid field, dynamic text division is used to use LumberChunker technology to generate text blocks suitable for fine-tuning training, and multiple rounds of Q&A Q&A on the T5 model are generated. Customized propt templates are designed and Embedding-FineTuning fine-tuning training is carried out to improve the embedded representation performance and accuracy of the model in the power grid field.

Benefits of technology

The performance and accuracy of large language models in the application scenarios of Q&A in the power grid field are improved, reliable reasoning of Q&A in the power grid field is realized, practicality and reliability of the model are enhanced, and search results with high accuracy are provided.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119621896A_ABST
    Figure CN119621896A_ABST
Patent Text Reader

Abstract

The invention provides an improved retrieval enhancement generation method based on Embedded-FineTuning, which is used for preprocessing and cleaning large-scale multi-source heterogeneous data of a power grid to ensure the consistency and quality of the data, and dynamically dividing the processed text data based on a LumberChunker technology, dividing text blocks according to semantic turning points, and generating text blocks suitable for fine tuning training; then, performing question and answer pair generation of multi-round questions and answers on the basis of the text block by adopting a T5 model to form a question and answer pair data set for fine tuning; the method comprises the following steps of: designing a customized prompt template for an actual application scene in the field of an electric power grid to guide a model generation process; embedded-FineTuning fine tuning training is carried out on a question and answer pair data set, and model parameters are optimized, so that the embedded representation performance and accuracy of the model in the field of electric power grids are improved.
Need to check novelty before this filing date? Find Prior Art

Citation Information

Patent Citations

  • Knowledge base construction method and question and answer dialogue method and system based on generative large language model

    CN117056471A

  • Domain question-answering system, domain question-answering construction method, electronic equipment and storage medium

    CN117909466A

  • Method and apparatus for self-supervised extractive question answering

    WO2023098971A1

Cited By

  • Method for generating fine-tuning question and answer data set based on retrieval enhancement

    CN120745855A

  • Large model fine tuning method and system based on power distribution network dispatching

    CN121094117A

  • Water resource scheduling instruction fine tuning data set construction method and device, equipment and medium

    CN121745319A

  • Large model corpus construction and fine adjustment method and device based on water conservancy data warehouse

    CN122285655A