Intelligent question-answering method, system, medium and device for infant growth and development

By combining LoRA fine-tuning and retrieval enhancement generation technology of pre-trained large models, an intelligent question-and-answer system is built for the growth and development of infants and young children, solving the problems that existing systems are difficult to understand complex queries and provide personalized answers, and achieving professionalism and privacy-protected Q&A system improves the quality of response and user experience.

CN118585635BActive Publication Date: 2024-11-19SHANDONG UNIV
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411073742.7
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-08-07
Publication Date
2024-11-19
Estimated Expiration
2044-08-07

AI Technical Summary

Technical Problem

The existing question and answer systems lack natural language processing capabilities, make it difficult to understand user complex queries, the answers provided lack personalization and depth, and lack professional knowledge base support for the growth and development of infants and young children, so they cannot provide professional guidance.

Method used

The pre-trained large models are fine-tuned by LoRA fine-tuning technology, combined with search enhancement generation technology, relying on professional field knowledge bases to design intelligent question-and-answer interaction systems, and the front-end communication between the back-end is achieved through the Flask framework, using the professional knowledge base and Open source embedding datasets improve the professionalism and answer quality of the model.

Benefits of technology

It realizes the provision of professional and authoritative answers for the growth and development of infants and young children while maintaining user privacy, significantly improving the professionalism and response quality of the question-and-answer system, and enhancing the efficiency of user interaction experience and information acquisition.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118585635B_ABST
    Figure CN118585635B_ABST
Patent Text Reader

Abstract

The present invention belongs to the field of intelligent question-answering systems, and specifically relates to an intelligent question-answering method, system, medium and device for the growth and development of infants and young children. The Flask framework is used to develop an intuitive and easy-to-operate front-end user interface to ensure that non-professional users can easily control it. Each user query is quickly transmitted to the model, and the personalized and accurate response generated by the model is immediately fed back to the user, forming a smooth and interactive question-answering closed loop, greatly improving the user experience and information acquisition efficiency. By adopting advanced fine-tuning strategies and effective retrieval enhancement technology, the present invention has successfully developed a professional-level intelligent question-answering system, which not only significantly improves the accuracy and efficiency of questions and answers, but also pays attention to user privacy protection, is applicable to various real application scenarios, and provides reliable knowledge services and solutions.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the field of intelligent question-answering systems, and specifically relates to an intelligent question-answering method, system, medium and equipment for the growth and development of infants and young children. Background Art

[0002] The knowledge in the field of infant growth and development is updated rapidly, and the knowledge that needs to be processed is relatively complex; many current question-answering systems lack natural language processing capabilities, making it difficult to understand users' complex queries, and the answers they provide lack personalization and depth. In addition, there is a lack of professional knowledge base support in the field of infant growth and development, and it is impossible to provide professional guidance. Therefore, it is particularly important to develop a professional question-answering system that can provide professional and personalized consulting services based on the latest medical research, child development theory, and scientific guidance programs. Summary of the invention

[0003] In order to make up for the defects in the prior art, the present invention uses LoRA fine-tuning technology and retrieval enhancement generation technology, relying on the professional knowledge base in the field to overcome the above obstacles. Specifically, the intelligent question-answering system for the large model in the field of infant growth and development relies on the professional knowledge base in the field, and uses LoRA fine-tuning technology to fine-tune the existing pre-trained large model. In addition, in order to improve the professionalism of the question-answering system, the input questions are searched and enhanced. In addition, in order to solve the problem of user interactivity, a front-end and back-end interaction form is formed, and the front-end page is designed to interact with the user. Its technical solution is:

[0004] An intelligent question-answering method for infant growth and development comprises the following steps:

[0005] S1: Build a professional knowledge base in the field of infant growth and development;

[0006] S2: Based on the pre-trained large model Baichuan2-7B-Chat, deep fine-tuning is performed through low-rank adaptation technology;

[0007] S3: Incorporating retrieval enhancement generation technology RAG, using prompts as a guide, effectively retrieving relevant information in the knowledge base and assisting the model reasoning process;

[0008] S4: Use the Flask framework to build the front-end user interface and use standardized JSON format data exchange to achieve communication between the front-end and the back-end.

[0009] Preferably, in step S1, knowledge on infant growth and development covering multiple dimensions of nutrition, health, safety and security, and early learning is systematically extracted and integrated from the World Health Organization guidelines, child health monographs, Chinese population physical activity guidelines, and the Merck Manual of Diagnosis and Treatment, to construct a professional knowledge base in the field of infant growth and development.

[0010] Preferably, in step S1, the professional knowledge base is organized by corresponding professionals based on various authoritative knowledge bases, and finally integrated into a question-and-answer format.

[0011] Preferably, in step S2, the steps of fine-tuning the pre-trained large model using LoRA technology are as follows:

[0012] S21. First, obtain the pre-trained model Baichuan2-7B-Chat, denoted as , use the professional knowledge base obtained in S1 as the fine-tuning dataset and set the corresponding fine-tuning parameters,

[0013] S22. If the current round number is less than the set round number , proceed to the next step, otherwise proceed to step S25;

[0014] S23. Continue fine-tuning and increase the number of rounds by 1;

[0015] S24. Enter S22;

[0016] S25. Get the tensor and corresponding parameters after LoRA fine-tuning, and the previous pre-trained model Merge to get the fine-tuned model ;

[0017] S26. End.

[0018] Preferably, in step S3, the steps of using the retrieval enhancement generation technology to enhance the fine-tuned large model obtained in S2 are as follows:

[0019] S31. Use the professional knowledge base obtained in S1 as the RAG dataset, and use the open source embedding dataset bge-large-zh-v1.5 as the embedding library dataset for the retrieval enhancement generation technology to transform the natural language text into a dense vector in a high-dimensional vector space to capture the intrinsic connection at the semantic level;

[0020] S32. Obtaining user input , convert it into a vector with the same dimension as the entry in the professional knowledge base, ensuring that the two can be effectively compared in the same semantic space;

[0021] S33.According to The embedded content of is matched with all text vectors in the RAG dataset to find the text entries corresponding to the top 10 most similar vectors;

[0022] S34. extracting the most relevant text item retrieved into a refined hint H, which not only summarizes the key knowledge points but also is closely connected with the original user question;

[0023] S35. Together with H, it is passed as the final input to ;

[0024] S36. After processing the final input, the model, with its powerful language understanding and generation capabilities, is able to generate detailed, coherent, and accurate answers to user questions based on a full understanding of the prompt content.

[0025] Preferably, the steps of using the Flask framework to use the large model obtained from S3 after fine-tuning and retrieval enhancement as the backend and designing the front-end page to implement the intelligent question-answering system are as follows:

[0026] S41. Use the front-end input box to obtain the user's question input;

[0027] S42. The acquired input is converted into JSON data format using the Flask framework and transmitted to the backend server;

[0028] S43. The backend server obtains the input from the frontend and transmits the corresponding input to the fine-tuned and retrieval-enhanced model for reasoning;

[0029] S44. Convert the inference output into JSON data format and transmit it to the front-end page;

[0030] S45. The front-end page receives the output from the back-end, designs the corresponding animation, and displays it to the user in the form of animation.

[0031] Preferably, in step S21, the learning rate is set to 1e-4 to ensure that the model converges stably to the optimal solution; β1 is set to 0.9 and β2 is set to 0.98 in the Adam optimizer to control the optimization, and the current round number is set to 1.

[0032] An intelligent question-answering system for infant growth and development, comprising a professional knowledge base module; an information processing module and an output module;

[0033] Professional knowledge base module: systematically extracts and integrates knowledge on infant growth and development in multiple dimensions, including nutrition, health, safety and security, and early learning, from the World Health Organization guidelines, child health monographs, Chinese population physical activity guidelines, and the Merck Manual of Diagnosis and Treatment, to build a professional knowledge base in the field of infant growth and development;

[0034] Information processing module: Based on the open source pre-trained large model Baichuan2-7B-Chat, it is deeply fine-tuned through low-rank adaptation technology, integrated with retrieval enhancement generation technology, and uses carefully designed prompts as guidance to associate with user input information, effectively retrieve related information in the knowledge base, and assist the model reasoning process;

[0035] Output module: Use the Flask framework to build the front-end user interface, and use standardized JSON format data exchange to achieve communication between the front-end and the back-end.

[0036] A computer device comprises at least one processor and a computer-readable medium storing a computer program, wherein the computer program implements the method of the present application when read and executed by the processor.

[0037] A computer-readable medium stores a computer program, and when the computer program is read and executed by a processor, the method of the present application is implemented.

[0038] Compared with the prior art, the advantages and benefits of the present invention are summarized as follows:

[0039] 1. The present invention is committed to using large-scale pre-trained models for specific fields to build an intelligent question-and-answer system to achieve the ability to provide professional and authoritative answers to questions in the field of infant growth and development in actual situations while maintaining user privacy. Traditional question-and-answer systems are often limited to providing answers to common sense and are unable to meet the consulting needs in professional fields. Furthermore, in addition to the possibility of personal privacy leakage, traditional physical medical treatment channels are also subject to various physical conditions. Although the fine-tuned large-scale model shows good performance in specific fields, it fails to achieve optimal performance due to the limited scale and professionalism of the adopted database. The present invention introduces retrieval enhancement generation technology to deepen the processing of the questions raised, and adopts an intuitive user interface design to promote human-computer communication.

[0040] 2. Design a general intelligent question-answering system architecture for large models in the field of infant growth and development, aiming to overcome the limitations of current question-answering systems in terms of professionalism and privacy protection. The system uses a small amount of highly targeted data sets to adjust the pre-trained model, prompting the model to show a high level of professionalism in a specific field. Combined with retrieval enhancement generation technology, the information retrieval function of the model is further strengthened, thereby significantly improving the quality of answers.

[0041] 3. Develop a user interface design to enhance the interactive experience, ensure that users can communicate with the system conveniently, and achieve a more humane interaction mode. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1It is a schematic diagram of the process of the present invention.

[0043] Figure 2 Flowchart for fine-tuning a pre-trained large model using LoRA technology.

[0044] Figure 3 The process of using retrieval enhancement generation technology to perform retrieval enhancement on a fine-tuned large model.

[0045] Figure 4 Flowchart of the interaction between the intelligent question answering system and users. DETAILED DESCRIPTION

[0046] In order to make the purpose, technical solutions and advantages of the present invention clearer, the present invention is further described in detail below in conjunction with the accompanying drawings and examples. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention. For those skilled in the art, the present invention may have various changes and variations. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention. In addition, the technical features involved in each embodiment of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0047] This paper designs an intelligent question-answering method for infant growth and development. The process is as follows: Figure 1 As shown, it mainly includes the following steps:

[0048] 1. From authoritative sources such as the World Health Organization (WHO) guidelines, child health monographs, the "Physical Activity Guidelines for the Chinese Population" and the "Merck Manual of Diagnosis and Treatment", we systematically extract and integrate infant and young child growth and development knowledge covering multiple dimensions such as nutrition, health, safety and early learning. This professional and detailed knowledge base constitutes the core information cornerstone of the intelligent question-and-answer system, ensuring that the output content is both scientific and practical.

[0049] In this step, professionals in the field first collect relevant information in different directions and organize it into a data set in the form of question and answer, which is also in line with the real-life scenarios of users interacting with question and answer systems in actual applications. The data set is then converted into a JSON file format and further organized into a data set that can be directly used by the large model.

[0050] Second: Relying on the open source pre-trained large model Baichuan2-7B-Chat as the foundation, deep fine-tuning is performed through low-rank adaptation (LoRA) technology. During this process, most pre-training parameters are cleverly frozen, and only key low-dimensional matrices are updated to enhance the model's understanding of the unique context and professional terms in the field of infant growth and development, and achieve accurate responses to questions in the field. The fine-tuning process is as follows Figure 2 As shown, specifically including:

[0051] (1) First obtain the pre-trained model M , use the professional knowledge base obtained in step 1 as the fine-tuning dataset, set the corresponding fine-tuning parameters, such as setting the learning rate to 1e-4 to ensure that the model can stably converge to the optimal solution; set β1 to 0.9 and β2 to 0.98 in the Adam optimizer to control the optimization, and set the current round number to 1;

[0052] (2) If the current round number is less than the set round number , go to the next step, otherwise go to step (5);

[0053] (3) Continue to fine-tune LoRA. The model needs to be further fine-tuned according to the set fine-tuning parameters, so the number of rounds is increased by one;

[0054] (4) Enter (2);

[0055] (5) Obtain the tensors and corresponding parameters after LoRA fine-tuning, and compare these parameters and tensors with the original pre-trained model M Carry out a merger;

[0056] (6) Finally, we obtain a model that provides professional answers in the field of infant growth and development after fine-tuning by LoRA. .

[0057] This step mainly explains how to use the LoRA fine-tuning technology to fine-tune the pre-trained large model in combination with the collected training set.

[0058] 3. Based on the fine-tuning model, the Retrieval Augmented Generation (RAG) technology is creatively integrated. With the help of carefully designed prompts as guidance and complementary to user input information, this technology can effectively retrieve related information in the knowledge base, assist the model reasoning process, significantly improve the quality of responses and knowledge coverage, and ensure that the output information is accurate and extensive.

[0059] Figure 3The process of using retrieval enhancement generation technology to perform retrieval enhancement on a fine-tuned large model is described. The specific steps are as follows:

[0060] (1) Use the professional knowledge base obtained in step 1 as the RAG dataset, and use the open source embedding dataset bge-large-zh-v1.5 as the embedding library dataset for the retrieval enhancement generation technology. The purpose is to convert natural language text into dense vectors in a high-dimensional vector space to capture the intrinsic connections at the semantic level.

[0061] (2) Obtaining user input , convert it into a vector with the same dimension as the entry in the knowledge base, ensuring that the two can be effectively compared in the same semantic space;

[0062] (3) Based on The embedded content is matched with all text vectors in the RAG dataset to find the text entries corresponding to the top 10 most similar vectors. This process is equivalent to accurately locating in a large-scale knowledge base to find the reference information most relevant to the user's question;

[0063] (4) Distill the most relevant text items retrieved into a refined prompt H , the prompt not only summarizes the key knowledge points, but is also closely linked to the original user question. This prompt design is intended to guide the subsequent generation model to focus on relevant domain knowledge and improve the quality and pertinence of the answer;

[0064] (5) and H As the final input I Pass to ;

[0065] (6) Processing final input I ,With its powerful language understanding and generation capabilities, the model can generate detailed, coherent and accurate answers to user questions based on a full understanding of the prompt content.

[0066] This step mainly explains how to use retrieval enhancement generation technology to perform retrieval enhancement on the large model after domain fine-tuning.

[0067] Four: Using the Flask framework, we developed an intuitive and easy-to-use front-end user interface to ensure that non-professional users can easily use it. This interface is seamlessly connected with the optimized back-end model, and with the help of standardized JSON format data exchange, efficient communication between the front-end and the back-end is achieved. Each user query is quickly transmitted to the model, and the personalized and accurate response generated by the model is immediately fed back to the user, forming a smooth and interactive question-and-answer closed loop, greatly improving the user experience and information acquisition efficiency.

[0068] This step mainly explains how the large model after retrieval enhancement and LoRA fine-tuning finally forms an intelligent question-answering system to interact with users. The overall process of the intelligent question-answering system interacting with users is as follows: Figure 4 As shown, the specific steps are as follows:

[0069] This step mainly explains how to use retrieval enhancement generation technology to perform retrieval enhancement on the large model after domain fine-tuning.

[0070] (1) Use the front-end input box and front-end prompts to prompt users to enter legal data in the specified location and obtain user input;

[0071] (2) Using the features of the Flask framework, the user input is converted into JSON data format, and the corresponding data is converted into a format that can be directly recognized and used by the backend, and then transmitted to the backend server;

[0072] (3) The backend server obtains the input from the frontend and passes the corresponding input to the fine-tuned and retrieval-enhanced model for reasoning, obtaining professional and authoritative results;

[0073] (4) The model converts the inference output into JSON data format and transmits it to the front-end page, and converts the data into a form that the front-end can recognize, so that it can be displayed and used by the front-end;

[0074] (5) The front-end page receives the output from the back-end, designs the corresponding typewriter animation, simulates the typewriter form, and displays the results of professional model reasoning after LoRA fine-tuning and retrieval enhancement to the user in an animated manner.

[0075] An intelligent question-answering system for infant growth and development, comprising a professional knowledge base module, an information processing module and an output module;

[0076] Professional knowledge base module: systematically extracts and integrates knowledge on infant growth and development in multiple dimensions, including nutrition, health, safety and security, and early learning, from the World Health Organization guidelines, child health monographs, Chinese population physical activity guidelines, and the Merck Manual of Diagnosis and Treatment, to build a professional knowledge base in the field of infant growth and development;

[0077] Information processing module: Based on the open source pre-trained large model Baichuan2-7B-Chat, it is deeply fine-tuned through low-rank adaptation technology, integrated with retrieval enhancement generation technology, and uses carefully designed prompts as guidance to associate with user input information, effectively retrieve related information in the knowledge base, and assist the model reasoning process;

[0078] Output module: Use the Flask framework to build the front-end user interface, and use standardized JSON format data exchange to achieve communication between the front-end and the back-end.

[0079] A computer device comprises at least one processor and a computer-readable medium storing a computer program, wherein the computer program implements the method of the present application when read and executed by the processor.

[0080] A computer-readable medium stores a computer program, and when the computer program is read and executed by a processor, the method of the present application is implemented.

[0081] The above embodiments are only exemplary embodiments of the present invention and are not intended to limit the present invention. The protection scope of the present invention is defined by the claims. Those skilled in the art may make various modifications or equivalent substitutions to the present invention within the essence and protection scope of the present invention, and such modifications or equivalent substitutions shall also be deemed to fall within the protection scope of the present invention.

Claims

1. An intelligent question-answering method for infant growth and development, It is characterized in that The following steps are involved: S1: Build a professional knowledge base in the field of infant growth and development; S2: Based on the pre-trained large model Baichuan2-7B-Chat, deep fine-tuning is performed through low-rank adaptation technology; The steps to fine-tune the pre-trained large model using LoRA technology are as follows: S21. First, obtain the pre-trained model Baichuan2-7B-Chat, denoted as M, use the professional knowledge base obtained in S1 as the fine-tuning data set, set the corresponding fine-tuning parameters, set the learning rate to 1e-4, and ensure that the model converges stably to the optimal solution; set β1 to 0.9 and β2 to 0.98 in the Adam optimizer to control the optimization, and set the current round number to 1; S22. If the current round number is less than the set round number R m , proceed to the next step, otherwise proceed to step S25; S23. Continue fine-tuning and increase the number of rounds by 1; S24. Enter S22; S25. Get the tensor and corresponding parameters after LoRA fine-tuning, merge them with the previous pre-trained model M, and get the fine-tuned model M lora ; S3: Incorporating retrieval enhancement generation technology RAG, using prompts as a guide, effectively retrieving relevant information in the knowledge base and assisting the model reasoning process; S31. Use the professional knowledge base obtained in S1 as the RAG dataset, and use the open source embedding dataset bge-large-zh-v1.5 as the embedding library dataset for the retrieval enhancement generation technology to transform the natural language text into a dense vector in a high-dimensional vector space to capture the intrinsic connection at the semantic level; S32. Obtaining user input I origin , convert it into a vector with the same dimension as the entry in the professional knowledge base, ensuring that the two can be effectively compared in the same semantic space; S33. According to I origin The embedded content of is matched with all text vectors in the RAG dataset to find the text entries corresponding to the top 10 most similar vectors; S34. extracting the most relevant text item retrieved into a hint H, which not only summarizes the key knowledge points but also is closely related to the original user question; S35. I origin Together with H, it is passed to M as the final input lora ; S36.M lora Processing to generate answers to user questions; S4: Use the Flask framework to build the front-end user interface and use standardized JSON format data exchange to achieve communication between the front-end and the back-end; The steps to design the front-end page to implement the intelligent question-answering system are as follows: S41. Use the front-end input box to obtain the user's question; S42. The acquired input is converted into JSON data format using the Flask framework and transmitted to the backend server; S43. The backend server obtains the input from the frontend and transmits the corresponding input to the fine-tuned and retrieval-enhanced model for reasoning; S44. Convert the inference output into JSON data format and transmit it to the front-end page; S45. The front-end page receives the output from the back-end, designs the corresponding animation, and displays it to the user in the form of animation.

2. The intelligent question-answering method for infant growth and development according to claim 1, It is characterized in that In step S1, we systematically extract and integrate knowledge on infant growth and development covering multiple dimensions of nutrition, health, safety and early learning from the World Health Organization guidelines, child health monographs, Chinese population physical activity guidelines, and the Merck Manual of Diagnosis and Treatment, and build a professional knowledge base in the field of infant growth and development.

3. The intelligent question-answering method for infant growth and development according to claim 1, It is characterized in that In step S1, the professional knowledge base is organized by relevant professionals based on various authoritative knowledge bases and finally integrated into a question-and-answer format.

4. A system for the intelligent question-answering method for infant growth and development according to any one of claims 1 to 3, It is characterized in that It includes professional knowledge base module, information processing module and output module; Professional knowledge base module: systematically extracts and integrates knowledge on infant growth and development in multiple dimensions, including nutrition, health, safety and security, and early learning, from the World Health Organization guidelines, child health monographs, Chinese population physical activity guidelines, and the Merck Manual of Diagnosis and Treatment, to build a professional knowledge base in the field of infant growth and development; Information processing module: Based on the open source pre-trained large model Baichuan2-7B-Chat, it is deeply fine-tuned through low-rank adaptation technology, integrated with retrieval enhancement generation technology, and uses prompts as guidance to associate with user input information, effectively retrieve related information in the knowledge base, and assist the model reasoning process; Output module: Use the Flask framework to build the front-end user interface, and use standardized JSON format data exchange to achieve communication between the front-end and the back-end.

5. A computer device, It is characterized in that The invention comprises at least one processor and a computer-readable medium storing a computer program, wherein when the computer program is read and executed by the processor, the method according to any one of claims 1 to 3 is implemented.

6. A computer readable medium, It is characterized in that The computer-readable medium stores a computer program, and when the computer program is read and executed by a processor, the method according to any one of claims 1 to 3 is implemented.