Chinese dental intelligent diagnosis and treatment question and answer method, electronic device and storage medium

Abstract

The present application belongs to the technical field of natural language processing and intelligent medical treatment, and discloses a Chinese dental intelligent diagnosis and treatment question and answer method, an electronic device and a storage medium. The dental intelligent diagnosis and treatment question and answer method comprises the following steps: acquiring real dental doctor-patient conversation data and preprocessing to obtain a dental doctor-patient conversation data set with consistent style; constructing a high-efficiency fine-tuned local open-source Chinese large language model; acquiring and constructing a dental doctor information knowledge base of a local hospital; constructing a professional information knowledge base for dental diseases; connecting the constructed dental doctor information knowledge base and the professional information knowledge base according to the fine-tuned local large model to realize a dental diagnosis and treatment inquiry system. The present application realizes the answering ability of the local large model to difficult and complicated diseases and the suggestion and recommendation of corresponding detailed doctor information to users by introducing two local professional knowledge bases into the fine-tuned local large model.

Description

Technical Field

[0001] This invention belongs to the field of natural language processing technology, and in particular relates to a method, electronic device and storage medium for intelligent Chinese dental diagnosis and treatment question answering. Background Technology

[0002] Since the emergence of question-answering large models based on GPT-3.5, generative large models have demonstrated their powerful versatility across many tasks and have been widely applied. Currently, how to leverage the advantages of general-purpose large models and combine them with the accumulated professional knowledge and business process standards of various vertical industries and fields to create domain-specific large models has become a major direction and trend in the exploration of large model application.

[0003] In the field of dental care, a generative large-scale model-based dental Q&A assistant can help answer users' dental care-related questions, providing personalized answers and suggestions. It interacts with users in natural language, understands their questions, and generates appropriate and detailed answers, providing information on symptoms, diseases, treatment plans, medication side effects, and health management. Furthermore, the assistant can offer personalized suggestions and recommendations based on the user's personal health information and description of their condition, helping users better manage and improve their health.

[0004] However, directly applying general large models to the field of dental treatment has the following limitations:

[0005] (1) Lack of specialized domain knowledge: Medical application scenarios have more stringent requirements for the output results of large models. In dental diagnosis and treatment, doctors need to comprehensively consider factors such as the patient's personal situation, medical history, and clinical manifestations to make appropriate diagnostic and treatment decisions. However, general large models are trained using massive amounts of general data and lack specialization for dental diagnosis and treatment problems.

[0006] (2) Sensitivity and privacy issues: The dental treatment field involves patients' personal health information and privacy. Ensuring data security and privacy protection is an important issue. When uploading to cloud services or fine-tuning online large model technology, data security issues such as leakage of personal privacy may occur.

[0007] (3) Low data timeliness and high training cost: When injecting new knowledge into a large model, it is necessary to reconstruct the dataset and retrain the model, resulting in low data timeliness.

[0008] (4) Data scarcity: General-purpose large models usually require a large amount of data for training in order to understand and solve various problems. In the field of dental treatment, due to the limited data resources, it may be difficult to obtain enough dental data to train these models. Existing doctor-patient dialogue data also suffers from problems such as inconsistent data formats and inconsistent language used by doctors when answering, which directly affects the quality of medical assistants' responses. Summary of the Invention

[0009] To address the problems existing in the prior art, this invention provides a method, electronic device, and storage medium for Chinese dental intelligent diagnosis and treatment question answering, aiming to solve the problem of low quality of intelligent answers from dental medical models caused by the above four limitations.

[0010] To achieve the above objectives, the present invention provides a Chinese-language intelligent dental consultation and question-and-answer method, comprising the following steps:

[0011] Step 1: Obtain real dental doctor-patient dialogue data and preprocess it to unify the response style of the collected data, resulting in a dental doctor-patient dialogue dataset with a consistent style.

[0012] Step 2: Construct a local open-source Chinese language model with fine-tuned parameters.

[0013] Step 3: Obtain and build a knowledge base of dentists' information from local hospitals.

[0014] Step 4: Build a professional information knowledge base for dental conditions.

[0015] Step 5: Based on the locally open-source Chinese language model with fine-tuned parameters, connect the constructed dentist information knowledge base and professional information knowledge base to realize intelligent Chinese dental diagnosis and treatment question and answer.

[0016] Furthermore, in step one, the collected real dental doctor-patient dialogue data is preprocessed to obtain a preprocessed dental doctor-patient dialogue dataset, including:

[0017] Due to the unique nature of dental doctor-patient dialogues, and the fact that these dialogues primarily revolve around professional knowledge related to dental disease diagnosis, treatment, and medications, the quality of dental doctor-patient dialogues collected online varies greatly. Some data may contain issues such as non-standard language, lack of content, and inconsistent terminology among different patients and doctors, which can affect data quality. Therefore, preprocessing of the collected real dental doctor-patient dialogue data is necessary: ​​anonymizing personal privacy information and rewriting the data in a unified style. Specifically, based on the different medical consultation needs of different users or patients in the real world, the doctor's responses are reconstructed using ChatGPT to ensure that the responses demonstrate comprehensive explanations and a high level of empathy, forming a unified style dental doctor-patient dialogue dataset. Furthermore, the dialogue pairs are converted into JSON format question-answer pairs required for fine-tuning the data, forming the dental doctor-patient dialogue dataset needed for fine-tuning the large model.

[0018] Furthermore, in step two, a high-performance local open-source large language model is selected as the base model. An efficient parameter fine-tuning method is used to introduce dental expertise into the base model, improving the accuracy, human-likeness, and interactivity of the model's output. Specifically, the local open-source Chinese large language model is a deep learning-based natural language processing model capable of learning the syntax and semantics of natural language, thus generating human-readable text. A high-performance Chinese large model that can be deployed locally is selected as the base model.

[0019] The parameter fine-tuning method involves fine-tuning only a small number or additional model parameters while fixing most of the pre-trained parameters. Specifically, LoRA parameter fine-tuning is performed on the base model: all pre-trained parameters of the base model are kept unchanged, a dimensionality reduction matrix A and an increased dimensionality matrix B are introduced to form a bypass, and the parameters are learned and updated to inherit the capabilities gained in the pre-training stage. Furthermore, the knowledge gained in the pre-training stage can be quickly transferred to specific dental diagnosis and answering tasks, efficiently adapting to various downstream application tasks and greatly reducing computational and storage costs.

[0020] Furthermore, step three, which involves acquiring and constructing a knowledge base of dentist information from the local hospital, includes:

[0021] Information on dentists in local hospitals was obtained through web scraping. While respecting website rules and ensuring the legality of data scraping, the main information obtained included: dentists' profiles, departments, personal medical expertise, affiliated hospitals, doctor titles, patient reviews, and consultation information, forming a local doctor information document.

[0022] When building a knowledge base of dental information for local hospitals, it is necessary to perform data cleaning on the acquired information, specifically including handling data incompleteness and data redundancy:

[0023] Data incompleteness handling: For gender (binarized), age, and years of medical practice, the mode is used to complete the data; for graduating school, personal medical expertise, patient evaluations, etc., predefined default values ​​are used to complete the data.

[0024] Data redundancy handling: For gender (binarized), age, and years of medical practice, only the maximum value of multiple redundant data is retained; for graduating school, personal medical expertise, patient evaluations, etc., only the longest text of multiple redundant data is retained.

[0025] After data cleaning, local doctor information documents are generated. A suitable text-to-vector projection method is selected to perform text segmentation and text vectorization on the local doctor information documents after data cleaning, and store them locally as a doctor information knowledge base. For newly added local doctor information, after adding it to the local doctor information document, text segmentation and text vectorization projection are performed again, and the local doctor information knowledge base is updated.

[0026] Furthermore, the construction of a professional information knowledge base for dental conditions in step four includes:

[0027] Collect and organize the necessary professional journals on dental diseases to form local professional information documents. After summarizing and organizing the texts of several professional journals on difficult dental diseases, perform text segmentation and text vectorization, and store them in the local professional information knowledge base.

[0028] Furthermore, the locally sourced open-source Chinese language model with fine-tuned parameters in step five is connected to the constructed dentist information knowledge base and professional information knowledge base to achieve intelligent Chinese dental diagnosis and treatment question answering, including:

[0029] When using the system, two local external knowledge bases are available for selection: a dentist information knowledge base and a professional information knowledge base. The system provides an answer based on the input question, combining the reasoning capabilities of the large model with the two local knowledge bases. Specifically, when connecting to the local dentist information knowledge base, personalized suggestions and recommendations are provided based on the user's personal health information and description of their condition, using the knowledge base provided. When connecting to the professional information knowledge base, the system retrieves and generates a professional answer based on the user's professional question, combining the reasoning capabilities of the large model with the professional information knowledge base.

[0030] Another objective of this invention is to provide an electronic device that applies the aforementioned Chinese dental intelligent diagnosis and treatment question-and-answer method. The intelligent question-and-answer assistant electronic device includes a text acquisition device, a display, a graphics processor, a communication interface, a memory, a central processing unit, and a communication bus.

[0031] The text acquisition device, the display, the graphics processor, the communication interface, the memory, and the central processing unit communicate with each other through the communication bus.

[0032] A text input device is used to input text and enable interactive consultation.

[0033] A display is used to transmit dialogue data;

[0034] Graphics processor, used for inference calculations of large models;

[0035] Memory, used to store computer programs;

[0036] The text acquisition device is used to input text and realize interactive consultation;

[0037] The display is used to display dialogue data;

[0038] The graphics processor is used for inference calculations of large models;

[0039] The memory is used to store computer programs;

[0040] The central processing unit, when executing the computer program stored in the memory, implements the Chinese dental intelligent diagnosis and treatment question-and-answer method.

[0041] Another object of the present invention is to provide an electronic device comprising a memory and a processor, the memory storing a computer program, which, when executed by the processor, causes the processor to perform the following steps:

[0042] The system acquires and preprocesses dental doctor-patient dialogue data; utilizes a local open-source Chinese language model with efficient parameter fine-tuning; acquires and constructs a knowledge base of dentists' information from a local hospital; constructs a professional knowledge base for dental diseases; and connects the constructed dentist information knowledge base and professional knowledge base based on the fine-tuned local model to achieve intelligent Chinese dental diagnosis and treatment question answering.

[0043] Another object of the present invention is to provide a computer-readable storage medium storing a computer program, which, when executed by a processor, causes the processor to perform the following steps:

[0044] The system acquires and preprocesses dental doctor-patient dialogue data; utilizes a local open-source Chinese language model with efficient parameter fine-tuning; acquires and constructs a knowledge base of dentists' information from a local hospital; constructs a professional knowledge base for dental diseases; and connects the constructed dentist information knowledge base and professional knowledge base based on the fine-tuned local model to achieve intelligent Chinese dental diagnosis and treatment question answering.

[0045] Combining all the above technical solutions, the advantages and positive effects of this invention are as follows: The dental diagnosis and consultation assistant design method provided by this invention introduces two local professional knowledge bases into the locally fine-tuned Chinese large model, thereby enabling the local large model to answer difficult and complicated cases and to suggest and recommend relevant detailed doctor information to users. Attached Figure Description

[0046] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the embodiments of the present invention will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0047] Figure 1 This is a flowchart of the Chinese dental intelligent diagnosis and treatment question-and-answer method provided in an embodiment of the present invention.

[0048] Figure 2 This is a schematic diagram of the Chinese dental intelligent diagnosis and treatment question-and-answer method provided in an embodiment of the present invention.

[0049] Figure 3 This is a schematic diagram of the response style rewriting of the dental doctor-patient dialogue dataset provided in an embodiment of the present invention.

[0050] Figure 4 This is a schematic diagram of the LoRA fine-tuning structure of the large dental model provided in this embodiment of the invention.

[0051] Figure 5 This is a graph showing the decrease in loss during the fine-tuning process provided in an embodiment of the present invention.

[0052] Figure 6 This is a diagram showing the effect of the finely tuned model provided in this embodiment of the invention in reasoning and answering dental questions.

[0053] Figure 7 This is a schematic diagram of the Chinese dental intelligent diagnosis and treatment question-and-answer structure provided in an embodiment of the present invention.

[0054] Figure 8 This is a comparison chart showing the effect of combining the dentist's information knowledge base with reasoning and answering, provided in an embodiment of the present invention.

[0055] Figure 9This is a schematic diagram of the structure of the Chinese dental intelligent diagnosis and treatment question-and-answer electronic device provided in an embodiment of the present invention.

[0056] Figure 10 This is a schematic diagram of the structure of a computer-readable storage medium for Chinese dental intelligent diagnosis and treatment questions provided in an embodiment of the present invention. Detailed Implementation

[0057] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0058] To address the problems existing in the prior art, the present invention provides a method, electronic device, and storage medium for a Chinese-language medical dental consultation assistant for local hospitals. The present invention will be described in detail below with reference to the accompanying drawings.

[0059] like Figure 1 As shown, the Chinese dental intelligent diagnosis and treatment question-and-answer method provided in this embodiment of the invention includes the following steps:

[0060] S101, Obtain and construct a fine-tuning dataset of dental doctor-patient dialogues;

[0061] S102, building a local open-source Chinese large language model with efficient parameter fine-tuning;

[0062] S103, Obtain and build a knowledge base of dentist information from local hospitals;

[0063] S104, Acquire and build a professional information knowledge base for dental diseases, especially for related difficult and complicated diseases;

[0064] S105 connects the constructed dentist information knowledge base and professional information knowledge base based on the finely tuned local large model.

[0065] The significance of this invention lies in:

[0066] (1) Further fine-tuning and training were carried out using a large-scale medical knowledge base and real dental patient datasets so that the dental diagnosis and treatment question-answering assistant could generate more accurate, coherent and detailed answers to various dental medical questions raised by users.

[0067] (2) The datasets required for fine-tuning of large models are built locally, and the fine-tuning training of models is deployed locally. Relevant privacy and data security regulations are strictly followed, including ensuring that data is anonymized or desensitized, and taking security measures to protect the privacy and security of sensitive data.

[0068] (3) Using an open-source local Chinese language model and combining it with the efficient parameter fine-tuning technology of the large model, a professional knowledge base for local dental diagnosis and treatment is constructed to improve the model's ability to answer new questions.

[0069] (4) The answer style of the collected real doctor-patient dialogue dataset is uniformly adjusted while the accuracy of the answer content is preserved, making the medical question-and-answer assistant more humanized and interactive.

[0070] The schematic diagram of the Chinese dental intelligent diagnosis and treatment question-and-answer method provided in this embodiment of the invention is as follows: Figure 2 As shown.

[0071] The technical solution of the present invention will be further described below with reference to specific embodiments.

[0072] Example 1

[0073] Currently, generative big oracle models have been developed in the medical field. However, existing general big oracle models have limitations in dental diagnosis and treatment question answering, such as lack of dental professional knowledge, data sensitivity and privacy issues, low data timeliness, high training cost and data scarcity.

[0074] Based on the problems mentioned above, please refer to Figure 2 , Figure 2 This is a flowchart illustrating the Chinese dental intelligent diagnosis and treatment question-and-answer method provided in this embodiment of the invention. This embodiment provides a Chinese-language medical dental consultation assistant method for local hospitals, which includes the following steps:

[0075] Step 1: Obtain dental doctor-patient dialogue data and perform preprocessing to obtain a dental doctor-patient dialogue fine-tuning dataset with consistent style.

[0076] Specifically, this embodiment filters the publicly available Chinese online medical consultation dataset ChatMed ConsultDataset to obtain an initial doctor-patient dialogue dataset. The ChatMed Consult Dataset contains over 500,000 Chinese consultation dialogues. The questions in the dataset are all real medical consultation questions from the internet, reflecting the medical consultation needs of different users / patients in the real world. The responses are provided by the OpenAI GPT-3.5 engine. This embodiment selects dental-related questions from the dataset, filters the information to form a dental doctor-patient dialogue dataset, which contains 27,896 question-answer pairs.

[0077] To improve the quality of this dataset and better facilitate fine-tuning of the local large model, the response portion of the dental doctor-patient dialogue dataset was rewritten before efficient parameter fine-tuning of the large model. Using the OpenAI GPT-3.5 general-purpose large language model, a customized Prompt was used to style-rewrite ChatGPT prompts. This ensured that the response portion of the dataset retained the accuracy of the answers while making medical Q&A more human-like and interactive. Subsequently, the data format was adjusted to meet the input format requirements for efficient fine-tuning of the local large model. Please refer to [link / reference]. Figure 3 , Figure 3 This is a schematic diagram of the response style rewriting of the dental doctor-patient dialogue dataset provided in the embodiment of the present invention, where instruction is the dental question from a real patient and output is the preprocessed question answer.

[0078] In this embodiment, each doctor-patient dialogue pair in the dental doctor-patient dialogue dataset is processed as described above to obtain a preprocessed dental doctor-patient dialogue fine-tuning dataset.

[0079] Step 2: Select a high-performance local open-source Chinese language model as the base model and perform efficient parameter fine-tuning.

[0080] Specifically, high-performance local open-source large models allow users to freely modify and customize them to suit specific needs and application scenarios. Users can adjust and optimize models based on their own datasets and tasks to achieve better performance and results. Local open-source large models can run on the user's own device, without relying on cloud services or third-party platforms, providing greater flexibility and privacy protection. Users can perform data processing and model inference locally, reducing dependence on external services. This ensures data and model security and privacy protection, allowing control over data processing and storage in a local environment and reducing the risk of sensitive data leakage.

[0081] ChatGLM2 is an open-source dialogue language model that performs well on Chinese tasks. It is based on the General Language Model (GLM) architecture. The ChatGLM2-6B version has 6.2 billion parameters, uses similar technology to ChatGPT, and is optimized for Chinese question-and-answer and dialogue. It has been trained on a large amount of labeled data in both Chinese and English, and is supported by techniques such as supervised fine-tuning, feedback self-help, and human feedback reinforcement learning. Although the model is not as large as a model with hundreds of billions of parameters, it greatly reduces the deployment threshold and can generate answers that are quite in line with human preferences.

[0082] The efficient parameter fine-tuning method only fine-tunes a small number of additional model parameters, fixes most pre-trained parameters, inherits the capabilities obtained in the pre-training stage, can quickly transfer the knowledge from the pre-training stage to the dental diagnosis and treatment question answering task, and efficiently adapts to various downstream application tasks. In this embodiment, the constructed dental doctor-patient dialogue dataset is used to perform LoRA efficient parameter fine-tuning on the local large language model.

[0083] LoRA fine-tuning indirectly trains some dense layers in the neural network by optimizing the rank decomposition matrix of dense layer changes during the adaptation process, while keeping the pre-trained weights unchanged. Please refer to Figure 4 , Figure 4 , which is a schematic structural diagram of LoRA fine-tuning for a dental large model provided by an embodiment of the present invention. During the training process, the original pre-trained weight W0 is fixed, and a dimensionality reduction matrix A and a dimensionality increase matrix B are newly added on the bypass. Assuming that a large language model needs to be fine-tuned in a professional field, the pre-trained model parameters need to be updated, and the formula is:

[0084] W0+ΔW(1)

[0085] Where W0 is the initialization parameter of the pre-trained model, and ΔW is the parameter that needs to be updated. For full-parameter fine-tuning, the number of parameters of ΔW is the same as that of W0, so the training difficulty is relatively high. LoRA fine-tuning can convert the update of ΔW into the following form:

[0086]

[0087] Based on the inherent low-rank property of large models, an additional bypass is formed by introducing the dimensionality reduction matrix A and the dimensionality increase matrix B. Matrix A is initialized by Gaussian distribution, and B is initialized to all zeros, which ensures that the bypass is a zero matrix at the beginning of training. The update of this bypass matrix is used to simulate the training of full-parameter fine-tuning, where the rank r<<min(d, k), and different ranks r are selected according to the difference between the domain fine-tuning data and the original pre-training data. During the training process, W0 remains fixed, and matrices A and B are training parameters. The number of parameters is significantly lower than that of full-parameter fine-tuning, the performance is better than other parameter-efficient fine-tuning methods, and the training cost and training time can be greatly reduced. For the decline of loss during training, please refer to Figure 5 , Figure 5 , which is a schematic diagram of loss decline during the fine-tuning process provided by an embodiment of the present invention. Where "original" represents the curve corresponding to the original loss function, and "smoothed" is the effect curve obtained after smoothing the original curve.

[0088] After training, during inference, the trained matrix product BA is simply added to the original weight matrix W0 as the new weight parameters, replacing the W of the original pre-trained language model. This does not introduce additional inference latency. The processing of input x during inference is as follows:

[0089] h=W0x+ΔWx=W0x+BAx (3)

[0090] Let the model's input be x, its output be h, and the size of the parameter matrix for full parameter fine-tuning be d*k. When simulating full parameter fine-tuning through low-rank decomposition, the parameter matrix size of the dimension-reduced matrix A is r*k, and the parameter matrix size of the dimension-increasing matrix B is d*r. Thus, the size of the matrix BA formed by matrices A and B is d*r, which is equal to the size of the original full parameter fine-tuning matrix.

[0091] After the model is fine-tuned, using this large dental model can provide effective answers to dental questions raised by users. Please refer to [link / reference]. Figure 6 , Figure 6 This is a schematic diagram illustrating how the fine-tuned large model provided in this embodiment reasones and answers dental questions.

[0092] Step 3: Obtain relevant information about local dentists, clean the data, and obtain a dentist information knowledge base;

[0093] Specifically, in the internet age, web crawlers primarily provide search engines with the most comprehensive and up-to-date data, serving as a valuable tool for data mining by collecting data from the internet. This example uses web crawling technology to obtain relevant information about local dentists. Scrapy, developed in Python, is an application framework for crawling website data and extracting structured data. It can be applied to a range of programs, including data mining, information processing, and storing historical data.

[0094] This example uses the Scrapy web crawler framework to obtain information such as the profile, department, personal medical expertise, affiliated hospital, doctor's title, patient reviews, and consultation information of local hospital dentists, while ensuring that the website rules are respected and the legality of data crawling is guaranteed. The structured information is then saved to a local file.

[0095] Further data cleaning is needed for the crawled structured local doctor information: the data cleaning process is specifically divided into data incompleteness handling and data redundancy handling.

[0096] Data Incompleteness Handling: The data may contain missing values, which may be due to reasons such as not being recorded or being unavailable. For gender, age, and years of medical practice, the mode is used for completion; for graduating school, personal medical expertise, patient ratings, etc., predefined default values ​​are used for completion. Information that does not meet the requirements is filtered: If no existing consultation records exist, the doctor is considered inactive, and the data row is cleared. Python is used to retrieve the number of consultations; if it is empty, the data row is cleared, and then Excel is used to locate and delete the row. Data Redundancy Handling: For gender, age, and years of medical practice, only the maximum value of multiple redundant data is retained; for graduating school, personal medical expertise, patient ratings, etc., only the longest text of multiple redundant data is retained. The cleaned data is saved locally in a doctor information document.

[0097] This example uses the M3E (Multimodal Multitask Meta-Embedding) Chinese embedding model to segment and vectorize the text in the local doctor information document, and store it locally as a doctor information knowledge base. For newly added local doctor information, after adding it to the local doctor information document, it is segmented and vectorized again before being updated to the local doctor information knowledge base.

[0098] Converting language model data into vectors is typically done using embedding models. The vectorized data can then be stored in a vector database, which provides an efficient way to store and index large amounts of vector data. It allows finding the most relevant text by searching for the vector closest to a given vector. The M3E model is an embedding model trained on a dataset of tens of millions of Chinese sentences. This model supports functions such as calculating similarity between homogeneous Chinese and English texts and retrieving heterogeneous texts.

[0099] Step 4: Collect and organize professional journals and other materials related to dental diseases to generate a professional information knowledge base;

[0100] Specifically, this embodiment transforms professional dental knowledge texts, such as oral anatomy and dental surgical techniques, into local professional information documents. Due to the limitation of the maximum input length of the local large language model, it is necessary to segment the text according to its actual situation. The segmented text is then projected into vectors using the same embedding model as in step three and stored locally as a professional information knowledge base.

[0101] Step 5: Introduce the construction of a doctor information knowledge base and a professional information knowledge base into the fine-tuned local large model, and combine the reasoning ability of the large model with local knowledge to answer questions.

[0102] Specifically, in order for the fine-tuned dental model to perform inference by combining the constructed doctor information knowledge base and professional information knowledge base during Chinese dental consultations, it is necessary to introduce the constructed doctor information knowledge base and professional information knowledge base into the fine-tuned local model. Please see [link to relevant documentation]. Figure 7 , Figure 7 This is a schematic diagram of the Chinese dental intelligent diagnosis and treatment question-and-answer structure provided in an embodiment of the present invention.

[0103] When a user poses a question to the dental database model, if neither of the two databases is selected, the model will directly reason about the question and generate relevant medical suggestions based on the user's dental symptoms. When the user chooses to connect to a local doctor information knowledge base, the model will project the user's personal health information and description of their condition into a vector using the M3E embedding model. The vectorized question text will then be compared with vectors in the dental doctor information database to find the most relevant document. The question and the document corresponding to the matching vector in the database will be embedded into a fixed prompt template. This template will be submitted to the dental database model, resulting in a response containing specific dentist information. Finally, personalized suggestions and recommendations will be provided. Please refer to [link to relevant documentation]. Figure 8 , Figure 8 This is a diagram illustrating the comparison of the reasoning and answering effects of combining the dental physician information knowledge base with the present embodiment. The upper question-and-answer pair represents the answer results without connecting to the dental physician information knowledge base, while the lower question-and-answer pair represents the answer results with connecting to the dental physician information knowledge base.

[0104] When a user chooses to connect to a professional information knowledge base, the system will project the user's professional question into a vector using the M3E embedding model. The vectorized question text will then be compared with the vectors in the professional information database for similarity. The system will also combine the reasoning capabilities of the large model with the professional information knowledge base to perform retrieval and generate professional answers.

[0105] This embodiment proposes a complete Chinese dental intelligent diagnosis and answering technology, which can solve many shortcomings of traditional general-purpose large models in dental diagnosis and answering, such as lack of dental professional knowledge, data sensitivity and privacy issues, low data timeliness, high training costs, and data scarcity. This embodiment provides support for intelligent diagnosis technology of large models in the dental field, making dental diagnosis and answering large models more practical, reliable, and popular. This embodiment can be widely used in intelligent dental diagnosis, dental teaching, and other applications in hospitals, homes, and academic education environments.

[0106] Example 2

[0107] Based on the above embodiment one, please refer to Figure 9 , Figure 9This is a schematic diagram of the structure of a Chinese dental intelligent diagnosis and treatment question-and-answer electronic device provided in an embodiment of the present invention. This embodiment provides a Chinese dental intelligent diagnosis and treatment question-and-answer electronic device, which includes a text acquisition device, a display, a graphics processor, a communication interface, a memory, a central processing unit, and a communication bus. The text acquisition device, display, graphics processor, communication interface, memory, and central processing unit communicate with each other through the communication bus.

[0108] A text input device is used to input text and enable interactive consultation.

[0109] A display is used to transmit dialogue data;

[0110] Graphics processor, used for inference calculations of large models;

[0111] Memory, used to store computer programs;

[0112] When a central processing unit (CPU) executes a computer program stored in memory, the computer program performs the following steps when executed by the processor:

[0113] Step 1: Obtain dental doctor-patient dialogue data and perform preprocessing to obtain a dental doctor-patient dialogue fine-tuning dataset with consistent style.

[0114] Specifically, dental doctor-patient dialogue data is acquired and preprocessed to obtain a style-consistent fine-tuned dataset of dental doctor-patient dialogue, including:

[0115] The response portion of the dental doctor-patient dialogue dataset was rewritten to preserve the accuracy of the responses while making the medical Q&A more humanized and interactive. Subsequently, the data format was adjusted to meet the input format requirements for efficient fine-tuning of the local large model.

[0116] Step 2: Select a high-performance local Chinese language model as the base model, perform efficient parameter fine-tuning, and save the training model parameters.

[0117] Specifically, step 2 of this embodiment includes:

[0118] The selected local Chinese language model was subjected to LoRA-efficient parameter fine-tuning using a constructed dental doctor-patient dialogue dataset. The pre-training parameters of the model were kept unchanged, and the training of full parameter fine-tuning was simulated by updating the side-path matrix. This enabled the model to inherit the capabilities gained in the pre-training stage and to quickly transfer the knowledge gained in the pre-training stage to the dental diagnosis and treatment question-answering task.

[0119] Step 3: Obtain relevant information about local dentists, clean the data, and obtain a dentist information knowledge base.

[0120] Specifically, step 3 of this embodiment uses web crawling technology to obtain information such as the local dentist's profile, department, personal medical expertise, affiliated hospital, doctor's title, patient reviews, and consultation information, and performs data cleaning. The data cleaning process is specifically divided into data incompleteness processing and data redundancy processing. The M3E Chinese embedding model is used to segment the text in the local doctor information document, project it into vectors, and store it locally as a doctor information knowledge base.

[0121] Step 4: Obtain relevant information about local dentists, clean the data, and obtain a dentist information knowledge base.

[0122] Specifically, professional dental knowledge texts such as oral anatomy and dental surgical techniques are transformed into local professional information documents. Due to the limitation of the maximum input token of the local large language model, it is necessary to segment the text according to the actual situation of each text. The segmented text is then projected into vectors using the same embedding model as in step three and stored locally as a professional information knowledge base.

[0123] Step 5: Introduce the construction of a doctor information knowledge base and a professional information knowledge base into the fine-tuned local large model, and combine the reasoning ability of the large model with local knowledge to answer questions.

[0124] Specifically, the finely tuned model parameters are loaded into the local large model, and two local external knowledge bases are available for selection during use. When connecting to the local doctor information knowledge base, personalized suggestions and recommendations are provided based on the user's personal health information and description of their condition, using the knowledge base of that dentist. When connecting to the professional information knowledge base, professional answers are generated based on the professional questions raised by the user, combining the reasoning ability of the large model with the professional information knowledge base.

[0125] The electronic device provided in this embodiment can execute the above-described Chinese dental intelligent diagnosis and treatment question-and-answer embodiment and the above-described Chinese dental intelligent diagnosis and treatment question-and-answer device embodiment. Its implementation principle and technical effect are similar, and will not be described again here.

[0126] Example 3

[0127] Based on the above embodiment two, please refer to Figure 10 , Figure 10 This is a schematic diagram of a computer-readable storage medium provided in an embodiment of the present invention. The computer-readable storage medium provided in this embodiment stores a computer program, which, when executed by a processor, performs the following steps:

[0128] Step 1: Obtain dental doctor-patient dialogue data and perform preprocessing to obtain a dental doctor-patient dialogue fine-tuning dataset with consistent style.

[0129] Specifically, dental doctor-patient dialogue data is acquired and preprocessed to obtain a style-consistent fine-tuned dataset of dental doctor-patient dialogue, including:

[0130] The response portion of the dental doctor-patient dialogue dataset was rewritten to preserve the accuracy of the responses while making the medical Q&A more humanized and interactive. Subsequently, the data format was adjusted to meet the input format requirements for efficient fine-tuning of the local large model.

[0131] Step 2: Select a high-performance local Chinese language model as the base model, perform efficient parameter fine-tuning, and save the training model parameters.

[0132] Specifically, step 2 of this embodiment includes:

[0133] The selected local Chinese language model was subjected to LoRA-efficient parameter fine-tuning using a constructed dental doctor-patient dialogue dataset. The pre-training parameters of the model were kept unchanged, and the training of full parameter fine-tuning was simulated by updating the side-path matrix. This enabled the model to inherit the capabilities gained in the pre-training stage and to quickly transfer the knowledge gained in the pre-training stage to the dental diagnosis and treatment question-answering task.

[0134] Step 3: Obtain relevant information about local dentists, clean the data, and obtain a dentist information knowledge base.

[0135] Specifically, step 3 of this embodiment uses web crawling technology to obtain information such as the local dentist's profile, department, personal medical expertise, affiliated hospital, doctor's title, patient reviews, and consultation information, and performs data cleaning. The data cleaning process is specifically divided into data incompleteness processing and data redundancy processing. The M3E Chinese embedding model is used to segment the text in the local doctor information document, project it into vectors, and store it locally as a doctor information knowledge base.

[0136] This embodiment provides a computer-readable storage medium that can execute the above-described embodiments of the Chinese dental intelligent diagnosis and treatment question-and-answer method and the above-described embodiments of the Chinese dental intelligent diagnosis and treatment question-and-answer electronic device. The implementation principle and technical effects are similar, and will not be described again here.

[0137] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented, in whole or in part, as a computer program product, the computer program product includes one or more computer instructions. When the computer program instructions are loaded or executed on a computer, all or part of the processes or functions described in the embodiments of the present invention are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., a solid-state drive (SSD)).

[0138] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any modifications, equivalent substitutions, and improvements made by those skilled in the art within the scope of the technology disclosed in the present invention, and within the spirit and principles of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A Chinese-language intelligent dental diagnosis and answering method, characterized in that, Includes the following steps: Step 1: Obtain real dental doctor-patient dialogue data and preprocess it to obtain a dental doctor-patient dialogue dataset with a consistent style. Step 2: Construct a local open-source Chinese language model with fine-tuned parameters; Step 3: Obtain and build a knowledge base of dentists' information from local hospitals; Step 4: Build a professional information knowledge base for dental conditions; Step 5: Based on the locally open-source Chinese language model with fine-tuned parameters, connect the constructed dentist information knowledge base and professional information knowledge base to realize intelligent Chinese dental diagnosis and treatment question and answer; The step three, namely acquiring and constructing a knowledge base of dentists' information from local hospitals, includes: Information on dentists in local hospitals is obtained through web scraping, including dentist profiles, departments, medical specialties, affiliated hospitals, professional titles, patient reviews, and consultation schedules. This information forms a local dentist information document. When building a knowledge base of dentist information for local hospitals, the obtained information needs to undergo data cleaning, including handling data incompleteness and redundancy. Data incompleteness handling: Gender is binarized and then padded with age and years of medical practice using the mode; for graduating school, personal medical expertise, and patient evaluations, predefined default values ​​are used to fill in the missing data. Data redundancy handling: Binarize gender and retain only the maximum value of multiple redundant data points along with age and length of medical practice; for graduating school, personal medical expertise, and patient reviews, retain only the longest text among multiple redundant data points. The text-to-vector projection method is selected to perform text segmentation and text vectorization on the local doctor information documents after data cleaning, and store them locally as a doctor information knowledge base. For newly added local doctor information, after adding it to the local doctor information document, text segmentation and text vectorization projection are performed again to update the local doctor information knowledge base. Step four, which involves building a professional information knowledge base for dental conditions, includes: After summarizing and organizing several professional journal articles on dental diseases, the text was segmented and vectorized, and then stored in a local professional information knowledge base. The local open-source Chinese language model with fine-tuned parameters in step five is connected to the constructed dentist information knowledge base and professional information knowledge base to realize intelligent Chinese dental diagnosis and treatment question answering, including: When using the system, users can choose between a dentist information knowledge base and a professional information knowledge base. When connecting to the dentist information knowledge base, personalized suggestions and recommendations are given based on the user's personal health information and medical condition description, combined with the constructed dentist information knowledge base. When connecting to the professional information knowledge base, professional answers are generated based on the professional questions raised by the user, combined with the reasoning ability of the large model and the professional information knowledge base.

2. The Chinese dental intelligent diagnosis and treatment question-and-answer method as described in claim 1, characterized in that, Step one involves preprocessing the collected real dental doctor-patient dialogue data to obtain a preprocessed dental doctor-patient dialogue dataset, including: Aiming at the medical consultation needs of different users or patients, the doctor's reply part is reconstructed using ChatGPT, so that the reply data presents comprehensive explanations and high-level empathy, a dental doctor-patient dialogue dataset with a unified style is formed, and the dialogue pair format is converted into the JSON-format question-answer pair form required for fine-tuning data, so as to form the dental doctor-patient dialogue dataset required for large model fine-tuning.

3. The Chinese dental intelligent diagnosis and treatment question-and-answer method as described in claim 1, characterized in that, The local open-source Chinese large language model takes a deep learning-based natural language processing model as the base model, which can learn the grammar and semantics of natural language, generate human-readable text, and can be deployed locally; LoRA parameter fine-tuning is performed on the base model: all pre-training parameters W0 of the base model are fixed and remain unchanged, a dimension reduction matrix A and a dimension increasing matrix B are introduced to form a bypass, and parameters are learned and updated, so as to inherit the capabilities obtained in the pre-training stage and quickly transfer the knowledge from the pre-training stage to the dental diagnosis and treatment question answering task.

4. The Chinese dental intelligent diagnosis and treatment question-and-answer method as described in claim 3, characterized in that, The LoRA parameter fine-tuning converts the update of the parameter ΔW to be updated into the following form: wherein, the dimension reduction matrix A is initialized by Gaussian distribution, the dimension increasing matrix B is initialized to all zeros, the training of full parameter fine-tuning is simulated by using the update of the bypass, the rank r<<min(d, k), and different ranks r are selected according to the difference between the domain fine-tuning data and the original pre-training data; After the training is completed, the trained matrix product BA is added to the original weight matrix W0 as the new weight parameter during inference to replace the weight parameter W of the original pre-trained language model; during inference, the processing process for an input x is as follows: h= W0x +ΔWx= W0x +BAx h is the output of the model, d*k is the parameter matrix size for full parameter fine-tuning, r*k is the parameter matrix size of the dimension reduction matrix A, and d*r is the parameter matrix size of the dimension increasing matrix B.

5. An electronic device applying the Chinese dental intelligent diagnosis and treatment question-and-answer method as described in any one of claims 1 to 4, characterized in that, The electronic device of the dental diagnosis and treatment consultation assistant comprises a text collector, a display, a graphics processor, a communication interface, a memory, a central processing unit and a communication bus; wherein, the display, the text collector, the graphics processor, the communication interface, the memory and the central processing unit communicate with each other through the communication bus; the text collector is configured to input text and realize consultation interaction; the display is configured to present dialogue data; the graphics processor is configured to perform inference calculation of the large model; the memory is configured to store a computer program; the central processing unit is configured to implement the design method of the dental diagnosis and treatment consultation assistant when executing the computer program stored in the memory.

6. An electronic device, characterized in that, the electronic device comprises a memory and a processor, the memory stores a computer program, and when the computer program is executed by the processor, the processor is enabled to execute the steps of the Chinese dental intelligent diagnosis and treatment question answering method according to any one of claims 1 to 4: The process involves: acquiring and preprocessing real dental doctor-patient dialogue data; efficiently fine-tuning a local open-source Chinese language model; acquiring and building a knowledge base of dentists' information from a local hospital; building a professional knowledge base of information on dental diseases; and connecting the built dentist information knowledge base and the professional knowledge base with the fine-tuned local model to achieve intelligent Chinese dental diagnosis and treatment question answering.

7. A computer-readable storage medium storing a computer program, which, when executed by a processor, causes the processor to perform the steps of the Chinese dental intelligent diagnosis and answering method as described in any one of claims 1 to 4: The process involves: acquiring and preprocessing real dental doctor-patient dialogue data; constructing a local open-source Chinese language model with efficient parameter fine-tuning; acquiring and constructing a knowledge base of dentists' information from local hospitals; constructing a professional knowledge base of information on dental diseases; and connecting the constructed dentist information knowledge base and the professional knowledge base with the fine-tuned local model to achieve intelligent Chinese dental diagnosis and treatment question answering.

Images