An AI question-answering method for traditional Chinese medicine knowledge based on a multimodal dialogue model

Abstract

The present invention relates to a TCM knowledge AI question-answering method based on a multimodal dialogue model, which belongs to the field of computer systems based on specific computing models. The method includes: using a multimodal dialogue model to intelligently analyze the target TCM diagnosis disease type based on various physiological parameters of the current user, multiple visual information of the tongue sub-screen and various pulse data of the current user; calling a recommended treatment prescription matching the target TCM diagnosis disease type from a TCM prescription database. Through the present invention, the technical problem that the speed, efficiency and accuracy of TCM consultation are difficult to take into account is solved without relying on a complex TCM knowledge graph. By selecting fully comprehensive multimodal basic data and customizing a multimodal dialogue model, the intelligent analysis of the TCM diagnosis disease type with the highest probability of the current user being ill is completed, and the recommended treatment prescription is determined based on the TCM prescription database, thereby solving the above technical problems.

Description

Technical Field

[0001] The present invention relates to the field of computer systems based on specific computing models, and in particular to an AI question-answering method for traditional Chinese medicine knowledge based on a multimodal dialogue model. Background Art

[0002] In simple terms, the so-called multimodal big model is an artificial intelligence model that can understand and process multiple types of basic data, where the type of data is also called modality, including text, pictures, audio, video, etc. This model can integrate information from multiple different modalities to perform more complex and intelligent tasks. Specifically, if it is used for AI question-answering, it is called a multimodal dialogue model. There are various specific application areas for multimodal dialogue models. For example, the use of multimodal dialogue models to perform specific AI questions and answers on traditional Chinese medicine knowledge is due to the mismatch between the number of doctors and the number of patients and the lack of experienced doctors. It is expected to use modern technology artificial intelligence models to match various treatment methods of classical Chinese medicine with current patient symptoms, and select the best treatment prescription after screening big data.

[0003] For example, the Chinese invention patent publication CN117851571A proposes a TCM knowledge question-and-answer method that integrates knowledge graph and multimodal dialogue model, which belongs to the application of artificial intelligence in the field of TCM. The purpose of the present invention is to solve the problems that the existing TCM knowledge question-and-answer method cannot process image information and the understanding of TCM knowledge is insufficient. The method includes: collecting questions about TCM, TCM knowledge texts and images of Chinese herbal medicines from authoritative channels such as "TCM think tanks" and ancient Chinese medicine books, and selecting appropriate information from them under the guidance of TCM experts to construct a TCM multimodal dialogue data set; a multimodal dialogue model that integrates Transformer-based feature fusion technology and TCM knowledge graph is proposed to perform TCM knowledge question-and-answer. Using the TCM knowledge question-and-answer method that integrates knowledge graph and multimodal dialogue model proposed by the present invention, it is possible to realize the comprehensive utilization of image information and TCM knowledge graph information, thereby improving the intelligence level of the question-and-answer system in the field of TCM.

[0004] For example, the Chinese invention patent publication CN118800407A proposes a TCM multi-diagnosis system based on a TCM AI big model, including: using image recognition technology to analyze the patient's tongue coating characteristics, complexion characteristics and pulse characteristics, establishing the patient's initial diagnosis information, retrieving corresponding diagnosis and treatment questions based on the initial diagnosis information to ask the patient questions, and collecting the patient's answers to the questions to establish consultation information, inferring the patient's health information based on the initial diagnosis information and the consultation information in combination with a preset TCM AI big model, constructing decision-making diagnosis and treatment recommendations based on the health information and the TCM knowledge graph, and adjusting the decision-making diagnosis and treatment based on the doctor's decision annotations, generating and displaying available recommendations, and integrating big data and artificial intelligence technologies to achieve rapid and accurate capture and analysis of TCM information, reduce human errors, and improve diagnosis speed and accuracy.

[0005] It can be seen that the above technical solutions all rely on complex TCM knowledge graphs to execute specific TCM knowledge questions and answers or patient diagnosis, in which the basic data used for patient diagnosis is not comprehensive enough and completely relies on the patient's tongue and pulse image information. It does not introduce direct pulse physical data including the depth of the pulse manifestation site, pulse frequency, pulse strength, pulse range, pulse fluency level, vascular tension, and pulse rhythm uniformity, as well as the patient's own physiological basic data. As a result, the reliability of the diagnostic results obtained by analysis is not high, and doctors still need to further determine the final type of disease. In addition, different doctors have different experiences. Even if the determined type of disease is effective, the TCM prescription given may not be the most classic prescription, which seriously affects the speed, efficiency and accuracy of TCM treatment. Summary of the invention

[0006] In order to solve the technical problems in the prior art, the present invention provides an AI question-answering method for TCM knowledge based on a multimodal dialogue model, which no longer relies on a complex TCM knowledge graph. By selecting sufficient and comprehensive multimodal basic data, a multimodal dialogue model with targeted structural design is customized for TCM knowledge AI question-answering, which completes the intelligent analysis of the TCM diagnosis disease type with the highest probability of the current user, and calls the TCM prescription that matches the TCM diagnosis disease type obtained by the intelligent analysis and has been used most frequently in history from the TCM prescription database as a recommended treatment prescription, thereby solving the technical problem that it is difficult to take into account the speed, efficiency and accuracy of TCM consultation.

[0007] According to the present invention, a TCM knowledge AI question-answering method based on a multimodal dialogue model is provided, the method comprising:

[0008] For the current user participating in the AI ​​Q&A of TCM knowledge, a screen capture of the environment where the tongue is located is performed to obtain a corresponding real-time capture screen, a tongue sub-screen in the real-time capture screen is identified based on the external imaging features of the human tongue, and the L component value, A component value, and B component value of each pixel point of the tongue sub-screen are obtained;

[0009] Performing on-site capture of various pulse data of the current user to obtain various pulse data of the current user, wherein the various pulse data include the depth of the pulse manifestation site, the pulse frequency, the pulse intensity, the length of the pulse axial range, the radial radius corresponding to the pulse radial range, the pulse fluency level, the vascular tension mark and the pulse rhythm uniformity mark;

[0010] Performing a preset number of training operations on the deep neural network to obtain a deep neural network after performing the preset number of training operations, and outputting the deep neural network after performing the preset number of training operations as a multimodal dialogue model;

[0011] The multimodal dialogue model is used to intelligently analyze the TCM diagnosis disease type of the current user with the highest probability of being ill based on various physiological parameters of the current user, L component values, A component values ​​and B component values ​​of various pixel points of the tongue sub-image and various pulse data of the current user, and output it as the target TCM diagnosis disease type;

[0012] Retrieving from the TCM prescription database the TCM prescription that matches the target TCM diagnosis disease type and is most frequently used in history as a recommended treatment prescription for on-site display;

[0013] The various physiological parameters of the current user are the ASCII code values ​​of the age information, gender identifier, weight information, height information and the type names of the types of diseases diagnosed by traditional Chinese medicine that the current user suffered in the past.

[0014] Compared with the prior art, the present invention has at least the following four outstanding substantive features:

[0015] First: Customize a multimodal dialogue model for AI Q&A on TCM knowledge, which is used to intelligently analyze the TCM diagnosis disease types that the current user is most likely to suffer from based on various forms of basic data including pictures and texts, thereby improving the current user's medical treatment efficiency and accuracy while reducing the labor and time costs of TCM consultation management;

[0016] Second, the structural customization of the multimodal dialogue model is reflected in the following aspects: the multimodal dialogue model is a deep neural network after executing a preset number of training operations. It is particularly important that the number of hidden layers of the deep neural network is monotonically positively correlated with the total number of TCM diagnosis disease types stored in the TCM prescription database, and the number of training operations of the deep neural network is proportional to the total number of pixels of the real-time acquisition screen, so as to build multimodal dialogue models with different structures for acquisition screens with different data volumes;

[0017] Third: targeted selection of multiple basic data of different modalities for intelligent analysis of the type of TCM diagnosis disease with the highest probability for the current user, including the L component value, A component value and B component value of each pixel point of the tongue sub-screen corresponding to the current user, the depth of the pulse manifestation part, pulse frequency, pulse intensity, pulse axial range length, radial radius corresponding to the pulse radial range, pulse fluency level, vascular tension mark and pulse rhythm uniformity mark of the current user, the age information, gender mark, weight information, height information of the current user and the ASCII code value of the type name of the TCM diagnosis disease type suffered in the past. The full and comprehensive selection of the above multiple basic data ensures the reliability and stability of the intelligent analysis results;

[0018] Fourthly: in each training operation performed on the deep neural network, the type of TCM diagnosed disease currently suffered by a certain user is used as the single output data of the deep neural network, and the various physiological parameters of the certain user, the L component value, A component value and B component value of each pixel point of the tongue sub-screen corresponding to the certain user and the various pulse data of the certain user are used as multiple input data of the deep neural network to complete this training operation, thereby ensuring the training effect of each training operation of the deep neural network. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The embodiments of the present invention will be described below with reference to the accompanying drawings, wherein:

[0020] Figure 1 It is a technical flow chart of the AI ​​question-answering method of traditional Chinese medicine knowledge based on the multimodal dialogue model according to the present invention.

[0021] Figure 2 The present invention is a flowchart of the steps of an AI question-answering method for traditional Chinese medicine knowledge based on a multimodal dialogue model according to Example 1 of the present invention.

[0022] Figure 3 The present invention is a flowchart of the steps of an AI question-answering method for traditional Chinese medicine knowledge based on a multimodal dialogue model according to Example 2 of the present invention.

[0023] Figure 4 The present invention is a flowchart of the steps of an AI question-answering method for traditional Chinese medicine knowledge based on a multimodal dialogue model according to Example 3 of the present invention.

[0024] Figure 5 The present invention is a flowchart of the steps of an AI question-answering method for traditional Chinese medicine knowledge based on a multimodal dialogue model according to Example 4 of the present invention.

[0025] Figure 6 A flowchart of the steps of the AI ​​question-answering method for traditional Chinese medicine knowledge based on a multimodal dialogue model according to Example 5 of the present invention DETAILED DESCRIPTION

[0026] like Figure 1 As shown, a technical flow chart of the AI ​​question-answering method for traditional Chinese medicine knowledge based on a multimodal dialogue model according to the present invention is given.

[0027] like Figure 1 As shown, the specific technical process of the present invention is as follows:

[0028] Technical process 1: A multimodal dialogue model with a targeted structure is customized for AI Q&A on TCM knowledge, which is used to perform intelligent analysis of the TCM diagnosis disease type that the current user is most likely to suffer from based on multimodal data;

[0029] like Figure 1 As shown, the AI ​​question-answering of TCM knowledge generally adopts the question-answering mode, i.e., the Q / A mode, to conduct question-answering interaction between the current user and the computer, and the multimodal dialogue model generally runs in the computer system in front of the current user, and the computer system can be a conventional computer system or a microcomputer system running on a portable terminal;

[0030] Specifically, the customization of the structure of the multimodal dialogue model is mainly reflected in the following aspects:

[0031] In a first aspect, the multimodal dialogue model is a deep neural network after executing a preset number of training operations, and the number of hidden layers of the deep neural network is monotonically positively correlated with the total number of TCM diagnosis disease types stored in the TCM prescription database;

[0032] Secondly, the number of training operations of the deep neural network, i.e., the value of the preset number, is proportional to the total number of pixels of the real-time collection screen obtained by visual data collection of the current user's tongue, thereby building multimodal dialogue models with different structures for collection screens with different data volumes;

[0033] Thirdly, in each training operation performed on the deep neural network, the type of TCM diagnosis disease currently suffered by a certain user is used as a single output data of the deep neural network, and various physiological parameters of the certain user, the L component value, A component value and B component value of each pixel point of the tongue sub-screen corresponding to the certain user and various pulse data of the certain user are used as multiple input data of the deep neural network to complete this training operation, thereby ensuring the training effect of each training operation of the deep neural network;

[0034] Technical process 2: Targeted selection of multiple basic data of different modalities for intelligent analysis of the type of TCM diagnosis disease with the highest probability of the current user;

[0035] Specifically, the multiple basic data of different modalities include not only the customized visualization information of the tongue of the current user, but also various physical parameters of the pulse of the current user, such as the depth of the pulse manifestation site of the current user, the pulse frequency, the pulse strength, the length of the pulse axial range, the radial radius corresponding to the pulse radial range, the pulse fluency level, the vascular tension mark and the pulse rhythm uniformity mark, and various physiological parameters of the current user, such as the age information, gender mark, weight information, height information of the current user and the ASCII code value of the type name of the type of TCM diagnosis disease type suffered by the current user in the past;

[0036] More specifically, the pulse frequency of the pulse manifestation site of the current user, i.e., the pulse number, refers to the number of pulse beats in the current user's breathing cycle; the pulse radial range of the pulse manifestation site of the current user, i.e., the pulse width, refers to the width range of the current user's pulse channel, and the radial radius corresponding to the pulse radial range of the pulse manifestation site of the current user refers to the maximum radial radius of the width range of the current user's pulse channel; the pulse smoothness level of the current user's pulse manifestation site refers to the smoothness of the pulse of the current user's pulse manifestation site. The smoother the pulse, i.e., the more it tends to be slippery pulse, the higher the pulse smoothness level; the vascular tension mark of the current user's pulse manifestation site is used to indicate whether the vascular vessels of the current user's pulse manifestation site are tight, and the pulsation rhythm uniformity mark of the current user's pulse manifestation site is used to indicate whether the pulsation rhythm of the vascular vessels of the current user's pulse manifestation site is uniform;

[0037] More specifically, the present invention provides a variety of acquisition methods to obtain the depth of the pulse manifestation site, pulse frequency, pulse strength, pulse axial range length, radial radius corresponding to the pulse radial range, pulse fluency level, vascular tension mark and pulse rhythm uniformity mark of the current user, and these methods are specifically as follows:

[0038] The first acquisition method: using a variety of different sensing elements to respectively acquire the depth of the pulse manifestation site, pulse frequency, pulse intensity, pulse axial range length, radial radius corresponding to the pulse radial range, pulse fluency level, vascular tightness mark and pulsation rhythm uniformity mark of the current user. For example, a visual detection element can be used to detect the pulse axial range length, radial radius corresponding to the pulse radial range and vascular tightness mark of the current user's pulse manifestation site, and a piezoelectric sensing element can be used in conjunction with a timing service element to measure the pulse fluency and pulsation rhythm uniformity of the current user's pulse manifestation site, thereby acquiring the pulse fluency level and pulsation rhythm uniformity mark of the current user's pulse manifestation site;

[0039] The second acquisition method: a Chinese medicine doctor manually feels the pulse of the current user to obtain the depth of the pulse manifestation site, pulse frequency, pulse strength, pulse axial range length, radial radius corresponding to the pulse radial range, pulse fluency level, vascular tension mark and pulse rhythm uniformity mark of the current user. Since these parameters are the 8 most common parameters of Chinese medicine pulse feeling, they are not described here;

[0040] The third acquisition method: some parameters are obtained by using a variety of different sensor elements, and the remaining parameters are obtained by taking the pulse manually by a traditional Chinese medicine doctor;

[0041] The fourth acquisition method: for each parameter, two sets of data are obtained electronically by the corresponding sensor element and manually by a TCM physician, and the average of the two sets of data is taken as the measured value of the parameter, so as to be used for intelligent analysis of the type of TCM diagnosis disease that the current user is most likely to suffer from;

[0042] More specifically, the customized visualization information of the tongue of the current user is the L component value, the A component value, and the B component value of each pixel point of the tongue sub-image corresponding to the current user;

[0043] It can be seen that the targeted screening of different modes of the above-mentioned multiple basic data involves three modes: visualization type, pulse physical parameter type, and physiological parameter type. The full and comprehensive selection of these basic data ensures the reliability and stability of the intelligent analysis results;

[0044] Technical process three: Adopt the multimodal dialogue model customized by technical process one, based on the basic data of multiple different modalities selected in technical process two, to intelligently analyze the type of TCM diagnosis disease that the current user is most likely to suffer from;

[0045] Technical process 4: calling from the TCM prescription database the TCM prescription that matches the TCM diagnosis disease type with the highest probability of the current user being ill obtained by intelligent analysis in technical process 3 and is the most frequently used TCM prescription in history as the recommended treatment prescription and performing on-site display;

[0046] In this way, not only are the TCM diagnosis disease types with the highest probability of illness provided to the current user, but the most classic treatment prescriptions matching the TCM diagnosis disease types with the highest probability of illness are also given in a timely manner, thus overcoming the drawbacks of TCM doctors' lack of experience and the non-classic treatment prescriptions given.

[0047] The key points of the present invention are: targeted screening of different modes involving visualization type, pulse physical parameter type and physiological parameter type, targeted structural design of multimodal dialogue model and determination of the most classic treatment prescription based on the traditional Chinese medicine prescription database.

[0048] Below, the AI ​​question-answering method for traditional Chinese medicine knowledge based on a multimodal dialogue model of the present invention will be specifically described in the form of an embodiment. Example 1

[0049] Figure 2 The present invention is a flowchart of the steps of an AI question-answering method for traditional Chinese medicine knowledge based on a multimodal dialogue model according to Example 1 of the present invention.

[0050] like Figure 2 As shown, the TCM knowledge AI question-answering method based on the multimodal dialogue model includes the following specific steps:

[0051] Step S201: performing screen capture of the tongue environment of the current user participating in the AI ​​Q&A of TCM knowledge to obtain a corresponding real-time capture screen, identifying a tongue sub-screen in the real-time capture screen based on the appearance imaging features of the human tongue, and obtaining the L component value, A component value, and B component value of each pixel point of the tongue sub-screen;

[0052] For example, a programmable logic device may be selected to identify a tongue sub-image in the real-time acquisition image based on the appearance imaging features of the human tongue, wherein the tongue sub-image is located in the real-time acquisition image, and the real-time acquisition image is a state where the current user is in an open mouth state with the tongue extended;

[0053] Step S202: performing on-site capture of various pulse data of the current user to obtain various pulse data of the current user, wherein the various pulse data include the depth of the pulse manifestation site, the pulse frequency, the pulse intensity, the length of the pulse axial range, the radial radius corresponding to the pulse radial range, the pulse fluency level, the vascular tension mark and the pulse rhythm uniformity mark;

[0054] Specifically, performing on-site capture of various pulse data of the current user to obtain various pulse data of the current user, wherein the various pulse data include the depth of the pulse manifestation site, the pulse frequency, the pulse intensity, the pulse axial range length, the radial radius corresponding to the pulse radial range, the pulse fluency level, the vascular tightness mark and the pulse rhythm uniformity mark, including: different parameter detection units can be selected to respectively detect the depth of the pulse manifestation site, the pulse frequency, the pulse intensity, the pulse axial range length, the radial radius corresponding to the pulse radial range, the pulse fluency level, the vascular tightness mark and the pulse rhythm uniformity mark of the current user;

[0055] More specifically, the present invention provides a variety of acquisition methods to obtain the depth of the pulse manifestation site, pulse frequency, pulse strength, pulse axial range length, radial radius corresponding to the pulse radial range, pulse fluency level, vascular tension mark and pulse rhythm uniformity mark of the current user, and these methods are specifically as follows:

[0056] The first acquisition method: using a variety of different sensing elements to respectively acquire the depth of the pulse manifestation site, pulse frequency, pulse intensity, pulse axial range length, radial radius corresponding to the pulse radial range, pulse fluency level, vascular tightness mark and pulsation rhythm uniformity mark of the current user. For example, a visual detection element can be used to detect the pulse axial range length, radial radius corresponding to the pulse radial range and vascular tightness mark of the current user's pulse manifestation site, and a piezoelectric sensing element can be used in conjunction with a timing service element to measure the pulse fluency and pulsation rhythm uniformity of the current user's pulse manifestation site, thereby acquiring the pulse fluency level and pulsation rhythm uniformity mark of the current user's pulse manifestation site;

[0057] The second acquisition method: a Chinese medicine doctor manually feels the pulse of the current user to obtain the depth of the pulse manifestation site, pulse frequency, pulse strength, pulse axial range length, radial radius corresponding to the pulse radial range, pulse fluency level, vascular tension mark and pulse rhythm uniformity mark of the current user. Since these parameters are the 8 most common parameters of Chinese medicine pulse feeling, they are not described here;

[0058] The third acquisition method: some parameters are obtained by using a variety of different sensor elements, and the remaining parameters are obtained by taking the pulse manually by a traditional Chinese medicine doctor;

[0059] The fourth acquisition method: for each parameter, two sets of data are obtained electronically by the corresponding sensor element and manually by a TCM physician, and the average of the two sets of data is taken as the measured value of the parameter, so as to be used for intelligent analysis of the type of TCM diagnosis disease that the current user is most likely to suffer from;

[0060] Step S203: performing a preset number of training operations on the deep neural network to obtain a deep neural network after performing the preset number of training operations, and outputting the deep neural network after performing the preset number of training operations as a multimodal dialogue model;

[0061] Specifically, performing a preset number of training operations on the deep neural network to obtain the deep neural network after performing the preset number of training operations, and outputting the deep neural network after performing the preset number of training operations as the multimodal dialogue model includes: after each training of the deep neural network, the model parameters of the corresponding multimodal dialogue model change;

[0062] In fact, an untrained deep neural network can be used as the multimodal dialogue model in the most initial state. Each time the deep neural network completes training, the model parameters of the multimodal dialogue model change. After executing a preset number of training operations, the deep neural network is the multimodal dialogue model in the final state.

[0063] Step S204: using the multimodal dialogue model to intelligently analyze the TCM diagnosis disease type with the highest probability of the current user suffering from the disease based on various physiological parameters of the current user, L component values, A component values ​​and B component values ​​of various pixel points of the tongue sub-image and various pulse data of the current user, and outputting the type as the target TCM diagnosis disease type;

[0064] Specifically, the multimodal dialogue model is used to intelligently analyze the TCM diagnosis disease type of the current user with the highest probability of being ill based on various physiological parameters of the current user, L component values, A component values ​​and B component values ​​of various pixels of the tongue sub-screen and various pulse data of the current user, and output as the target TCM diagnosis disease type, including: the number of pixels of various pixels of the tongue sub-screen is less than the number of pixels of various pixels of the real-time acquisition screen;

[0065] Step S205: Retrieving the TCM prescription that matches the target TCM diagnosis disease type and is most frequently used in history from the TCM prescription database as a recommended treatment prescription for on-site display;

[0066] For example, calling the TCM prescription that matches the target TCM diagnosis disease type and is most frequently used in history from the TCM prescription database as a recommended treatment prescription for on-site display includes: the TCM prescription database is a network database or a hierarchical database;

[0067] The various physiological parameters of the current user are the ASCII code values ​​of the age information, gender identifier, weight information, height information and the type name of the traditional Chinese medicine diagnosis disease type of the current user;

[0068] The method further comprises: performing a preset number of training operations on the deep neural network to obtain the deep neural network after performing the preset number of training operations, and outputting the deep neural network after performing the preset number of training operations as the multimodal dialogue model, including: the number of training operations of the deep neural network is proportional to the total number of pixels of the real-time acquisition screen;

[0069] For example, the number of training operations of the deep neural network is proportional to the total number of pixels of the real-time acquisition picture, including: the greater the total number of pixels of the real-time acquisition picture, the greater the number of training operations of the deep neural network;

[0070] The method further comprises: performing a preset number of training operations on the deep neural network to obtain a deep neural network after performing the preset number of training operations, and outputting the deep neural network after performing the preset number of training operations as a multimodal dialogue model, wherein the number of hidden layers of the deep neural network is monotonically positively correlated with the total number of TCM diagnosis disease types stored in the TCM prescription database;

[0071] For example, the number of hidden layers of the deep neural network is monotonically positively correlated with the total number of TCM diagnosis disease types stored in the TCM prescription database, including: the total number of TCM diagnosis disease types stored in the TCM prescription database is in the millions, the number of hidden layers of the deep neural network is 10, the total number of TCM diagnosis disease types stored in the TCM prescription database is in the five hundred thousand level, the number of hidden layers of the deep neural network is 7, the total number of TCM diagnosis disease types stored in the TCM prescription database is in the one hundred thousand level, the number of hidden layers of the deep neural network is 5, and so on;

[0072] Wherein, performing a preset number of training operations on the deep neural network to obtain a deep neural network after performing the preset number of training operations, and outputting the deep neural network after performing the preset number of training operations as a multimodal dialogue model further comprises: in each training operation performed on the deep neural network, using the type of TCM diagnosis disease currently suffered by a certain user as a single output data of the deep neural network, using various physiological parameters of the certain user, L component values, A component values ​​and B component values ​​of various pixel points of the tongue sub-screen corresponding to the certain user and various pulse data of the certain user as multiple input data of the deep neural network to complete this training operation;

[0073] Among them, performing screen capture of the tongue environment of the current user participating in the AI ​​question and answer of traditional Chinese medicine knowledge to obtain a corresponding real-time capture screen, identifying the tongue sub-screen in the real-time capture screen based on the external imaging features of the human tongue, and obtaining the L component value, A component value and B component value of each pixel point of the tongue sub-screen includes: using an image acquisition component to complete the screen capture of the real-time capture screen, using a feature recognition component connected to the image acquisition component to complete the recognition of the tongue sub-screen, and using a tongue image detection component connected to the feature recognition component to complete the acquisition of the L component value, A component value and B component value of each pixel point of the tongue sub-screen;

[0074] And wherein, performing on-site capture of various pulse data of the current user to obtain various pulse data of the current user, wherein each pulse data includes the depth of the pulse manifestation site, the pulse frequency, the pulse intensity, the length of the axial range of the pulse, the radial radius corresponding to the radial range of the pulse, the pulse fluency level, the vascular tension mark and the pulse rhythm uniformity mark, comprises: using a pulse capture component to perform on-site capture of various pulse data of the current user. Example 2

[0075] Figure 3 The present invention is a flowchart of the steps of an AI question-answering method for traditional Chinese medicine knowledge based on a multimodal dialogue model according to Example 2 of the present invention.

[0076] like Figure 3 As shown, Figure 2 Different from the embodiment in, after step S205, the method further includes:

[0077] Step S301: After the recommended treatment prescription and the user code value of the current user are entered into a network data packet, the network data packet is sent to a big data service network element for executing AI question-and-answer management of traditional Chinese medicine knowledge through a wireless communication link;

[0078] Alternatively, a blockchain service network element or a cloud computing service network element may be selected to replace the big data service network element to perform AI question-and-answer management of traditional Chinese medicine knowledge. Example 3

[0079] Figure 4 The present invention is a flowchart of the steps of an AI question-answering method for traditional Chinese medicine knowledge based on a multimodal dialogue model according to Example 3 of the present invention.

[0080] like Figure 4 As shown, Figure 2 Different from the embodiment in, after step S205, the method further includes:

[0081] Step S401: generating respective course dosages of various Chinese medicine raw materials corresponding to the weight information of the current user according to the confirmation operation of the current user on the recommended treatment prescription;

[0082] Specifically, generating respective course dosages of various Chinese medicinal raw materials corresponding to the current user's weight information according to the current user's confirmation operation on the recommended treatment prescription includes: when the current user confirms that the recommended treatment prescription is close to his or her physical condition, the current user enters his or her weight information to complete the implementation of the confirmation operation. Example 4

[0083] Figure 5 The present invention is a flowchart of the steps of an AI question-answering method for traditional Chinese medicine knowledge based on a multimodal dialogue model according to Example 4 of the present invention.

[0084] like Figure 5 As shown, Figure 2 Different from the embodiment in, after step S205, the method further includes:

[0085] Step S501: Binding the user code value of the current user with the recommended treatment prescription to be stored on-site as a consultation result data based on AI question and answer of traditional Chinese medicine knowledge;

[0086] Specifically, binding the user code value of the current user with the recommended treatment prescription to be stored on-site as a medical consultation result data based on AI question and answer of traditional Chinese medicine knowledge includes: selecting a TF storage chip, an MMC storage chip or a static storage chip to bind the user code value of the current user with the recommended treatment prescription to be stored on-site as a medical consultation result data based on AI question and answer of traditional Chinese medicine knowledge. Example 5

[0087] Figure 6 The present invention is a flowchart of the steps of an AI question-answering method for traditional Chinese medicine knowledge based on a multimodal dialogue model according to Example 5 of the present invention.

[0088] like Figure 6 As shown, Figure 2 Different from the embodiment in, after step S203, the method further includes:

[0089] Step S601: using a local storage component to complete model storage of the multimodal dialogue model by storing various model parameters of the multimodal dialogue model;

[0090] By way of example, adopting a local storage component to complete model storage of the multimodal dialogue model by storing various model parameters of the multimodal dialogue model includes: the local storage component is a FLASH storage chip.

[0091] Next, various method embodiments of the present invention are described in detail.

[0092] In the AI ​​question-answering method for TCM knowledge based on a multimodal dialogue model according to various method embodiments of the present invention:

[0093] For the current user participating in the AI ​​question and answer session on TCM knowledge, performing screen capture of the environment where the tongue is located to obtain a corresponding real-time capture screen, identifying a tongue sub-screen in the real-time capture screen based on the external imaging features of the human tongue, and obtaining the L component value, A component value, and B component value of each pixel point of the tongue sub-screen, further comprising: the L component value, A component value, and B component value of each pixel point of the tongue sub-screen are the component values ​​of each pixel point of the tongue sub-screen in the LAB space;

[0094] Among them, performing screen capture of the tongue environment of the current user participating in the AI ​​question and answer of traditional Chinese medicine knowledge to obtain a corresponding real-time capture screen, identifying the tongue sub-screen in the real-time capture screen based on the appearance imaging features of the human tongue, and obtaining the L component value, A component value and B component value of each pixel point of the tongue sub-screen also includes: the appearance imaging features of the human tongue are various standard tongue shape patterns corresponding to various standard human tongue shapes;

[0095] For example, the appearance imaging feature of the human tongue is various standard tongue patterns corresponding to various standard human tongue shapes, including: the various standard tongue patterns have the same resolution, that is, have the same total number of pixels.

[0096] In the AI ​​question-answering method for TCM knowledge based on a multimodal dialogue model according to various method embodiments of the present invention:

[0097] The multimodal dialogue model is used to intelligently analyze the TCM diagnosis disease type of the current user with the highest probability of being ill based on the various physiological parameters of the current user, the L component value, the A component value and the B component value of each pixel point of the tongue sub-screen and the various pulse data of the current user, and output it as the target TCM diagnosis disease type, including: performing binary value conversion processing on the various physiological parameters of the current user, the L component value, the A component value and the B component value of each pixel point of the tongue sub-screen and the various pulse data of the current user, and then synchronously inputting them into the multimodal dialogue model;

[0098] Among them, the multimodal dialogue model is used to intelligently analyze the type of TCM diagnosis of the disease with the highest probability that the current user is currently suffering from the disease based on the various physiological parameters of the current user, the L component value, the A component value and the B component value of each pixel point of the tongue sub-screen and the various pulse data of the current user, and output it as the target TCM diagnosis of the disease type, which also includes: the intelligently analyzed type of TCM diagnosis of the disease with the highest probability that the current user is currently suffering from the disease is numerically represented by the ASCII code value of the corresponding type name.

[0099] And in the AI ​​question-answering method for TCM knowledge based on a multimodal dialogue model according to various method embodiments of the present invention:

[0100] Performing on-site capture of various pulse data of the current user to obtain various pulse data of the current user, wherein the various pulse data include the depth of the pulse manifestation site, the pulse frequency, the pulse intensity, the pulse axial range length, the radial radius corresponding to the pulse radial range, the pulse fluency level, the vascular tension mark and the pulsation rhythm uniformity mark, and further comprising: using different values ​​of the vascular tension mark to indicate whether the vascular of the current user is tight or relaxed, and using different values ​​of the pulsation rhythm uniformity mark to indicate whether the pulsation rhythm of the current user is uniform or uneven;

[0101] Wherein, calling the TCM prescription that matches the target TCM diagnosis disease type and is most frequently used in history from the TCM prescription database as the recommended treatment prescription for on-site display includes: using a touch display component to perform on-site display of the recommended treatment prescription;

[0102] The number of hidden layers of the deep neural network is monotonically positively correlated with the total number of TCM diagnosis disease types stored in the TCM prescription database, including: the deep neural network includes multiple hidden layers, a single output layer and a single input layer, the multiple hidden layers are located at the rear end of the single input layer, and the single output layer is located at the rear end of the multiple hidden layers;

[0103] The number of hidden layers of the deep neural network is monotonically positively correlated with the total number of TCM diagnosis disease types stored in the TCM prescription database, further comprising: using a numerical mapping function to represent a numerical mapping relationship that monotonically positively correlates the number of hidden layers of the deep neural network with the total number of TCM diagnosis disease types stored in the TCM prescription database;

[0104] Among them, the use of a numerical mapping function to represent the numerical mapping relationship of the monotonically positive correlation between the number of hidden layers of the deep neural network and the total number of TCM diagnosis disease types stored in the TCM prescription database includes: in the numerical mapping function, the total number of TCM diagnosis disease types stored in the TCM prescription database is the input data of the numerical mapping function;

[0105] And wherein, the numerical mapping relationship that uses a numerical mapping function to represent the monotonically positive correlation between the number of hidden layers of the deep neural network and the total number of TCM diagnosis disease types stored in the TCM prescription database also includes: in the numerical mapping function, the number of hidden layers of the deep neural network corresponding to the total number of TCM diagnosis disease types stored in the TCM prescription database is the output data of the numerical mapping function.

[0106] In addition, the present invention may also cite the following technical contents to highlight the significant technical progress of the present invention:

[0107] The pulse data of the current user are captured on the spot to obtain the pulse data of the current user, wherein the pulse data include the depth of the pulse appearance part, the pulse frequency, the pulse intensity, the pulse axial range length, the radial radius corresponding to the pulse radial range, the pulse fluency level, the vascular tightness mark and the pulsation rhythm uniformity mark, and further include: when the vascular tightness mark adopts the value of 0B00 and 0B01, it indicates whether the vascular of the current user is tight or relaxed respectively;

[0108] Wherein, the on-site capture of various pulse data of the current user is performed to obtain various pulse data of the current user, wherein the various pulse data include the depth of the pulse manifestation site, the pulse frequency, the pulse intensity, the length of the pulse axial range, the radial radius corresponding to the pulse radial range, the pulse fluency level, the vascular tension mark and the pulse rhythm uniformity mark, and further include: when the pulse rhythm uniformity mark adopts the values ​​of 0B10 and 0B11, it respectively indicates whether the pulse rhythm of the current user is uniform or uneven;

[0109] The method further comprises: performing a preset number of training operations on the deep neural network to obtain the deep neural network after performing the preset number of training operations, and outputting the deep neural network after performing the preset number of training operations as the multimodal dialogue model, further comprising: using a numerical simulation mode to implement testing and simulation of performing the preset number of training operations on the deep neural network;

[0110] For example, using the numerical simulation mode to implement the test and simulation of each preset number of training operations performed on the deep neural network includes: using the MATLAB toolbox to implement the test and simulation of each preset number of training operations performed on the deep neural network;

[0111] And wherein, the multimodal dialogue model is used to intelligently analyze the type of TCM diagnosis disease with the highest probability that the current user is currently suffering from the disease based on the various physiological parameters of the current user, the L component value, the A component value and the B component value of each pixel point of the tongue sub-screen and the various pulse data of the current user, and output it as the target TCM diagnosis disease type, also including: the L component value, the A component value and the B component value of each pixel point are all between 0-255.

[0112] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present disclosure, rather than to limit it. Although the present disclosure has been described in detail with reference to the above embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the above embodiments, or replace some or all of the technical features therein with equivalents.

Claims

1. A TCM knowledge AI question-answering method based on a multimodal dialogue model, characterized in that: The method comprises: For the current user participating in the AI ​​Q&A of TCM knowledge, a screen capture of the environment where the tongue is located is performed to obtain a corresponding real-time capture screen, a tongue sub-screen in the real-time capture screen is identified based on the external imaging features of the human tongue, and the L component value, A component value, and B component value of each pixel point of the tongue sub-screen are obtained; Performing on-site capture of various pulse data of the current user to obtain various pulse data of the current user, wherein the various pulse data include the depth of the pulse manifestation site, the pulse frequency, the pulse intensity, the length of the pulse axial range, the radial radius corresponding to the pulse radial range, the pulse fluency level, the vascular tension mark and the pulse rhythm uniformity mark; Performing a preset number of training operations on the deep neural network to obtain a deep neural network after performing the preset number of training operations, and outputting the deep neural network after performing the preset number of training operations as a multimodal dialogue model; The multimodal dialogue model is used to intelligently analyze the TCM diagnosis disease type of the current user with the highest probability of being ill based on various physiological parameters of the current user, L component values, A component values ​​and B component values ​​of various pixel points of the tongue sub-image and various pulse data of the current user, and output it as the target TCM diagnosis disease type; Retrieving from the TCM prescription database the TCM prescription that matches the target TCM diagnosis disease type and is most frequently used in history as a recommended treatment prescription for on-site display; The various physiological parameters of the current user are the ASCII code values ​​of the age information, gender identifier, weight information, height information and the type name of the traditional Chinese medicine diagnosis disease type of the current user; The method further comprises: performing a preset number of training operations on the deep neural network to obtain the deep neural network after performing the preset number of training operations, and outputting the deep neural network after performing the preset number of training operations as the multimodal dialogue model, including: the number of training operations of the deep neural network is proportional to the total number of pixels of the real-time acquisition screen; The method further comprises: performing a preset number of training operations on the deep neural network to obtain a deep neural network after performing the preset number of training operations, and outputting the deep neural network after performing the preset number of training operations as a multimodal dialogue model, further comprising: the number of hidden layers of the deep neural network is monotonically positively correlated with the total number of TCM diagnosis disease types stored in the TCM prescription database; Among them, the pulse frequency of the pulse display part of the current user refers to the number of pulse beats in the current user's breathing cycle, the pulse radial range of the current user's pulse display part refers to the maximum radial radius of the width range of the current user's pulse channel; the pulse fluency level of the current user's pulse display part refers to the fluency of the pulse of the current user's pulse display part, the vascular tension mark of the current user's pulse display part is used to indicate whether the vascular vessels of the current user's pulse display part are tight, and the pulsation rhythm uniformity mark of the current user's pulse display part is used to indicate whether the pulsation rhythm of the vascular vessels of the current user's pulse display part is uniform; Wherein, performing a preset number of training operations on the deep neural network to obtain a deep neural network after performing the preset number of training operations, and outputting the deep neural network after performing the preset number of training operations as a multimodal dialogue model further comprises: in each training operation performed on the deep neural network, using the type of TCM diagnosis disease currently suffered by a certain user as a single output data of the deep neural network, using various physiological parameters of the certain user, L component values, A component values ​​and B component values ​​of various pixel points of the tongue sub-screen corresponding to the certain user and various pulse data of the certain user as multiple input data of the deep neural network to complete this training operation; Among them, performing screen capture of the tongue environment of the current user participating in the AI ​​question and answer of traditional Chinese medicine knowledge to obtain a corresponding real-time capture screen, identifying the tongue sub-screen in the real-time capture screen based on the external imaging features of the human tongue, and obtaining the L component value, A component value and B component value of each pixel point of the tongue sub-screen includes: using an image acquisition component to complete the screen capture of the real-time capture screen, using a feature recognition component connected to the image acquisition component to complete the recognition of the tongue sub-screen, and using a tongue image detection component connected to the feature recognition component to complete the acquisition of the L component value, A component value and B component value of each pixel point of the tongue sub-screen; Wherein, performing on-site capture of various pulse data of the current user to obtain various pulse data of the current user, wherein the various pulse data include the depth of the pulse manifestation site, the pulse frequency, the pulse intensity, the pulse axial range length, the radial radius corresponding to the pulse radial range, the pulse fluency level, the vascular tension mark and the pulse rhythm uniformity mark, comprises: performing on-site capture of various pulse data of the current user using a pulse capture component; Among them, on-site capture of various pulse data of the current user is performed to obtain various pulse data of the current user, and the various pulse data include the depth of the pulse manifestation site, pulse frequency, pulse intensity, pulse axial range length, radial radius corresponding to the pulse radial range, pulse fluency level, vascular tension mark and pulsation rhythm uniformity mark. It also includes: when the vascular tension mark adopts the values ​​of 0B00 and 0B01, it respectively indicates whether the blood vessels of the current user are tight or relaxed, and when the pulsation rhythm uniformity mark adopts the values ​​of 0B10 and 0B11, it respectively indicates whether the pulsation rhythm of the current user is uniform or uneven.

2. The AI ​​question-answering method for TCM knowledge based on a multimodal dialogue model as claimed in claim 1, characterized in that: After retrieving the TCM prescription that matches the target TCM diagnosis disease type and is most frequently used in history from the TCM prescription database as a recommended treatment prescription for on-site display, the method further includes: After the recommended treatment prescription and the user code value of the current user are entered into a network data packet, the network data packet is sent to a big data service network element for executing AI question-and-answer management of traditional Chinese medicine knowledge through a wireless communication link.

3. The AI ​​question-answering method for TCM knowledge based on a multimodal dialogue model as claimed in claim 1, characterized in that: After retrieving the TCM prescription that matches the target TCM diagnosis disease type and is most frequently used in history from the TCM prescription database as a recommended treatment prescription for on-site display, the method further includes: According to the current user's confirmation operation on the recommended treatment prescription, respective doses of treatment courses corresponding to various Chinese medicinal raw materials matching the current user's weight information are generated.

4. The AI ​​question-answering method for TCM knowledge based on a multimodal dialogue model as claimed in claim 1, characterized in that: After retrieving the TCM prescription that matches the target TCM diagnosis disease type and is most frequently used in history from the TCM prescription database as a recommended treatment prescription for on-site display, the method further includes: The user code value of the current user is bound to the recommended treatment prescription and stored on-site as a consultation result data based on AI question and answer of traditional Chinese medicine knowledge.

5. The AI ​​question-answering method for TCM knowledge based on a multimodal dialogue model as claimed in claim 1, characterized in that: After performing a preset number of training operations on the deep neural network to obtain a deep neural network after performing the preset number of training operations, and outputting the deep neural network after performing the preset number of training operations as a multimodal dialogue model, the method further includes: A local storage component is used to complete the model storage of the multimodal dialogue model by storing various model parameters of the multimodal dialogue model.

6. The AI ​​question-answering method for TCM knowledge based on a multimodal dialogue model as claimed in claim 5, characterized in that: For the current user participating in the AI ​​question and answer session on TCM knowledge, performing screen capture of the environment where the tongue is located to obtain a corresponding real-time capture screen, identifying a tongue sub-screen in the real-time capture screen based on the external imaging features of the human tongue, and obtaining the L component value, A component value, and B component value of each pixel point of the tongue sub-screen, further comprising: the L component value, A component value, and B component value of each pixel point of the tongue sub-screen are the component values ​​of each pixel point of the tongue sub-screen in the LAB space; Among them, the screen of the tongue environment of the current user participating in the AI ​​question and answer of traditional Chinese medicine knowledge is captured to obtain the corresponding real-time capture screen, the tongue sub-screen in the real-time capture screen is identified based on the shape imaging features of the human tongue, and the L component value, A component value and B component value of each pixel point of the tongue sub-screen are obtained. It also includes: the shape imaging features of the human tongue are various standard tongue shape patterns corresponding to various standard human tongue shapes.

7. The AI ​​question-answering method for TCM knowledge based on a multimodal dialogue model as claimed in claim 5, characterized in that: The multimodal dialogue model is used to intelligently analyze the TCM diagnosis disease type of the current user with the highest probability of being ill based on the various physiological parameters of the current user, the L component value, the A component value and the B component value of each pixel point of the tongue sub-screen and the various pulse data of the current user, and output it as the target TCM diagnosis disease type, including: performing binary value conversion processing on the various physiological parameters of the current user, the L component value, the A component value and the B component value of each pixel point of the tongue sub-screen and the various pulse data of the current user, and then synchronously inputting them into the multimodal dialogue model; Among them, the multimodal dialogue model is used to intelligently analyze the type of TCM diagnosis of the disease with the highest probability that the current user is currently suffering from the disease based on the various physiological parameters of the current user, the L component value, the A component value and the B component value of each pixel point of the tongue sub-screen and the various pulse data of the current user, and output it as the target TCM diagnosis of the disease type, which also includes: the intelligently analyzed type of TCM diagnosis of the disease with the highest probability that the current user is currently suffering from the disease is numerically represented by the ASCII code value of the corresponding type name.

8. The AI ​​question-answering method for TCM knowledge based on a multimodal dialogue model as claimed in claim 5, characterized in that: Performing on-site capture of various pulse data of the current user to obtain various pulse data of the current user, wherein the various pulse data include the depth of the pulse manifestation site, the pulse frequency, the pulse intensity, the pulse axial range length, the radial radius corresponding to the pulse radial range, the pulse fluency level, the vascular tension mark and the pulsation rhythm uniformity mark, and further comprising: using different values ​​of the vascular tension mark to indicate whether the vascular of the current user is tight or relaxed, and using different values ​​of the pulsation rhythm uniformity mark to indicate whether the pulsation rhythm of the current user is uniform or uneven; Wherein, calling the TCM prescription that matches the target TCM diagnosis disease type and is most frequently used in history from the TCM prescription database as the recommended treatment prescription for on-site display includes: using a touch display component to perform on-site display of the recommended treatment prescription; The number of hidden layers of the deep neural network is monotonically positively correlated with the total number of TCM diagnosis disease types stored in the TCM prescription database, including: the deep neural network includes multiple hidden layers, a single output layer and a single input layer, the multiple hidden layers are located at the rear end of the single input layer, and the single output layer is located at the rear end of the multiple hidden layers; The number of hidden layers of the deep neural network is monotonically positively correlated with the total number of TCM diagnosis disease types stored in the TCM prescription database, further comprising: using a numerical mapping function to represent a numerical mapping relationship that monotonically positively correlates the number of hidden layers of the deep neural network with the total number of TCM diagnosis disease types stored in the TCM prescription database; Among them, the use of a numerical mapping function to represent the numerical mapping relationship of the monotonically positive correlation between the number of hidden layers of the deep neural network and the total number of TCM diagnosis disease types stored in the TCM prescription database includes: in the numerical mapping function, the total number of TCM diagnosis disease types stored in the TCM prescription database is the input data of the numerical mapping function; Among them, the numerical mapping relationship that uses a numerical mapping function to represent the monotonically positive correlation between the number of hidden layers of the deep neural network and the total number of TCM diagnosis disease types stored in the TCM prescription database also includes: in the numerical mapping function, the number of hidden layers of the deep neural network corresponding to the total number of TCM diagnosis disease types stored in the TCM prescription database is the output data of the numerical mapping function.

Images