Traditional Chinese medicine voice inquiry system and method for senile diseases

Abstract

The invention relates to the field of traditional Chinese medicine voice inquiry, in particular to a traditional Chinese medicine voice inquiry system and method for senile diseases. The method comprises the following steps: acquiring illness state description voice information and touch input data; performing voice preprocessing and enhancement on the illness state description voice information to obtain an optimized voice signal; performing voice recognition and conversion on the optimized voice signal, obtaining a comprehensive symptom text sequence in combination with touch input data, and performing feature extraction to obtain an emotion feature vector and a rhythm feature vector; and performing semantic disambiguation on the comprehensive symptom text sequence, performing syndrome type matching based on the comprehensive symptom text after semantic disambiguation in combination with the emotional feature vector, dynamically updating a dialectical result, and finally generating a structured diagnosis report. The problems that the elderly patient speech recognition accuracy is low, the collection integrity is insufficient, the traditional inquiry technology cannot associate the'symptom-syndrome-disease 'core logic, and the system performance cannot be continuously optimized along with clinical data accumulation are solved.

Description

Technical Field

[0001] This invention relates to the field of TCM voice consultation, and in particular to a TCM voice consultation system and method for geriatric diseases. Background Technology

[0002] With the accelerating aging of the global population, the proportion of people aged 60 and above has exceeded 20%. The annual growth rate of clinical treatment demand for chronic diseases in the elderly, such as hypertension, diabetes, and osteoporosis, is reaching 12%. The unique advantages of Traditional Chinese Medicine's (TCM) concepts of "prevention of disease" and syndrome differentiation in the prevention and treatment of geriatric diseases are becoming increasingly prominent. Traditional TCM consultation models heavily rely on the physician's clinical experience and human interaction. However, the elderly population generally suffers from three core problems: first, a decline in speech physiological characteristics, manifested as fluctuating speech rate (0.6-1.0 words / second, 40% lower than that of middle-aged and young people), abnormal tone (fundamental frequency range compressed to 100-250Hz), and unclear pronunciation (blurred distinction between voiced and unvoiced sounds); second, a decline in language expression ability, reduced logical coherence, and fragmented symptom descriptions; and third, memory loss, easily overlooking key disease information such as symptom triggers and duration. These problems result in a traditional consultation information collection completeness of less than 65% and a disease description deviation rate exceeding 20%, directly affecting diagnostic accuracy and treatment efficiency.

[0003] Existing intelligent consultation technologies mostly focus on general medical scenarios, exhibiting significant shortcomings in adapting to the elderly and traditional Chinese medicine (TCM) scenarios: First, speech recognition algorithms are not optimized for the characteristics of elderly speech, with general models generally achieving an accuracy rate below 85% in elderly speech scenarios, especially with an error rate exceeding 30% for TCM terms such as "qi stagnation and blood stasis" and "phlegm-dampness obstruction." Second, semantic parsing models lack adaptation to TCM diagnostic logic, only achieving symptom keyword matching and failing to connect the core TCM logic of "symptom-syndrome-disease." Third, a closed-loop technology of "collection-analysis-feedback-iteration" has not been constructed, preventing continuous optimization of system performance with the accumulation of clinical data. Therefore, developing an intelligent consultation system deeply adapted to the characteristics of elderly speech and the TCM diagnostic system is of significant practical importance for improving the quality of TCM diagnosis and treatment of geriatric diseases and alleviating the pressure on medical resources. Summary of the Invention

[0004] This invention provides a TCM voice-based consultation system and method for geriatric diseases to address four specific technical problems: First, the degeneration of the physiological characteristics of speech in the elderly leads to low accuracy in general speech recognition, especially in TCM terminology; second, traditional consultation techniques lack adaptation to TCM diagnostic logic, only achieving symptom keyword matching and failing to link the core logic of "symptom-syndrome-disease"; third, the lack of a closed-loop technology of "collection-analysis-feedback-iteration" prevents continuous optimization of system performance with the accumulation of clinical data; and fourth, the decline in expressive ability and memory in elderly patients results in insufficient completeness of consultation information collection due to existing interaction methods.

[0005] The present invention provides a traditional Chinese medicine voice consultation system and method for geriatric diseases, specifically including the following technical solutions: A method for TCM voice-based diagnosis of geriatric diseases includes the following steps: S1. Acquire elderly patients' medical condition description voice information and touch input data through dual-modal acquisition; preprocess and enhance the medical condition description voice information to obtain an optimized voice signal; perform speech recognition and conversion on the optimized voice signal, and combine it with touch input data to obtain a comprehensive symptom text sequence; extract features from the comprehensive symptom text sequence to obtain emotion feature vector and prosodic feature vector; S2. Construct a structured TCM professional knowledge base, and perform semantic disambiguation on the comprehensive symptom text sequence to obtain semantically disambiguated comprehensive symptom text; perform semantic parsing on the semantically disambiguated comprehensive symptom text to obtain semantic feature vectors; fuse the emotional feature vectors with the semantic feature vectors to generate fused feature vectors; and perform syndrome matching based on the fused feature vectors to generate syndrome differentiation results. S3. Feedback the diagnosis results to the patient, trigger step-by-step follow-up questions based on the diagnosis results, update the diagnosis results, and finally generate a structured diagnostic report.

[0006] Preferably, S1 specifically includes: A two-level recognition architecture of "acoustic layer-language layer" is constructed. At the acoustic layer, the optimized speech signal is decomposed and matched for phoneme segmentation to generate a preliminary text sequence and calculate the acoustic score. At the language layer, a customized BERT model for the field of traditional Chinese medicine is constructed to perform semantic correction and integration, generate a text sequence that conforms to the professional expression of traditional Chinese medicine, and judge the semantic coherence of the preliminary text sequence and calculate the semantic score. Based on the semantic score and the acoustic score, it is determined whether touch input is triggered. The touch input data is converted into standardized text and combined with the text sequence that conforms to the professional expression of traditional Chinese medicine to generate a comprehensive symptom text sequence.

[0007] Preferably, S2 specifically includes: By calculating the semantic similarity between the comprehensive symptom text sequence and the standard expressions in the structured TCM professional knowledge base, semantic correction is performed to obtain the semantically disambiguated comprehensive symptom text.

[0008] Preferably, S2 specifically includes: Based on semantic feature vectors and combined with emotion feature vectors, multi-dimensional features are integrated through vector concatenation and weight optimization to obtain a fused feature vector.

[0009] Preferably, S2 specifically includes: Based on the fusion feature vector and the rhythm feature vector, a standardized integrated feature vector is generated; based on the standardized integrated feature vector, the predicted probability value of the candidate syndrome is calculated to obtain the syndrome differentiation result.

[0010] Preferably, S3 specifically includes: Based on the diagnostic results, voice feedback information is generated and feedback text is displayed simultaneously. Based on the diagnostic results, it is determined whether a tiered follow-up questioning is triggered. When a tiered follow-up questioning is triggered, supplementary symptoms are acquired through bimodal acquisition, and the supplementary symptoms are subjected to feature extraction and integration processing as described in steps S1 and S2 to obtain a standardized integrated feature vector corresponding to the supplementary symptoms. Based on the standardized integrated feature vector corresponding to the supplementary symptoms, combined with the standard feature vector of the target syndrome, the diagnostic results are updated to obtain an updated diagnostic results. The diagnostic results at the point where the tiered follow-up questioning stops are taken as the final diagnostic results, and the maximum value of the final diagnostic results is taken to obtain the highest diagnostic results. Based on the highest diagnostic results, a structured diagnostic report is generated.

[0011] Preferably, S3 specifically includes: The system encrypts and backs up the patient's basic information locally and in the cloud, including the voice description of the illness, the comprehensive symptom text after semantic disambiguation, the highest diagnostic result, and the patient's basic information. It also performs stratified triage based on the highest diagnostic result and records the content of physician intervention. A two-layer annotation platform is built to use the physician intervention content as annotation data to update the diagnostic logic, forming a complete closed loop of collection-analysis-feedback-iteration.

[0012] A traditional Chinese medicine voice consultation system for geriatric diseases includes the following components: The system includes modules for data acquisition and preliminary processing, speech enhancement and feature extraction, TCM semantic analysis and syndrome matching, result feedback and model iteration, hierarchical triage and collaborative diagnosis and treatment, and data storage and security management. The data acquisition and preliminary processing module acquires elderly patients' medical condition description voice information and touch input data through dual-modal acquisition. It performs preliminary noise reduction and effective information filtering on the medical condition description voice data to generate a valid voice segment sequence. Simultaneously, it performs standardization processing on the touch input data to generate standardized text converted from touch input. The valid voice segment sequence and the standardized text converted from touch input are then passed to the speech enhancement and feature extraction module. Finally, the medical condition description voice information and touch input data are simultaneously sent to the data storage and security management module. The speech enhancement and feature extraction module enhances and optimizes effective speech segments to obtain optimized speech signals; it performs speech recognition and conversion on the optimized speech signals, and combines them with standardized text converted from touch input to generate a comprehensive symptom text sequence; it extracts features from the comprehensive symptom text sequence to obtain prosodic feature vectors and emotional feature vectors; and it passes the comprehensive symptom text sequence, prosodic feature vectors, and emotional feature vectors to the TCM semantic analysis and syndrome matching module, and simultaneously sends them to the data storage and security management module. The TCM semantic parsing and syndrome matching module constructs a structured TCM professional knowledge base, performs semantic disambiguation on the comprehensive symptom text sequence to obtain semantically disambiguated comprehensive symptom text; performs semantic parsing on the semantically disambiguated comprehensive symptom text to obtain semantic feature vectors; fuses the semantic feature vectors with the emotional feature vectors to generate a fused feature vector; based on the fused feature vector and combined with the prosodic feature vector, generates a standardized integrated feature vector, and performs syndrome matching to generate the syndrome differentiation result; the syndrome differentiation result is passed to the result feedback and model iteration module, and the semantically disambiguated comprehensive symptom text is simultaneously synchronized to the data storage and security management module. The results feedback and model iteration module generates voice and text feedback based on the diagnostic results and determines whether a tiered follow-up questioning is triggered. When a tiered follow-up questioning is triggered, supplementary symptoms are collected through bimodal acquisition, and feature extraction and integration processing are performed on the supplementary symptoms by modules such as data acquisition and preliminary processing, voice enhancement and feature extraction, and TCM semantic analysis and syndrome matching to obtain a standardized integrated feature vector corresponding to the supplementary symptoms. Based on the standardized integrated feature vector corresponding to the supplementary symptoms, the diagnostic results are updated to obtain the updated diagnostic results. The diagnostic results at the point where the tiered follow-up questioning stops are taken as the final diagnostic results. The maximum value in the final diagnostic results is selected as the highest diagnostic results. Based on the candidate syndrome corresponding to the highest diagnostic results and combined with a structured TCM professional knowledge base, a structured diagnostic report is generated. The highest diagnostic results are passed to the hierarchical triage and treatment collaboration module and simultaneously sent to the data storage and security management module. The stratified triage and treatment collaboration module completes stratified triage based on the highest diagnostic result, records the physician's intervention content and transmits it to the data storage and security management module, uses the intervention data as labeled data, updates the diagnostic logic, and finally forms a complete closed loop of "collection-analysis-feedback-iteration". The data storage and security management module is responsible for storing data throughout the entire process. It adopts a layered storage approach, combining local and cloud storage, and uses transmission encryption and access control to ensure data security.

[0013] The beneficial effects of the technical solution of the present invention are: 1. Improve the accuracy of speech recognition for the elderly: To address the problem of speech degeneration in the elderly, a dual-modal input and spectral subtraction filtering optimization are adopted. A hybrid architecture of Gaussian mixture model-hidden Markov model and BERT model in the field of traditional Chinese medicine is used to improve the accuracy of speech recognition for the elderly, reduce the error rate of TCM terminology recognition, and significantly improve the completeness of symptom collection, thus improving efficiency compared with traditional consultation.

[0014] 2. Enhancing the reliability of TCM syndrome differentiation and diagnosis: Based on the Neo4j TCM knowledge base and a three-layer knowledge graph, the CNN-LSTM-Attention deep learning model and semantic disambiguation technology are used to significantly improve the accuracy of syndrome classification. Dynamic updates to the syndrome differentiation results and tiered follow-up questions reduce diagnostic bias and lessen reliance on physician experience.

[0015] 3. Achieve continuous system evolution: Construct a closed loop of "collection-analysis-feedback-iteration", accumulate data through "AI pre-labeling + physician review", fine-tune the customized BERT model in the field of traditional Chinese medicine for every 10,000 labeled data, improve the accuracy of recognition and diagnosis, and regularly update the structured TCM professional knowledge base to adapt to changes in diagnosis and treatment needs.

[0016] 4. Ensure data security and efficiency: Through local + cloud layered storage, AES-256 encryption, information desensitization, transmission encryption and access control, it complies with medical standards and introduces hash index to achieve data retrieval in seconds, so as to balance privacy and efficiency. Attached Figure Description

[0017] Figure 1 This is a structural diagram of a traditional Chinese medicine voice consultation system for geriatric diseases as described in this invention; Figure 2 This is a flowchart of a traditional Chinese medicine voice consultation method for geriatric diseases as described in this invention. Detailed Implementation

[0018] To further illustrate the technical means and effects adopted by the present invention to achieve its intended purpose, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0019] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0020] The following description, in conjunction with the accompanying drawings, details the specific scheme of the traditional Chinese medicine voice consultation system and method for geriatric diseases provided by this invention.

[0021] See attached document Figure 1 The diagram illustrates a structural diagram of a traditional Chinese medicine voice consultation system for geriatric diseases provided by an embodiment of the present invention. The system includes the following components: The system includes modules for data acquisition and preliminary processing, speech enhancement and feature extraction, TCM semantic analysis and syndrome matching, result feedback and model iteration, hierarchical triage and collaborative diagnosis and treatment, and data storage and security management. The data acquisition and preliminary processing module collects elderly patients' medical condition description voice information and touch input data in a dual-modal manner. The voice data undergoes preliminary noise reduction and effective information filtering to generate a valid voice segment sequence; the touch input data is simultaneously standardized to generate standardized text converted from touch input. The valid voice segment sequence and the standardized text converted from touch input are then passed to the voice enhancement and feature extraction module; and the medical condition description voice information and touch input data are simultaneously passed to the data storage and security management module. The speech enhancement and feature extraction module enhances and optimizes effective speech segment sequences to obtain optimized speech signals; it performs speech recognition and conversion on the optimized speech signals, and combines them with standardized text converted from touch input to generate a comprehensive symptom text sequence; it extracts features from the comprehensive symptom text sequence to obtain prosodic feature vectors and emotional feature vectors; and it passes the comprehensive symptom text sequence, prosodic feature vectors, and emotional feature vectors to the TCM semantic analysis and syndrome matching module, and simultaneously sends them to the data storage and security management module. The TCM semantic parsing and syndrome matching module constructs a structured TCM professional knowledge base, performs semantic disambiguation on the comprehensive symptom text sequence to obtain the semantically disambiguated comprehensive symptom text, performs semantic parsing on the semantically disambiguated comprehensive symptom text to obtain semantic feature vectors, fuses the semantic feature vectors with the emotional feature vectors to generate fused feature vectors, performs syndrome matching based on the fused feature vectors and combines them with prosodic feature vectors to generate syndrome differentiation results, and passes the syndrome differentiation results to the result feedback and model iteration module, while simultaneously synchronizing the semantically disambiguated comprehensive symptom text to the data storage and security management module to achieve "multi-directional distribution of analysis results"; The results feedback and model iteration module generates voice and text feedback based on the diagnostic results. When the diagnostic result is lower than 0.9, it initiates a three-round tiered follow-up questioning process. Each round dynamically updates the diagnostic result by supplementing information until the updated diagnostic result is ≥0.9 or the tiered follow-up questioning reaches three rounds, at which point the follow-up questioning ends, yielding the final diagnostic result. The maximum value among the final diagnostic results is selected as the highest diagnostic result. Based on the candidate syndrome type corresponding to the highest diagnostic result and combined with a structured TCM professional knowledge base, a structured diagnostic report is finally generated. The highest diagnostic result is then passed to the hierarchical triage and treatment collaboration module and simultaneously sent to the data storage and security management module. When data is needed, historical labeled data is also retrieved from the data storage and security management module for iteration. The stratified diagnosis and treatment collaboration module completes stratified diagnosis based on the highest diagnostic results, records the physician intervention content and transmits it to the data storage and security management module. The intervention data will be used as labeled data to help the result feedback and model iteration module optimize the model and the TCM semantic analysis and syndrome matching module update the diagnostic logic, ultimately forming a complete closed loop of "collection-analysis-feedback-iteration". The data storage and security management module stores all data in the process (voice, text, identification results, etc.), and adopts local and cloud-based layered storage to ensure encrypted storage of voice, results and other data. Data privacy and retrieval efficiency are guaranteed through de-identification processing and hash indexing, and data security is ensured through transmission encryption and access control.

[0022] See attached document Figure 2 The diagram illustrates a flowchart of a traditional Chinese medicine voice consultation method for geriatric diseases provided by an embodiment of the present invention. The method includes the following steps: S1. Acquire elderly patients' medical condition description voice information and touch input data through dual-modal acquisition; preprocess and enhance the medical condition description voice information to obtain an optimized voice signal; perform speech recognition and conversion on the optimized voice signal, and combine it with touch input data to obtain a comprehensive symptom text sequence; extract features from the comprehensive symptom text sequence to obtain emotion feature vector and prosodic feature vector; The TCM voice consultation system for geriatric diseases uses a dual-modal input function to collect the voice information describing the patient's condition. It also combines the voice characteristics of the elderly population to complete and preprocess the collected voice information to ensure accurate acquisition of voice information. The whole process is divided into four main stages: dual-modal input acquisition, voice preprocessing and enhancement, voice recognition and conversion, and multi-dimensional feature extraction. The first step is dual-modal input acquisition. Two modes, primary and secondary, are used for both voice and touch input. The primary mode uses a ReSpeaker4-MicArray microphone array (4 channels, 3-meter pickup radius, -38dB±3dB sensitivity) paired with an XMOSXUF216 noise reduction chip to acquire voice information, ensuring accurate voice data acquisition even in noisy environments. The input voice information is then converted from digital to digital (16kHz sampling rate, 16-bit precision), and then subjected to energy-zero-crossing rate dual-threshold endpoint detection (silence threshold < -40dB, zero-crossing rate threshold < 50 times / second) to generate a valid voice segment sequence. The secondary mode uses touch input and is activated when a resampling mechanism is triggered; this mechanism will be explained in detail later. The second step is speech preprocessing and enhancement. To reduce noise and improve the quality of the speech information, the effective speech segment sequence is first preprocessed and enhanced using a combination of spectral subtraction (Hanning window, 25ms window length, 10ms step size) and adaptive filtering. Spectral subtraction reduces the influence of background noise, achieving a signal-to-noise ratio of over 35dB. Adaptive filtering uses a 128th-order least mean square adaptive filter to further optimize the effective speech segment sequence. Both spectral subtraction and adaptive filtering are well-known techniques in the field and will not be elaborated upon here. After the above processing, the effective speech segment sequence successfully filters out environmental noise, such as household noise and equipment interference, outputting a "high-definition, low-interference" optimized speech signal that can be directly used in subsequent speech recognition stages, avoiding recognition errors caused by noise. The third step is speech recognition and conversion. Existing Gaussian Mixture Model-Hidden Markov Model (GMM-HMM) and BERT models from the Traditional Chinese Medicine domain are used for speech recognition and text transcription. A two-tiered recognition architecture, "acoustic layer-language layer," is constructed to accurately convert the optimized speech signal into text. At the acoustic layer, GMM-HMM is used to finely depict the "temporal dynamic process" of phoneme pronunciation, improving phoneme matching accuracy. GMM-HMM sets five states for each phoneme, including pronunciation onset, transition, stability, decay, and termination. After 200 rounds of iterative training, the optimized speech signal is split and matched into phonemes. Continuous speech is decoded into a basic phoneme sequence, which is then concatenated into words and short sentences according to existing phoneme combination rules, generating a preliminary text sequence. The basic phoneme sequence and existing standard phoneme templates are then compared. The matching probability is used to obtain the acoustic score in the [0,1] interval. At the language layer, based on the mature BERT architecture, the existing vocabulary of more than 5,000 TCM-specific terms is expanded, and fine-tuned using 50,000 TCM consultation records of geriatric diseases and symptom description standardization and polysemous term mapping data from the Neo4j TCM knowledge base. This completes the customization of TCM scenarios and constructs a TCM-specific BERT model, enabling it to accurately parse TCM consultation-related expressions such as "inspection, auscultation, inquiry, and palpation." TCM consultation records can be obtained from public databases. The construction method of the above-mentioned TCM-specific BERT model is a well-known technique in the art and will not be elaborated here. Through the TCM-specific BERT model, the basic phoneme sequence is semantically corrected and integrated to correct recognition errors, and finally a text sequence that conforms to TCM professional expressions is generated. Then, the TCM-specific BERT model is combined to judge the semantic coherence of the preliminary text sequence output by the acoustic layer, such as judging whether "qi deficiency accompanied by fatigue" conforms to TCM logic, and obtaining the semantic score in the [0,1] interval. The acoustic score and semantic score are weighted and summed to obtain the confidence score. The weights for acoustic and semantic scores can be set to 40% and 60% respectively, when the confidence level... When the system is active, an automatic re-sampling mechanism is triggered, activating the auxiliary mode for touch input. This allows patients to input supplementary information by clicking on over 300 TCM keywords such as "dizziness" and "fatigue." Based on over 3,000 symptom descriptions in the Neo4j TCM knowledge base, the keywords selected by the patient are directly converted into standardized text. If the auxiliary mode is used for information completion, the text sequence generated in the main mode that conforms to TCM professional terminology needs to be merged and deduplicated with the standardized text converted from touch input in the auxiliary mode. This merging and deduplication uses existing standardized alignment + semantic similarity matching methods based on the TCM symptom specification library. For example, if the text sequence generated in the main mode that conforms to TCM professional terminology is "recently experiencing dizziness and fatigue," and the standardized text converted from touch input is "dizziness symptoms, fatigue symptoms," the two are determined to have semantic overlap by matching core symptom words and merged into "recently experiencing dizziness and fatigue (dizziness symptoms, fatigue symptoms)," ultimately forming a complete and non-redundant comprehensive symptom text sequence to ensure sufficient raw material for subsequent processing. The fourth step is multi-dimensional feature extraction. Emotional and prosodic feature vectors are extracted based on the optimized speech signal. When extracting prosodic and emotional features from the audio, the Mel-frequency cepstral coefficient (MFCC) algorithm is first used to extract 13th-order prosodic features from the optimized speech signal. Then, the first and second-order differences of the 13th-order prosodic features are added to expand them into a 15-dimensional prosodic feature vector, which more accurately represents the dynamic changes in rhythm and intonation of elderly speech, such as speech rate fluctuations and pitch changes. The 15-dimensional prosodic feature vector can effectively represent the rhythm and intonation changes of speech, providing a foundation for subsequent analysis. Then, an SVM classifier (using the RBF kernel function with a penalty coefficient C=10) is used to identify the emotional features in the optimized speech signal, classifying emotions into five categories: anxiety, pain, calmness, irritability, and fatigue, outputting a 5-dimensional emotional feature vector. ,in This serves as the time dimension identifier for feature extraction. Each element in the emotion feature vector corresponds to the intensity value of a certain emotion, with the intensity value ranging from [0,1]. A larger value indicates a stronger emotion. The Mel frequency cepstral coefficients and SVM classifiers mentioned above are techniques well-known to those skilled in the art and will not be elaborated upon here.

[0023] S2. Construct a structured TCM professional knowledge base, and perform semantic disambiguation on the comprehensive symptom text sequence to obtain semantically disambiguated comprehensive symptom text; perform semantic parsing on the semantically disambiguated comprehensive symptom text to obtain semantic feature vectors; fuse the emotional feature vectors with the semantic feature vectors to generate fused feature vectors; and perform syndrome matching based on the fused feature vectors to generate syndrome differentiation results. First, a dual-driven semantic parsing engine of "Neo4j knowledge base + Word2Vec reasoning" is adopted to eliminate the semantic ambiguity of the comprehensive symptom text sequence from two aspects: term specification and context matching. In the construction of the knowledge base, based on the professional knowledge logic in the field of traditional Chinese medicine, a structured traditional Chinese medicine professional knowledge base is sorted out and constructed, storing more than 400 mapping relationships of polysemous terms. Based on clinical experience, historical medical interview data, and classic traditional Chinese medicine literature, a probability value in the range of [0,1] is assigned to each standard semantics that the symptom may correspond to (such as "getting angry" corresponding to "heat syndrome", "yin deficiency", "excessive liver fire"), for example, "getting angry" corresponds to "heat syndrome (0.6), yin deficiency (0.3), excessive liver fire (0.1)", quantifying the semantic association; at the same time, more than 3,000 symptom description specifications are included, such as standardizing "always feeling tired" to "fatigue - moderate", unifying the symptom expression standard; based on national / industry standard specifications, classic medical books and authoritative works, the clinical experience of old Chinese medicine practitioners and real medical records, and authoritative discipline textbooks, the matching relationships of different syndrome types and standard treatment plans are integrated to provide a theoretical basis for personalized treatment plans. The structured traditional Chinese medicine professional knowledge base is stored in the Neo4j traditional Chinese medicine knowledge base, and the complex associations between nodes are efficiently processed using the structure of the traditional Chinese medicine knowledge graph that stores traditional Chinese medicine term knowledge and association relationships to support fast retrieval and reasoning; Perform disambiguation processing on the comprehensive symptom text sequence. Use 50,000 traditional Chinese medicine medical interview records as training corpus to train the Word2Vec model, so that the Word2Vec model can understand the semantics of traditional Chinese medicine texts; then embed the comprehensive symptom text sequence and the basic information and past medical history of the patients collected in the login session of the traditional Chinese medicine voice medical interview system for the elderly into the complete traditional Chinese medicine medical interview record, which can be used as a supplementary basis for verifying the similarity matching results later and can assist in completing the similarity judgment results of the Word2Vec model; use the trained Word2Vec model to calculate the semantic similarity between the comprehensive symptom text sequence and the standard expressions (such as "heat syndrome", "yin deficiency") in the structured traditional Chinese medicine professional knowledge base: when the semantic similarity ≥ 0.7, the Word2Vec model matches the context to complete semantic correction, and the comprehensive symptom text after semantic disambiguation is obtained, such as parsing "always getting angry recently, dry mouth, dry stools" into "heat syndrome - dry mouth symptom - constipation symptom", so as to effectively solve the problem of semantic ambiguity in traditional Chinese medicine medical interviews; Furthermore, use the Text Convolutional Neural Network (TextCNN) model to perform semantic parsing on the comprehensive symptom text after semantic disambiguation. The TextCNN model can automatically extract the key semantic information in the comprehensive symptom text after semantic disambiguation through the convolutional neural network structure, and finally output a 256-dimensional semantic feature vector Text convolutional neural networks are a well-known technique in the field and will not be elaborated upon here. The 256-dimensional semantic feature vector digitally represents the semantic information of the semantically disambiguated comprehensive symptom text, facilitating computer processing and analysis. To fully leverage the advantages of the extracted semantic feature vectors of different dimensions, a vector concatenation and weight optimization method is used to integrate multi-dimensional features, resulting in a fused feature vector. The specific formula is as follows:

[0024] in, This is a vector concatenation operator used to concatenate 256-dimensional semantic feature vectors. and 5-dimensional emotion feature vector The vectors are concatenated to construct a 261-dimensional initial vector, which integrates both semantic and emotional information. The feature weight matrix is ​​261×128 dimensional. Based on the correlation of TCM symptoms, it is generated by optimizing multi-class cross-entropy. Multi-class cross-entropy is a technique well-known to those skilled in the art and will not be elaborated here. Through matrix multiplication, a 128-dimensional fused feature vector is finally output. , This is the feature vector dimension index, with a value range of 1. ; Furthermore, based on professional knowledge in the field of Traditional Chinese Medicine and experience in the diagnosis and treatment of geriatric diseases, a knowledge graph was constructed. First, over 120 common syndrome types, over 300 characteristic symptoms, and over 50 related diseases in geriatric diseases were identified, clarifying the logical relationship of "symptoms corresponding to syndromes, and syndromes related to diseases." Then, the information obtained was organized into a knowledge graph according to a three-layer structure of "symptom-syndrome-disease," and combined with a CNN-LSTM-Attention deep learning model for syndrome differentiation and classification. Specifically, a 15-dimensional prosodic feature vector and a 128-dimensional fused feature vector were simultaneously input into the CNN-LSTM-Attention deep learning model. Two convolutional kernels were set in the CNN layer, with kernel sizes of 3×128 and 5×128 respectively. When extracting local features from the over 300 common characteristic symptoms of geriatric diseases, dynamic information such as "speech rate and energy" from the 15-dimensional prosodic feature vector was simultaneously incorporated. For example, "slow speech rate (below 0.6 words / second) and low speech energy (<...)" were considered as... Features such as "-35dB" are associated with the local symptom features of "Qi deficiency syndrome". The LSTM layer contains two layers, each with 256 hidden units. When capturing the temporal features of symptoms, it incorporates the "fundamental frequency fluctuation" information of the 15-dimensional prosodic feature vector. For example, "fundamental frequency continuously below 120Hz and fluctuation amplitude <30Hz" may correspond to the chronic trend of symptom development, which helps to judge the logic of syndrome evolution. The Attention layer adopts an additive attention mechanism. For key features such as "pain nature" and "attack frequency", it assigns differentiated dynamic weights according to different syndromes. At the same time, it further increases the weight of abnormal features such as "speech interruption (zero crossing rate fluctuation >60 times / second) and stuttering (effective speech segment interval >1.5 seconds)" in the weight calculation of the corresponding associated syndrome (the weight ratio is increased to 0.15-0.2), so that the CNN-LSTM-Attention deep learning model focuses on the disease clues at the dynamic level of speech and accurately matches specific syndromes. The CNN-LSTM-Attention deep learning model integrates a 15-dimensional prosodic feature vector and a 128-dimensional fusion feature vector to obtain a standardized integrated feature vector. Based on this standardized integrated feature vector, and combined with the Softmax function, predicted probability values ​​for candidate syndrome types are generated, providing reliable intelligent assistance for the diagnosis of geriatric diseases using traditional Chinese medicine. The specific formula is as follows:

[0025] in, The result of the dialectical analysis represents time. Next Predicted probability values ​​of candidate certificate types, The value range is [0,1]; This is the candidate syndrome index, i.e., the index of common syndromes in geriatric diseases. The sum of the syndrome differentiation results of all candidate syndromes is 1. As an activation function, its core function is to convert the linear output of "feature weighted summation + bias correction" into a distribution form that conforms to the probability axiom. The input is the linear calculation result, and the output is the probability value in the interval [0,1], so as to ensure that the predicted probability values ​​of each candidate type are mutually exclusive and normalized. Here, attention weights represent the number of attention weights. The standardized integrated features for the first The specific contribution weight of the candidate certificate type, where the subscript An index for the integrated features after standardization. The value range is [0,1], which is obtained by the CNN-LSTM-Attention deep learning model through backpropagation iteration, and can accurately characterize the differential impact of a single feature on different syndrome types; Represents time The next A standardized integrated feature It is a fusion feature vector Obtained through a CNN-LSTM-Attention deep learning model; As a bias term, it is the first... The specific correction term for the candidate certificate type is obtained by optimization using the gradient descent method, and its value range is [-0.1, 0.1]. The gradient descent method is a well-known technique in the art and will not be elaborated here.

[0026] S3. Feedback the diagnosis results to the patient, trigger step-by-step follow-up questions based on the diagnosis results, update the diagnosis results, and finally generate a structured diagnostic report.

[0027] Based on dialectical results Using the existing STRAIGHT parametric synthesis algorithm, it generates speech that is more adapted to the hearing and comprehension habits of the elderly, providing clear and easy-to-understand voice interaction support for the TCM voice consultation system for geriatric diseases, helping the elderly to smoothly obtain relevant voice information feedback results, such as "Based on your description, the preliminary judgment may be Qi deficiency syndrome. To be more accurate, how long have these symptoms lasted?", and the feedback text is displayed simultaneously on the touch screen. When the dialectical result When the value is less than 0.9, up to three rounds of step-by-step questioning are triggered. The elderly user will then supplement their symptoms again via voice (collected by a microphone array) or touchscreen input. The supplemented symptoms will undergo feature extraction and integration processing as in steps S1 and S2 to obtain a standardized integrated feature vector corresponding to the supplemented symptoms. Based on the standardized integrated feature vector corresponding to the supplementary symptoms, the syndrome differentiation results are updated using a dynamic optimization formula based on the TCM syndrome differentiation results, resulting in the updated syndrome differentiation results. The specific formula is as follows:

[0028] in, It is in time According to the updated supplementary symptoms The predicted probability value of the candidate certificate type represents the probability of the first type. The dialectical result of the candidate syndrome type, that is, the updated dialectical result; The feedback weighting coefficient represents the importance of the supplementary symptoms, with a value range of [0.1, 0.3]. It is dynamically adjusted based on the importance of the supplementary symptoms, including key information such as symptom duration. The value is 0.3, and secondary information such as minor discomfort is included. The value is 0.1; To assess the similarity between the standardized integrated feature vector corresponding to the supplementary symptom and the standard feature vector of the corresponding syndrome, cosine similarity is used, with a value range of [0,1], representing the degree of matching between the supplementary symptom and the standard feature of the target syndrome. for The standard feature vectors corresponding to the syndrome types are extracted from the knowledge graph constructed in the syndrome classification process.

[0029] After each update, the dialectical results are reassessed, and the updated dialectical results... When the value is ≥0.9 or after 3 rounds of follow-up questioning, the questioning is terminated, and the corresponding dialectical result is taken as the final dialectical result. Then, the maximum value among the final dialectical results is selected as the highest dialectical result, and the candidate syndrome types corresponding to the highest dialectical result are determined. The system retrieves standard treatment plans corresponding to the syndrome types from a structured TCM professional knowledge base, and the combination of these two approaches generates a structured diagnostic report. The structured diagnostic report includes six modules: "basic information, symptom summary, syndrome differentiation results, pathogenesis analysis, treatment suggestions, and precautions." TCM terminology is accompanied by simplified explanations (e.g., "Qi deficiency syndrome: simply put, it means insufficient 'qi' in the body, leading to fatigue and weakness"). After the diagnostic report is generated, the geriatric TCM voice consultation system automatically stores the original patient description voice information, semantically disambiguated comprehensive symptom text, the highest syndrome differentiation result, and the patient's basic information using a "local encrypted storage + cloud-based layered backup" architecture. Voice data is encrypted using AES-256, and the patient's basic information collected during the consultation login process is anonymized, such as age blurring and name replacement, and retrieval efficiency is optimized through hash indexing. Data transmission uses the TLS 1.3 protocol throughout, with access controlled by a three-level permission system: "patient-physician-administrator," ensuring data security and compliance.

[0030] Based on the highest diagnostic result in the final diagnosis, a tiered triage system is implemented: when the highest diagnostic result is ≥0.95, the structured diagnostic report will be directly pushed to the attending physician and simultaneously shared with the patient's family; when the highest diagnostic result is ≤0.8 and <0.95, an online review process by the physician is triggered; when the highest diagnostic result is <0.8, offline consultation appointments are prioritized. The geriatric TCM voice consultation system can automatically record physician interventions, such as correcting syndrome types and supplementing symptoms, and incorporate them into the labeled dataset for subsequent iterations and knowledge base updates, supporting collaborative diagnosis and treatment.

[0031] Finally, by building a two-layer annotation platform of "AI pre-annotation + physician review", the original dataset formed by the five-part data of "original speech - recognized text - disambiguation results - final diagnosis results - patient desensitization information" is automatically stored. The AI ​​pre-labeling uses a rule-matching algorithm, and the labeled dataset is obtained by scoring by TCM physicians at the deputy director level or above (3 points or above is considered a valid label). For every 10,000 labeled data points accumulated, the geriatric TCM voice consultation system will fine-tune the customized BERT model for TCM using the PyTorch framework to improve the accuracy of recognition and diagnosis. At the same time, it will regularly update the structured TCM professional knowledge base to ensure that new knowledge is introduced in a timely manner.

[0032] In summary, a TCM voice-based consultation system and method for geriatric diseases has been developed.

[0033] The order of the embodiments is for illustrative purposes only and does not represent the superiority or inferiority of the embodiments. The processes depicted in the drawings do not necessarily require a specific or sequential order to achieve the desired result. In some embodiments, multitasking and parallel processing are possible or may be advantageous.

[0034] The various embodiments in this specification are described in a progressive manner. The same or similar parts between the various embodiments can be referred to each other. Each embodiment focuses on describing the differences from other embodiments.

[0035] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included within the protection scope of the present invention.

Claims

1. A method for TCM voice-based diagnosis of geriatric diseases, characterized in that, Includes the following steps: S1. Acquire elderly patients' medical condition description voice information and touch input data through dual-modal acquisition; preprocess and enhance the medical condition description voice information to obtain an optimized voice signal; perform speech recognition and conversion on the optimized voice signal, and combine it with touch input data to obtain a comprehensive symptom text sequence; extract features from the comprehensive symptom text sequence to obtain emotion feature vector and prosodic feature vector; S2. Construct a structured TCM professional knowledge base, and perform semantic disambiguation on the comprehensive symptom text sequence to obtain semantically disambiguated comprehensive symptom text; perform semantic parsing on the semantically disambiguated comprehensive symptom text to obtain semantic feature vectors; fuse the emotional feature vectors with the semantic feature vectors to generate fused feature vectors; based on By fusing feature vectors, performing syndrome matching, and generating diagnostic results; S3. Feedback the diagnosis results to the patient, trigger step-by-step follow-up questions based on the diagnosis results, update the diagnosis results, and finally generate a structured diagnostic report.

2. The method for TCM voice-based diagnosis of geriatric diseases according to claim 1, characterized in that, S1 specifically includes: A two-level recognition architecture of "acoustic layer-language layer" is constructed. At the acoustic layer, the optimized speech signal is decomposed and matched for phoneme segmentation to generate a preliminary text sequence and calculate the acoustic score. At the language layer, a customized BERT model for the field of traditional Chinese medicine is constructed to perform semantic correction and integration, generate a text sequence that conforms to the professional expression of traditional Chinese medicine, and judge the semantic coherence of the preliminary text sequence and calculate the semantic score. Based on the semantic score and the acoustic score, it is determined whether touch input is triggered. The touch input data is converted into standardized text and combined with the text sequence that conforms to the professional expression of traditional Chinese medicine to generate a comprehensive symptom text sequence.

3. The method for TCM voice-based diagnosis of geriatric diseases according to claim 1, characterized in that, S2 specifically includes: By calculating the semantic similarity between the comprehensive symptom text sequence and the standard expressions in the structured TCM professional knowledge base, semantic correction is performed to obtain the semantically disambiguated comprehensive symptom text.

4. The method for TCM voice-based diagnosis of geriatric diseases according to claim 1, characterized in that, S2 specifically includes: Based on semantic feature vectors and combined with emotion feature vectors, multi-dimensional features are integrated through vector concatenation and weight optimization to obtain a fused feature vector.

5. The method for TCM voice-based diagnosis of geriatric diseases according to claim 4, characterized in that, S2 specifically includes: Based on the fusion feature vector and the rhythm feature vector, a standardized integrated feature vector is generated; based on the standardized integrated feature vector, the predicted probability value of the candidate syndrome is calculated to obtain the syndrome differentiation result.

6. The method for TCM voice-based diagnosis of geriatric diseases according to claim 1, characterized in that, S3 specifically includes: Based on the diagnostic results, voice feedback information is generated and feedback text is displayed simultaneously. Based on the diagnostic results, it is determined whether a tiered follow-up questioning is triggered. When a tiered follow-up questioning is triggered, supplementary symptoms are acquired through bimodal acquisition, and the supplementary symptoms are subjected to feature extraction and integration processing as described in steps S1 and S2 to obtain a standardized integrated feature vector corresponding to the supplementary symptoms. Based on the standardized integrated feature vector corresponding to the supplementary symptoms, combined with the standard feature vector of the target syndrome, the diagnostic results are updated to obtain an updated diagnostic results. The diagnostic results at the point where the tiered follow-up questioning stops are taken as the final diagnostic results, and the maximum value of the final diagnostic results is taken to obtain the highest diagnostic results. Based on the highest diagnostic results, a structured diagnostic report is generated.

7. A method for TCM voice-based diagnosis of geriatric diseases according to claim 6, characterized in that, S3 specifically includes: The system encrypts and backs up the patient's basic information locally and in the cloud, including the voice description of the illness, the comprehensive symptom text after semantic disambiguation, the highest diagnostic result, and the patient's basic information. It also performs stratified triage based on the highest diagnostic result and records the content of physician intervention. A two-layer annotation platform is built to use the physician intervention content as annotation data to update the diagnostic logic, forming a complete closed loop of collection-analysis-feedback-iteration.

8. A traditional Chinese medicine voice consultation system for geriatric diseases, applied to the traditional Chinese medicine voice consultation method for geriatric diseases as described in claim 1, characterized in that, Includes the following parts: The system includes modules for data acquisition and preliminary processing, speech enhancement and feature extraction, TCM semantic analysis and syndrome matching, result feedback and model iteration, hierarchical triage and collaborative diagnosis and treatment, and data storage and security management. The data acquisition and preliminary processing module acquires elderly patients' medical condition description voice information and touch input data through dual-modal acquisition. It performs preliminary noise reduction and effective information filtering on the medical condition description voice data to generate effective voice segment sequences. Simultaneously, it performs standardization processing on the touch input data to generate standardized text converted from touch input. The effective voice segment sequences and standardized text converted from touch input are then passed to the speech enhancement and feature extraction module. Synchronize the voice information describing the illness and the touch input data to the data storage and security management module; The speech enhancement and feature extraction module enhances and optimizes effective speech segments to obtain optimized speech signals; it performs speech recognition and conversion on the optimized speech signals, and combines them with standardized text converted from touch input to generate a comprehensive symptom text sequence; it extracts features from the comprehensive symptom text sequence to obtain prosodic feature vectors and emotional feature vectors; and it passes the comprehensive symptom text sequence, prosodic feature vectors, and emotional feature vectors to the TCM semantic analysis and syndrome matching module, and simultaneously sends them to the data storage and security management module. The TCM semantic parsing and syndrome matching module constructs a structured TCM professional knowledge base, performs semantic disambiguation on the comprehensive symptom text sequence to obtain the semantically disambiguated comprehensive symptom text, performs semantic parsing on the semantically disambiguated comprehensive symptom text to obtain semantic feature vectors, and fuses the semantic feature vectors with the emotional feature vectors to generate a fused feature vector; based on... The feature vectors are fused and combined with the prosodic feature vectors to generate a standardized integrated feature vector, and then the syndrome matching is performed to generate the syndrome differentiation result. The syndrome differentiation result is passed to the result feedback and model iteration module, and the semantically disambiguated comprehensive symptom text is synchronized to the data storage and security management module. The results feedback and model iteration module generates voice and text feedback based on the dialectical results and determines whether to trigger step-by-step follow-up questions. When the step-by-step questioning is triggered, supplementary symptoms are collected through bimodal acquisition. These supplementary symptoms are then processed through modules such as data acquisition and preliminary processing, speech enhancement and feature extraction, and TCM semantic analysis and syndrome matching to obtain standardized integrated feature vectors corresponding to the supplementary symptoms. Based on these standardized integrated feature vectors, the diagnostic results are updated to obtain the updated diagnostic results. The diagnostic results at the point where the step-by-step questioning stops are taken as the final diagnostic results. The maximum value among the final diagnostic results is selected as the highest diagnostic result. Based on the candidate syndrome types corresponding to the highest diagnostic result, and combined with the structured TCM professional knowledge base, a structured diagnostic report is generated. The highest diagnostic results are transmitted to the tiered triage and treatment collaboration module, and simultaneously transmitted to the data storage and security management module. The stratified triage and treatment collaboration module completes stratified triage based on the highest diagnostic result, records the physician's intervention content and transmits it to the data storage and security management module, uses the intervention data as labeled data, updates the diagnostic logic, and finally forms a complete closed loop of "collection-analysis-feedback-iteration"; The data storage and security management module is responsible for storing data throughout the entire process. It adopts a layered storage approach, combining local and cloud storage, and uses transmission encryption and access control to ensure data security.

Images