Voice interaction analysis system for monitoring and correcting human voice change in perioperative period

Abstract

A voice interaction analysis system for monitoring and correcting human voice changes in the perioperative period identifies the types of pronunciation function damage by analyzing the specific sound wave characteristics of different human voice pronunciation changes, and quickly finds the cause of pronunciation change. The problem that at present, the pronunciation change degree in the perioperative period cannot be quantified is solved, quantitative grading of the injury degree is conducted according to the sound wave condition, and the recovery degree in the treatment recovery period can also be evaluated and monitored. By using artificial intelligence machine learning and voice interaction technologies, through sound wave data analysis, and using a loudspeaker voice device, different pronunciation function damage conditions such as hoarseness and low sound can be improved and corrected. Therefore, the project can meet scientific research, clinical and actual requirements of patients, and has an extremely wide market prospect.

Description

Technical Field

[0001] The present invention relates to the field of medical rehabilitation, and in particular to a voice interaction analysis system for perioperative human voice change monitoring and correction. Background Art

[0002] In recent years, with the development of the economy and society and the improvement of diagnostic technology, the incidence of thyroid nodules has increased year by year. Thyroid surgery is currently a commonly used treatment method. The changes in human voice during the perioperative period of thyroid surgery are a major clinical problem that troubles surgeons and patients. Common causes include laryngeal nerve injury, mechanical injury of anesthesia intubation, vocal cord polyps, vocal cord inflammatory edema, etc. Laryngeal nerve injury includes recurrent laryngeal nerve injury and superior laryngeal nerve injury. The pronunciation of recurrent laryngeal nerve injury is hoarseness, and the pronunciation of superior laryngeal nerve injury is deep. Both of these conditions affect the patient's normal pronunciation function, affect the patient's social ability, and even seriously affect the patient's quality of life. Anesthesia intubation injury is often caused by injury to the vocal cords or pronunciation-related muscles or joints, and pathological changes such as vocal cord polyps and vocal cord inflammatory edema can also cause changes in pronunciation function.

[0003] At present, the commonly used examination method is invasive electronic laryngoscopy. Patients often feel uncomfortable, the medical experience is poor, and it cannot be performed anytime and anywhere. Moreover, the laryngoscope can only observe the activity of the vocal cords, there is no refined severity grading, and it cannot diagnose superior laryngeal nerve damage. Another examination method is laryngeal electromyography. During the test, the needle electrode is inserted into the laryngeal muscle of the person to be tested, and the action potential is amplified by the amplifier, and the laryngeal myoelectric potential graph is displayed on the oscilloscope. This is also an invasive examination, and there is a certain technical difficulty in accurately inserting the needle electrode into the laryngeal muscle. In addition, the laryngeal electromyography is mostly electrical rest in the early stage after nerve damage. Abnormal spontaneous potentials such as fibrillation potentials and positive sharp waves can only appear 10 days after nerve damage, and laryngeal nerve damage cannot be judged immediately. Therefore, there is currently a lack of effective, convenient and non-invasive differential diagnostic tools for changes in pronunciation function caused by the recurrent laryngeal nerve, superior laryngeal nerve and other reasons.

[0004] There is also a lack of quantifiable and refined assessment methods for the quantitative grading of the degree of impairment of pronunciation function, as well as the assessment and monitoring of the degree of recovery during the treatment recovery period. Laryngoscopy can only roughly determine the situation of vocal cord activity restriction, and the assessment results are related to the subjective judgment of the examiner. Repeated comparative examinations of laryngeal electromyography are not only traumatic and increase the patient's pain, but also the examination results are easily affected by the different positions of the needle electrodes each time. Therefore, the above-mentioned examination methods also have certain limitations.

[0005] There is a lack of effective means to improve pronunciation function and correct impaired pronunciation, especially for patients with recurrent laryngeal nerve injury and superior laryngeal nerve injury. Neurotrophic drugs, circulation-improving drugs, and anti-inflammatory and detumescent drugs only have a certain effect on temporary laryngeal nerve injury, but are basically ineffective for permanent nerve damage. Voice training can appropriately improve the deep voice caused by superior laryngeal nerve injury, but it takes a long time and the effect is uncertain, and there are large individual differences. Surgical treatments such as laryngeal nerve reinnervation surgery are difficult to operate and have a low success rate, and cannot be used as a routine treatment.

[0006] Application Contents

[0007] The purpose of this application is to provide a speech interaction analysis system for perioperative human voice change monitoring and correction to solve the technical problems described in the background technology.

[0008] The technical solutions adopted in this application are as follows:

[0009] A speech interaction analysis system for perioperative human voice change monitoring and correction, comprising: a speech element recognition unit, a treatment recovery assessment unit, and a postoperative pronunciation correction unit;

[0010] Voice element recognition unit: A human voice collection system is formed by combining the traditional sound pickup mode with throat-attached sensor devices. High-precision voice recognition of basic voice elements (phonemes, tones, and words) is performed through electroacoustic analysis. The basic voice elements are recorded before surgery, on the day of surgery, 10 days after surgery, 1 month after surgery, 3 months after surgery, and 1 year after surgery. The voice features before and after surgery are compared, and both normal voice and voice with impaired pronunciation are recorded. The electroacoustic manifestations of different impaired voice functions are recorded separately to explore the characteristics of their phonemes, tones, and waveforms, so as to compare normal and abnormal human voices, and diagnose and identify impaired pronunciation functions caused by different reasons.

[0011] Treatment recovery assessment unit: Through the refined analysis of the characteristics of human voice elements, tones, and waveforms, a visual and quantifiable assessment is conducted, the critical value and differentiation of the degree of damage are set, and the degree of damage is quantitatively graded according to the sound wave conditions. It can also assess and monitor the degree of recovery of patients with confirmed impaired pronunciation function during the treatment recovery period;

[0012] Postoperative pronunciation correction unit: Utilizing artificial intelligence machine learning and voice interaction technology, the original voice of the subject before surgery is first recorded, and through voice data analysis, the voice characteristics of the subject in a healthy state are recorded, and machine learning and digital processing are performed to form machine memory. Once the patient's pronunciation function is impaired after surgery, the machine can detect abnormalities in their voice elements on the one hand, and on the other hand, through machine learning memory, automatically record the damaged sound state and make digital adjustments, and then play the adjusted sound close to the patient's healthy state through the connected voice speaker through the player, thereby improving and correcting different pronunciation function impairments such as hoarseness and low voice.

[0013] Compared with the prior art, the advantages and positive effects of the present invention are: the voice interaction analyzer for monitoring and correcting human voice changes during perioperative period of thyroid surgery designed and invented by the applicant makes up for the above three deficiencies of the prior art: by analyzing the specific sound wave characteristics of different human voice changes, the types of pronunciation function damage are identified, and the causes of voice changes are quickly found; the problem of the current unquantifiable degree of pronunciation changes during perioperative period is solved, the degree of damage is quantitatively graded according to the sound wave situation, and the degree of recovery during the treatment recovery period can also be evaluated and monitored; the use of artificial intelligence machine learning and voice interaction technology, through sound wave data analysis, using external voice equipment, improves and corrects different pronunciation function impairments such as hoarseness and low voice. Therefore, this project can meet the actual needs of scientific research, clinical and patients, and has a very broad market prospect. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0015] Figure 1 This is a schematic diagram of sound collection for a speech interaction analysis system for perioperative voice change monitoring and correction.

[0016] Figure 2 The present invention is a schematic diagram of an embodiment of a speech interaction analysis system for perioperative human voice change monitoring and correction.

[0017] Figure 3 The present invention is a workflow diagram of a speech element recognition unit of an embodiment of a speech interaction analysis system for perioperative human voice change monitoring and correction.

[0018] Figure 4 The present invention is a schematic diagram of the extended structure of a speech element recognition unit of an embodiment of a speech interaction analysis system for perioperative human voice change monitoring and correction. DETAILED DESCRIPTION

[0019] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0020] As the instruction manual Figure 1-4 As shown:

[0021] A speech interaction analysis system for perioperative human voice change monitoring and correction, comprising: a speech element recognition unit, a treatment recovery assessment unit, and a postoperative pronunciation correction unit;

[0022] Voice element recognition unit: A human voice collection system is formed by combining the traditional sound pickup mode with throat-attached sensor devices. High-precision voice recognition of basic voice elements (phonemes, tones, and words) is performed through electroacoustic analysis. The basic voice elements are recorded before surgery, on the day of surgery, 10 days after surgery, 1 month after surgery, 3 months after surgery, and 1 year after surgery. The voice features before and after surgery are compared, and both normal voice and voice with impaired pronunciation are recorded. The electroacoustic manifestations of different impaired voice functions are recorded separately to explore the characteristics of their phonemes, tones, and waveforms, so as to compare normal and abnormal human voices, and diagnose and identify impaired pronunciation functions caused by different reasons.

[0023] Treatment recovery assessment unit: Through the refined analysis of the characteristics of human voice elements, tones, and waveforms, a visual and quantifiable assessment is conducted, the critical value and differentiation of the degree of damage are set, and the degree of damage is quantitatively graded according to the sound wave conditions. It can also assess and monitor the degree of recovery of patients with confirmed impaired pronunciation function during the treatment recovery period;

[0024] Postoperative pronunciation correction unit: Utilizing artificial intelligence machine learning and voice interaction technology, the original voice of the subject before surgery is first recorded, and through voice data analysis, the voice characteristics of the subject in a healthy state are recorded, and machine learning and digital processing are performed to form machine memory. Once the patient's pronunciation function is impaired after surgery, the machine can detect abnormalities in their voice elements on the one hand, and on the other hand, through machine learning memory, automatically record the damaged sound state and make digital adjustments, and then play the adjusted sound close to the patient's healthy state through the connected voice speaker through the player, thereby improving and correcting different pronunciation function impairments such as hoarseness and low voice.

[0025] Artificial Intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technology in computer science that attempts to understand the essence of intelligence and produce a new intelligent machine that can respond in a similar way to human intelligence. Artificial intelligence is to study the design principles and implementation methods of various intelligent machines so that machines have the functions of perception, reasoning and decision-making.

[0026] Machine Learning (ML) is a multi-disciplinary subject that involves probability theory, statistics, approximation theory, convex analysis, algorithm complexity theory and other disciplines. It specializes in studying how computers simulate or implement human learning behavior to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve their performance. Machine learning is the core of artificial intelligence and the fundamental way to make computers intelligent. Its applications are spread across all areas of artificial intelligence. Machine learning and deep learning usually include artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and self-learning.

[0027] In this embodiment, a voice interaction analysis system for monitoring and correcting perioperative human voice changes analyzes the specific sound wave characteristics of different human voice changes, identifies the types of pronunciation function damage, and quickly finds the cause of voice change; and quantitatively grades the degree of damage according to the sound wave situation, and can also evaluate and monitor the degree of recovery during the treatment recovery period, solving the current problem of the unquantifiable degree of perioperative pronunciation change; using artificial intelligence machine learning and voice interaction technology, through sound wave data analysis, using external voice equipment, improve and correct different pronunciation function damages such as hoarseness and low voice. Therefore, this project can meet the actual needs of scientific research, clinical and patients, and has a very broad market prospect.

[0028] Specifically, the speech element recognition unit includes: a phoneme recognition module, a tone recognition module, a word recognition module and a comprehensive recognition module;

[0029] S100: De-noise and normalize the original speech signal, and extract features from the pre-processed speech signal, including: Mel-frequency cepstral coefficients (MFCCs), fundamental frequency, pitch, sound duration, and energy;

[0030] S110: using the patient's normal speech data to train an acoustic phoneme model, a tone classification model, and a word recognition model;

[0031] S120: The acoustic phoneme model maps the features (Mel-frequency cepstral coefficients) extracted from the patient's newly input speech signal to phonemes, and decodes the input speech features into a phoneme sequence A1 through the acoustic model; S122: The tone classification model classifies the features (fundamental frequency, pitch) of the patient's newly input speech signal and identifies its tone A2; S124: The word recognition model inputs the speech features (pitch, tone length, energy) extracted from the patient's newly input speech signal into the acoustic model and identifies the word A3;

[0032] S130: The comprehensive recognition module generates a speech element recognition result A. The speech element recognition result A data consists of a phoneme sequence A1, a tone A2, and a word A3. The speech element recognition result A=α1A1+α2A2+α3A3, wherein α1 is a weighting coefficient of the first speech element, α2 is a weighting coefficient of the second speech element, and α3 is a weighting coefficient of the third speech element. 0<α1,

[0033] α2, α3≤1.

[0034] Specifically, the speech element recognition unit also includes: the tone classification model performs post-processing on the classification result, including removing erroneous tone marks, merging consecutive identical tone marks, etc., and correcting the tone A2; the word recognition model optimizes the recognition word decoding result, including removing repeated words, correcting grammatical errors, etc., and correcting the word A3; the comprehensive recognition module adjusts the speech element recognition result A according to the correction result.

[0035] Specifically, the speech element recognition unit performs identification and diagnosis of perioperative vocal diseases by analyzing the basic speech elements of the patient's voice using Fourier analysis technology, specially selected digital filters and deep learning neural networks; Fourier analysis is used to preprocess the speech spectrum and isolate its frequency characteristics, perform Fourier transform and determine the main features of the patient's newly input speech signal: Mel-frequency cepstral coefficients (MFCCs), fundamental frequency, pitch, duration, energy; also includes: stability of the fundamental tone, type of sound (treble, mid-range, bass, etc.), amplitude range, timbre, vibrato characteristics, etc.; select a digital filter to sample data that characterizes the deviation of the poor speech spectrum from the patient's normal speech spectrum, and cut off auxiliary data that does not affect the diagnosis.

[0036] In this embodiment, the speech spectrum is preprocessed and its frequency characteristics are isolated, data characterizing the deviation between the bad speech spectrum and the patient's normal speech spectrum are selected, and auxiliary data that does not affect the diagnosis is cut off, which greatly saves computing power and speeds up the patient's diagnosis and recovery.

[0037] Specifically, the speech element recognition unit also identifies time-domain features in the patient's speech: low-energy frame ratio, zero-crossing rate, and energy;

[0038] Low Energy Frame Ratio——

[0039] Where N represents the number of audio frames, E(n) represents the short-time energy value, which represents the average energy of the segment, and r is the threshold coefficient, which can be used to weight the average energy to control the energy threshold.

[0040] Zero crossing rate

[0041] Where N is the number of sampling points, x(i) is the amplitude of the signal at the i-th sampling point;

[0042] energy--

[0043] In audio analysis, frames can be used as units to form frame energy sequences.

[0044] In this embodiment, by analyzing the time domain characteristics such as low-energy frame ratio, zero-crossing rate, energy, etc. in the patient's voice, the noise part is filtered out or reduced, the clarity of the speech is improved, the unvoiced and voiced sounds in the patient's voice are effectively identified, the accuracy of factor identification is improved, the speech speed and intonation are analyzed, and the accuracy of emotion recognition is improved.

[0045] Specifically, the speech element recognition unit comprehensive recognition module analyzes the patient's speech element recognition result A, analyzes the changes in the patient's speech and clinical data spectrum characteristics, and shows the patient's pathological condition. For example, in myasthenia gravis, it can be observed that the main voice drops sharply or does not exist; with vocal cord hemorrhage, the frequency and amplitude range of the voice decrease, and the frequency and sound quality of the voice are unstable; low-pitched accent disorder is characterized by the frequency of the main voice moving to low frequencies, the dynamic range narrowing, and the unstable timbre.

[0046] Specifically, the speech element recognition unit also includes a visualization module, which extracts time domain features of the patient's newly input speech signal, extracts the time domain feature value electromyography root mean square, and performs windowing processing to obtain a series of time domain values ​​reflecting the electromyography energy intensity; through software algorithm processing, the electromyography energy intensity is corresponded to color to form a three-dimensional dynamic energy distribution map of the face and neck, and the dynamic visualization time and space information of the pronunciation muscle group is obtained, and the time and space characteristics of the multi-channel electromyography collected by the pronunciation muscle group of the face and neck are visualized; at the same time, the electromyography is analyzed in the frequency domain to extract the electromyography spectrum distribution map and the time-frequency distribution map, and the electromyography frequency domain distribution characteristics during the pronunciation process are visualized.

[0047] In this embodiment, the dynamic change information of the pronunciation muscle groups during the pronunciation stage is obtained, and the dynamic movement of the pronunciation muscle groups working together is obtained in real time and intuitively, which solves the problem that the dynamic information of the activities of the relevant muscle groups during the pronunciation process cannot be obtained in real time and intuitively at this stage.

[0048] Specifically, the treatment recovery evaluation unit includes: a refined analysis module, a quantitative evaluation module, and a recovery degree monitoring module. The refined analysis module extracts features from the processed speech element recognition result A, and extracts the refined analysis data B with the characteristics of the patient's phonemes, tones, and waveforms; the quantitative evaluation module performs template matching and differential analysis on the refined analysis data B1 with the characteristics of the patient's normal pronunciation database, analyzes the patient's pronunciation function abnormalities and the degree of damage, and obtains the critical value B2 and discrimination B3 of the degree of damage; the recovery degree monitoring module monitors the recovery value B, and the critical value B2 of the degree of damage and the discrimination B3 together constitute the recovery value B, and the recovery value B=β1B2+β2B3, wherein β1 is the first recovery value weighting coefficient, β2 is the second recovery value weighting coefficient, 0<β1, β2≤1.

[0049] In this way, the patient's pronunciation function is comprehensively and finely evaluated from multiple angles, and template matching and differentiation analysis are performed with the characteristics of the normal pronunciation function database. The abnormality of the patient's pronunciation function and the degree of damage are analyzed, and an objective quantitative assessment of the patient's injury degree is achieved, ensuring the reliability of the treatment recovery evaluation function.

[0050] Specifically, the postoperative pronunciation correction unit includes: an age and gender module, a word correction module, a tone correction module, an emotion adjustment module, and a volume adjustment module; the age and gender module identifies the patient's age and gender, including: children, teenagers, teenage women, teenage men, adult women, and adult men; after the patient's pronunciation function is impaired after surgery, the patient is assisted in word correction, tone correction, emotion adjustment, and volume adjustment based on the patient's age and gender and the patient's current recovery value B; the word correction module prompts the patient's current word pronunciation accuracy, rhythm difference, and single word pronunciation duration adjustment method based on the word features in the speech element recognition result A; the tone correction module improves the rise, fall, and maintenance of the tone change by tracking the changes in the tone of the patient's input sound signal; the emotion adjustment module prompts the patient to improve emotional expression, including: joy, excitement, tension, sadness, contemplation, etc.; the volume adjustment module prompts the patient to improve the single word volume, including increasing the volume, decreasing the volume, and maintaining the volume.

[0051] Specifically, the postoperative pronunciation correction unit also utilizes the three-dimensional dynamic energy distribution map, electromyography spectrum distribution map, and time-frequency distribution map output by the visualization module of the speech element recognition unit, and simultaneously applies the hidden Markov model and the resonance peak map to compare with each other to derive the user's pronunciation problem type, and provides diagnostic results that accurately suggest directions for improving the speaker's mouth shape or tongue position, thereby providing a more accurate and practical pronunciation correction effect.

[0052] In this embodiment, through artificial intelligence and voice learning of a large clinical sample, the voice reserve, detection and correction functions of the device are continuously improved to make it safer and more effective.

[0053] The method and related apparatus provided by the embodiment of the present application are described with reference to the method flow chart and / or structural diagram provided by the embodiment of the present application. Specifically, each process and / or box in the method flow chart and / or structural diagram, as well as the combination of the processes and / or boxes in the flow chart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable device to generate a machine, so that the instructions executed by the processor of the computer or other programmable device generate instructions for implementing the process in the process. Figure 1 A process or multiple processes and / or structures Figure 1 These computer program instructions can also be stored in a computer-readable memory that can guide a computer or other programmable device to work in a specific way, so that the instructions stored in the computer-readable memory produce a product including an instruction device, which implements the functions specified in the process. Figure 1 A process or multiple processes and / or structures Figure 1 These computer program instructions can also be loaded onto a computer or other programmable device so that a series of operation steps are executed on the computer or other programmable device to produce a computer-implemented process, so that the instructions executed on the computer or other programmable device provide the functions for implementing the process. Figure 1 A flow or multiple flows and / or structures illustrate the steps of the functions specified in one block or multiple blocks.

[0054] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0055] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A voice interaction analysis system for monitoring and correcting human voice changes during perioperative period, characterized in that: include: Speech element recognition unit, treatment recovery assessment unit, postoperative pronunciation correction unit; Voice element recognition unit: A human voice collection system is formed by combining the traditional sound pickup mode with throat-attached sensor devices. High-precision voice recognition of basic voice elements (phonemes, tones, and words) is performed through electroacoustic analysis. The basic voice elements are recorded before surgery, on the day of surgery, 10 days after surgery, 1 month after surgery, 3 months after surgery, and 1 year after surgery. The voice features before and after surgery are compared, and both normal voice and voice with impaired pronunciation are recorded. The electroacoustic manifestations of different impaired voice functions are recorded separately to explore the characteristics of their phonemes, tones, and waveforms, so as to compare normal and abnormal human voices, and diagnose and identify impaired pronunciation functions caused by different reasons. Treatment recovery assessment unit: Through the refined analysis of the characteristics of human voice elements, tones, and waveforms, a visual and quantifiable assessment is conducted, the critical value and differentiation of the degree of damage are set, and the degree of damage is quantitatively graded according to the sound wave conditions. It can also assess and monitor the degree of recovery of patients with confirmed impaired pronunciation function during the treatment recovery period; Postoperative pronunciation correction unit: Utilizing artificial intelligence machine learning and voice interaction technology, the original voice of the subject before surgery is first recorded, and through voice data analysis, the voice characteristics of the subject in a healthy state are recorded, and machine learning and digital processing are performed to form machine memory. Once the patient's pronunciation function is impaired after surgery, the machine can detect abnormalities in their voice elements on the one hand, and on the other hand, through machine learning memory, automatically record the damaged sound state and make digital adjustments, and then play the adjusted sound close to the patient's healthy state through the connected voice speaker through the player, thereby improving and correcting different pronunciation function impairments such as hoarseness and low voice.

2. A voice interaction analysis system for perioperative voice change monitoring and correction as described in claim 1, characterized in that: The speech element recognition unit includes: a phoneme recognition module, a tone recognition module, a word recognition module and a comprehensive recognition module; S100: De-noise and normalize the original speech signal, and extract features from the pre-processed speech signal, including: Mel-frequency cepstral coefficients (MFCCs), fundamental frequency, pitch, sound duration, and energy; S110: using the patient's normal speech data to train an acoustic phoneme model, a tone classification model, and a word recognition model; S120: The acoustic phoneme model maps the features (Mel-frequency cepstral coefficients) extracted from the patient's newly input speech signal to phonemes, and decodes the input speech features into a phoneme sequence A1 through the acoustic model; S122: The tone classification model classifies the features (fundamental frequency, pitch) of the patient's newly input speech signal and identifies its tone A2; S124: The word recognition model inputs the speech features (pitch, tone length, energy) extracted from the patient's newly input speech signal into the acoustic model and identifies the word A3; S130: The comprehensive recognition module generates a speech element recognition result A. The speech element recognition result A data consists of a phoneme sequence A1, a tone A2, and a word A3. The speech element recognition result A=α1A1+α2A2+α3A3, wherein α1 is a weighting coefficient of the first speech element, α2 is a weighting coefficient of the second speech element, and α3 is a weighting coefficient of the third speech element. 0<α1, α2、α3≤1。 3. A speech interaction analysis system for perioperative voice change monitoring and correction as described in claim 2, characterized in that: Also includes: The speech element recognition unit also includes: the tone classification model performs post-processing on the classification result, including removing erroneous tone marks, merging consecutive identical tone marks, etc., and correcting the tone A2; the word recognition model optimizes the decoding result of the recognized word, including removing repeated words, correcting grammatical errors, etc., and correcting the word A3; the comprehensive recognition module adjusts the speech element recognition result A according to the correction result.

4. A speech interaction analysis system for monitoring and correcting perioperative human voice changes as described in any one of claims 1 to 3, characterized in that: Also includes: The speech element recognition unit performs perioperative recognition and diagnosis of human voice diseases by analyzing the basic speech elements of the patient's voice using Fourier analysis technology, specially selected digital filters and deep learning neural networks; Fourier analysis is used to preprocess the speech spectrum and isolate its frequency characteristics, perform Fourier transform and determine the main characteristics of the patient's newly input speech signal: Mel-frequency cepstral coefficients (MFCCs), fundamental frequency, pitch, duration, energy; also includes: fundamental tone stability, type of voice (high, middle, bass, etc.), amplitude range, timbre, vibrato characteristics, etc. A digital filter is selected to sample data representing deviations of the bad speech spectrum from the patient's normal speech spectrum and to cut off auxiliary data that does not affect the diagnosis.

5. A speech interaction analysis system for perioperative human voice change monitoring and correction as claimed in any one of claims 1 to 3, characterized in that: Also includes: The speech element recognition unit also identifies the time domain features in the patient's speech: low-energy frame ratio, zero-crossing rate, and energy; by analyzing the time domain features such as low-energy frame ratio, zero-crossing rate, and energy in the patient's speech, it filters out or reduces the noise part, improves the clarity of the speech, effectively identifies the unvoiced and voiced sounds in the patient's speech, improves the accuracy of factor recognition, analyzes the speaking speed and intonation, and improves the accuracy of emotion recognition.

6. A voice interaction analysis system for perioperative voice change monitoring and correction as claimed in any one of claims 1 to 3, characterized in that: Also includes: The comprehensive recognition module of the speech element recognition unit analyzes the patient's speech element recognition result A, analyzes the changes in the spectrum characteristics of the patient's speech and clinical data, and displays the patient's pathological condition.

7. A speech interaction analysis system for perioperative human voice change monitoring and correction as claimed in any one of claims 1 to 3, characterized in that: Also includes: The speech element recognition unit also includes a visualization module, which extracts time domain features of the patient's newly input speech signal, extracts the time domain feature value electromyography root mean square, and performs windowing processing to obtain a series of time domain values ​​reflecting the electromyography energy intensity; through software algorithm processing, the electromyography energy intensity is corresponded to color to form a three-dimensional dynamic energy distribution map of the face and neck, and the dynamic visualization time and space information of the pronunciation muscle group is obtained, and the time and space characteristics of the multi-channel electromyography collected by the pronunciation muscle group of the face and neck are visualized; at the same time, the electromyography is analyzed in the frequency domain to extract the electromyography spectrum distribution map and the time-frequency distribution map, and the electromyography frequency domain distribution characteristics during the pronunciation process are visualized.

8. A speech interaction analysis system for monitoring and correcting perioperative human voice changes as claimed in any one of claims 1 to 3, characterized in that: Also includes: The treatment recovery assessment unit includes: a refined analysis module, a quantitative assessment module, and a recovery degree monitoring module. The refined analysis module extracts features from the processed speech element recognition result A, and extracts the refined analysis data B of the patient's phonemes, tones, and waveform characteristics; the quantitative assessment module performs template matching and differential analysis on the refined analysis data B1 and the patient's normal pronunciation database features, analyzes the abnormality of the patient's pronunciation function and the degree of damage, and obtains the critical value B2 and the discrimination degree B3 of the degree of damage; The recovery degree monitoring module monitors the recovery value B. The damage degree critical value B2 and the discrimination degree B3 together constitute the recovery value B, and the recovery value B=β1B2+β2B3.

9. A speech interaction analysis system for perioperative human voice change monitoring and correction as claimed in any one of claims 1 to 3, characterized in that: Also includes: The postoperative pronunciation correction unit includes: age and gender module, word correction module, tone correction module, emotion adjustment module, and volume adjustment module; the age and gender module identifies the patient's age and gender, including: children, teenagers, teenage women, teenage men, adult women, and adult men; after the patient's pronunciation function is impaired after surgery, the patient is assisted in word correction, tone correction, emotion adjustment, and volume adjustment according to the patient's age and gender and the patient's current recovery value B; the word correction module prompts the patient's current word pronunciation accuracy, rhythm difference, and single word pronunciation duration adjustment method based on the word features in the speech element recognition result A; the tone correction module improves the rise, fall, and maintenance of the tone change by tracking the changes in the tone of the patient's input sound signal; the emotion adjustment module prompts the patient to improve emotional expression, including: joy, excitement, tension, sadness, contemplation, etc.; the volume adjustment module prompts the patient to improve the single word volume, including increasing the volume, decreasing the volume, and maintaining the volume.

10. A speech interaction analysis system for perioperative human voice change monitoring and correction as claimed in claim 9, characterized in that: Also includes: The postoperative pronunciation correction unit also uses the three-dimensional dynamic energy distribution map, electromyography spectrum distribution map, and time-frequency distribution map output by the visualization module of the speech element recognition unit, and applies the hidden Markov model and resonance peak map to compare with each other to derive the user's pronunciation problem type, and provide accurate diagnostic results that suggest directions for improving the speaker's mouth shape or tongue position.

Images