Postoperative oral function rehabilitation training analysis system and method for oral maxillofacial surgery

Abstract

The invention discloses an oral function rehabilitation training analysis system and method after oral maxillofacial surgery, and the system carries out the real-time collection and intelligent evaluation of the rehabilitation training process data of a patient through a closed occlusion training evaluation module, an opening degree training evaluation module, a pronunciation training evaluation module, and a swallowing training evaluation module. The training effect of the patient is fed back in real time through the evaluation result, quantification of the rehabilitation effect is achieved, a scientific basis is provided for a rehabilitation plan made by medical staff, the occlusal force, lip closing force and pronunciation rehabilitation rate of the patient is effectively improved, and the life quality of the patient is remarkably improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of medical equipment application, and in particular relates to an oral function rehabilitation training analysis system and method after oral and maxillofacial surgery. Background Art

[0002] Patients who have undergone oral and maxillofacial surgery often face problems such as difficulty opening their mouths, difficulty swallowing, drooling, and slurred speech, which significantly reduce the quality of life of patients. The medical training therapy (MTT) in the prior art formulates a systematic physical exercise training plan, and adjusts the training parameters such as training load and interval time in a timely manner during the exercise training according to the pre-planned plan. Medical training therapy requires that both the evaluation and training links can be recorded and quantified. Most of the existing methods for biomechanical analysis of the oral and maxillofacial region use cadaver specimens as test subjects, or implant sensors between vertebrae to collect data. Such methods have strict implementation conditions, are highly invasive, and have huge risks. They cannot be applied to the oral and maxillofacial rehabilitation training process. In addition, due to the limitation of medical resources, it is difficult for medical staff to provide detailed guidance and supervision for the rehabilitation training of each patient. The patient has a long recovery period after surgery, and still needs long-term functional training after discharge. Summary of the invention

[0003] In view of the above-mentioned deficiencies in the prior art, the present application provides a system and method for analyzing oral function rehabilitation training after oral and maxillofacial surgery.

[0004] In the first aspect, the present application proposes an oral function rehabilitation training analysis system after oral and maxillofacial surgery, including a closed mouth occlusion training assessment module, a mouth opening training assessment module, a pronunciation training assessment module and a swallowing training assessment module;

[0005] The closed-mouth bite training evaluation module is used to collect and analyze the data of bite force training and lip closure force training, and to correct incorrect training actions through the analysis results to improve the recovery rate of bite force and lip closure force training;

[0006] The mouth opening degree training evaluation module is used to construct a facial key point detection network model, perform image recognition and facial motion capture on the patient's face through the facial key point detection network model, obtain mouth opening degree data, analyze and compare the mouth opening degree data collected multiple times, and evaluate the mouth opening training effect according to the comparison results;

[0007] The pronunciation training evaluation module is used to collect audio data when the patient reads the preset voice, compare the audio data with the preset voice, and judge whether the patient's pronunciation is accurate based on the comparison result, so as to evaluate the pronunciation training effect;

[0008] The swallowing training evaluation module is used to obtain the patient's electromyographic signal change data, jaw swing force data and jaw swing amplitude data, determine whether the patient has completed the swallowing action based on the electromyographic signal change data, and generate a rehabilitation training cycle for the corresponding patient based on the patient's swallowing situation in combination with the jaw swing force data and the jaw swing amplitude data, and guide the patient to perform rehabilitation training.

[0009] In some optional implementations of some embodiments, the closed-mouth occlusion training evaluation module includes an oral data acquisition unit, an oral signal preprocessing unit, an oral data standardization unit, a closed-mouth occlusion data feature extraction unit, and a closed-mouth occlusion training effect evaluation unit;

[0010] The oral data acquisition unit is used to arrange a high-sensitivity pressure sensor in the patient's oral cavity to monitor the force changes during biting and lip closure in real time;

[0011] The oral signal preprocessing unit is used to collect raw signals containing fluctuations caused by patient movements and environmental noise, and use wavelet transform to denoise the signals to extract real bite and closing force signals;

[0012] The oral data standardization unit is used to perform standardization processing on the extracted signal using a Z-score standardization method;

[0013] The closed-mouth occlusal data feature extraction unit is used to extract key features from the preprocessed signal, including: maximum closing force, average closing force and force stability;

[0014] The closed-mouth occlusion training effect evaluation unit is used to evaluate the closed-mouth occlusion training effect using a statistical method.

[0015] In some optional implementations of some embodiments, the mouth opening degree training evaluation module includes a facial image acquisition unit, a face detection unit, a facial feature extraction unit, a facial motion capture unit, and a mouth opening degree training feedback unit;

[0016] The facial image acquisition unit is used to acquire the patient's facial image in real time;

[0017] The face detection unit is used to quickly and accurately locate the face in the collected image using a face detection algorithm based on deep learning, providing accurate input for feature point positioning;

[0018] The facial feature extraction unit is used to locate key feature points on the patient's face through a key point detection network model, and extract geometric and texture features from the key feature points;

[0019] The facial motion capture unit is used to capture the patient's facial movements by comparing the coordinate position changes of key feature points in consecutive frames;

[0020] The mouth opening training feedback unit is used to calculate the mouth opening according to the change of coordinate position:

[0021]

[0022] in, and They represent the coordinates of the feature points of the left and right mouth corners respectively. The value of the mouth opening directly reflects the size of the patient's mouth opening;

[0023] Based on the calculated mouth opening value, the system generates training feedback. If the mouth opening degree does not reach the preset threshold, the system will prompt the patient to increase the mouth opening amplitude. If it exceeds the threshold, the system will prompt the patient to appropriately reduce the amplitude. The training effect is evaluated by comparing the changes in mouth opening degree before and after training.

[0024] In some optional implementations of some embodiments, it is characterized in that: the pronunciation training evaluation module includes a speech signal acquisition unit, a speech data preprocessing unit, a speech feature extraction unit, a pronunciation detection unit and a pronunciation feedback training unit;

[0025] The voice signal acquisition unit is used to capture the patient's voice signal through a high-sensitivity microphone array to ensure that the voice signal has a sufficient signal-to-noise ratio;

[0026] The speech data preprocessing unit is used to perform noise reduction, silence detection and normalization processing on the speech signal to remove background noise and other non-pronunciation related sound waves contained in the speech signal;

[0027] The speech feature extraction unit is used to extract key speech features from the preprocessed speech signal, including spectrum features and long-term average amplitude;

[0028] The pronunciation detection unit is used to train a deep neural network model through key speech features, use the trained speech model to perform real-time detection of the patient's pronunciation, identify phonemes and syllables in the pronunciation, and identify incorrect pronunciation by comparing the model prediction results with the actual pronunciation;

[0029] The pronunciation feedback training unit is used to generate pronunciation feedback information according to the recognition result of the incorrect pronunciation, and provide pronunciation training suggestions according to the feedback information.

[0030] In some optional implementations of some embodiments, the swallowing training assessment module includes an electromyographic signal acquisition unit, a jaw swing data acquisition unit, a swallowing action detection unit, and a swallowing training feedback unit;

[0031] The electromyographic signal acquisition unit is used to obtain the electromyographic signal change data of the patient's thyrohyoid muscle and masticatory muscle through the electromyographic sensor;

[0032] The jaw swing data acquisition unit is used to obtain the patient's jaw swing force data through a pressure sensor, and to detect the patient's jaw swing amplitude data through a motion sensor;

[0033] The swallowing action detection unit is used to determine whether the patient has completed the swallowing action according to the change of the electromyographic signal;

[0034] The swallowing training feedback unit is used to detect whether the swing force and swing amplitude reach a preset safety threshold. If so, a training cycle corresponding to the patient is generated based on the swing force and swing amplitude of the patient's lower jaw, and the patient is guided to perform rehabilitation training. If not, the electromyographic signal change data is re-detected to see if it meets the changes in the swallowing action.

[0035] In some optional implementations of some embodiments, the face detection algorithm used in the face detection unit is an SSD algorithm or a YOLO algorithm.

[0036] In some optional implementations of some embodiments, the key point detection network model used in the facial feature extraction unit is an MHRB-net convolutional neural network model.

[0037] In some optional implementations of some embodiments, the network architecture of the deep neural network model used in the pronunciation detection unit is a recurrent neural network or a long short-term memory network.

[0038] In a second aspect, the present application proposes a method for analyzing oral function rehabilitation training after oral and maxillofacial surgery, comprising the following steps:

[0039] Collect and analyze the data of bite force training and lip closure force training, and correct incorrect training movements through analysis results to improve the recovery rate of bite force and lip closure force training;

[0040] Construct a facial key point detection network model, use the facial key point detection network model to perform image recognition and facial motion capture on the patient's face, obtain mouth opening data, analyze and compare the mouth opening data collected multiple times, and evaluate the mouth opening training effect based on the comparison results;

[0041] Collect audio data when the patient reads the preset voice, compare the audio data with the preset voice, and judge whether the patient's pronunciation is accurate based on the comparison results, so as to evaluate the pronunciation training effect;

[0042] Obtain the patient's electromyographic signal change data, jaw swing force data and jaw swing amplitude data, determine whether the patient has completed the swallowing action based on the electromyographic signal change data, and generate a rehabilitation training cycle for the corresponding patient based on the patient's swallowing situation combined with the jaw swing force data and the jaw swing amplitude data, and guide the patient to perform rehabilitation training.

[0043] In a third aspect, the present application proposes a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the above method are implemented.

[0044] Beneficial effects of the present invention:

[0045] Through the closed mouth occlusion training assessment module, mouth opening training assessment module, pronunciation training assessment module and swallowing training assessment module, the patient's rehabilitation training process data is instantly collected and intelligently evaluated. Through the evaluation results, the patient's training effect is instantly fed back to quantify the rehabilitation effect, providing a scientific basis for the rehabilitation plan formulated by medical staff, thereby effectively improving the patient's bite force, lip closure force and pronunciation recovery rate, and significantly improving the patient's quality of life. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 This is the principle block diagram of this system.

[0047] Figure 2 It is the overall flow chart of the present invention. DETAILED DESCRIPTION

[0048] The exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments described herein; on the contrary, these embodiments are provided to enable a more thorough understanding of the present disclosure and to fully convey the scope of the present disclosure to those skilled in the art.

[0049] In the first aspect, the present application proposes a postoperative oral function rehabilitation training analysis system for oral and maxillofacial surgery, such as Figure 1 As shown, it includes a closed mouth occlusion training assessment module, a mouth opening training assessment module, a pronunciation training assessment module and a swallowing training assessment module;

[0050] The closed-mouth bite training evaluation module is used to collect and analyze the data of bite force training and lip closure force training, and to correct incorrect training actions through the analysis results to improve the recovery rate of bite force and lip closure force training;

[0051] In some optional implementations of some embodiments, the closed-mouth occlusion training evaluation module includes an oral data acquisition unit, an oral signal preprocessing unit, an oral data standardization unit, a closed-mouth occlusion data feature extraction unit, and a closed-mouth occlusion training effect evaluation unit;

[0052] The oral data acquisition unit is used to arrange a high-sensitivity pressure sensor in the patient's oral cavity to monitor the force changes during biting and lip closure in real time;

[0053] Among them, a highly sensitive pressure sensor is placed in the patient's mouth to monitor the force changes during biting and lip closure in real time. The sensor is connected to the data acquisition system to ensure accurate data recording. In order to avoid interference, the sensor is designed to be small and fits the oral mucosa to reduce patient discomfort.

[0054] The oral signal preprocessing unit is used to collect raw signals containing fluctuations caused by patient movements and environmental noise, and use wavelet transform to denoise the signals to extract real bite and closing force signals;

[0055] The collected raw signal contains fluctuations caused by patient movements and environmental noise. Wavelet transform is used to denoise the signal to extract the true bite and closing force signals. The denoising process is achieved by the following steps:

[0056] 1. Perform multi-scale wavelet decomposition on the original signal.

[0057] 2. Use threshold processing to remove noise components.

[0058] 3. Perform inverse wavelet transform on the processed signal to reconstruct a clean bite force signal.

[0059] The oral data standardization unit is used to perform standardization processing on the extracted signal using a Z-score standardization method;

[0060] In order to facilitate comparison and analysis, the extracted signal needs to be standardized. The Z-score standardization method is used, and the calculation formula is as follows:

[0061]

[0062] in, is the original data, is the mean of all data points, is the standard deviation, and the standardized data has zero mean and unit variance.

[0063] The closed-mouth occlusal data feature extraction unit is used to extract key features from the preprocessed signal, including: maximum closing force, average closing force and force stability;

[0064] Extract key features from the preprocessed signal, including:

[0065] Maximum closing force ( ): Record the maximum bite force or lip closure force during the training period.

[0066] Average closing force ( ): Calculate the average bite force or lip closure force over the entire training cycle.

[0067] Strength stability ( ): The consistency of bite or closing force is evaluated by calculating the standard deviation of the force signal.

[0068] The closed-mouth occlusion training effect evaluation unit is used to evaluate the closed-mouth occlusion training effect using a statistical method.

[0069] Use statistical methods to evaluate training effectiveness, such as:

[0070] Paired sample t-test was used to compare whether the strength changes before and after training were significant.

[0071] Calculate the effect index , the formula is as follows:

[0072]

[0073] in, For strength after training, For pre-training strength.

[0074] Take patient Zhang San as an example. His preoperative bite force was 100N, and the initial bite force after surgery dropped to 60N. Rehabilitation training was carried out through the above scheme. After data collection, signal preprocessing and standardization were performed, and features were extracted and evaluated. Assuming that the bite force after training increased to 80N, according to the effect index calculation, the training effect of Zhang San is:

[0075]

[0076] The results show that Zhang San's bite force training is effective. Through the closed mouth bite training evaluation module, patients and medical staff can intuitively understand the training effect and adjust the training plan according to the evaluation results.

[0077] The mouth opening degree training evaluation module is used to construct a facial key point detection network model, perform image recognition and facial motion capture on the patient's face through the facial key point detection network model, obtain mouth opening degree data, analyze and compare the mouth opening degree data collected multiple times, and evaluate the mouth opening training effect according to the comparison results;

[0078] In some optional implementations of some embodiments, the mouth opening degree training evaluation module includes a facial image acquisition unit, a face detection unit, a facial feature extraction unit, a facial motion capture unit, and a mouth opening degree training feedback unit;

[0079] The facial image acquisition unit is used to acquire the patient's facial image in real time;

[0080] A high-resolution camera is used to capture the patient's facial image in real time, ensuring that the image quality meets the accuracy requirements of subsequent processing. The facial image acquisition unit includes autofocus and light compensation functions to adapt to different environmental conditions.

[0081] The face detection unit is used to quickly and accurately locate the face in the collected image using a face detection algorithm based on deep learning, providing accurate input for feature point positioning;

[0082] Use deep learning-based face detection algorithms, such as SSD (Single Shot MultiBox Detector) or YOLO (You Only Look Once), to quickly and accurately locate faces in the captured images. This step aims to separate the facial area from the background and provide accurate input for subsequent feature point positioning.

[0083] The facial feature extraction unit is used to locate key feature points on the patient's face through a key point detection network model, and extract geometric and texture features from the key feature points;

[0084] The MHRB-net key point detection network model is used. This model can accurately locate the key feature points of the face, such as the corners of the eyes, the tip of the nose, and the corners of the mouth, through the convolutional neural network (CNN) structure. The positioning accuracy of these feature points directly affects the accuracy of motion capture and mouth opening calculation. Geometric and texture features are extracted from the located feature points, which are used to describe subtle changes in facial movements. Feature extraction can include calculating the distance and angle between feature points, and extracting texture information using methods such as local binary patterns (LBP).

[0085] The facial motion capture unit is used to capture the patient's facial movements by comparing the coordinate position changes of key feature points in consecutive frames;

[0086] By comparing the position changes of feature points in consecutive frames, the patient's facial movements are captured. Specifically for mouth opening movements, the lateral displacement of feature points on both sides of the mouth corners is mainly monitored.

[0087] The mouth opening training feedback unit is used to calculate the mouth opening according to the change of coordinate position:

[0088]

[0089] in, and They represent the coordinates of the feature points of the left and right mouth corners respectively. The value of the mouth opening directly reflects the size of the patient's mouth opening;

[0090] Based on the calculated mouth opening value, the system generates training feedback. If the mouth opening degree does not reach the preset threshold, the system will prompt the patient to increase the mouth opening amplitude. If it exceeds the threshold, the system will prompt the patient to appropriately reduce the amplitude. By comparing the changes in mouth opening degree before and after training, the training effect is evaluated. Evaluation indicators may include the average percentage increase in mouth opening degree, etc.

[0091] Take patient Li Si as an example. His mouth opening degree after the operation was only 2 cm. Through the above scheme for rehabilitation training, the system collects facial images in real time and accurately detects facial feature points. After continuous motion capture and mouth opening calculation, the system found that Li Si's mouth opening degree gradually increased to 3 cm. Training feedback prompted Li Si to continue to maintain the current training intensity. After a period of training, the final mouth opening degree reached the normal 4 cm. The training effect evaluation showed that the mouth opening degree increased by 100%. Through the analysis of the mouth opening training evaluation module, patients can clearly see the progress of the training, thereby enhancing their confidence in rehabilitation.

[0092] The pronunciation training evaluation module is used to collect audio data when the patient reads the preset voice, compare the audio data with the preset voice, and judge whether the patient's pronunciation is accurate based on the comparison result, so as to evaluate the pronunciation training effect;

[0093] In some optional implementations of some embodiments, it is characterized in that: the pronunciation training evaluation module includes a speech signal acquisition unit, a speech data preprocessing unit, a speech feature extraction unit, a pronunciation detection unit and a pronunciation feedback training unit;

[0094] The voice signal acquisition unit is used to capture the patient's voice signal through a high-sensitivity microphone array to ensure that the voice signal has a sufficient signal-to-noise ratio;

[0095] In order to evaluate the effect of pronunciation training on patients, it is necessary to collect the patient's voice signal first. A high-sensitivity microphone array is used to capture the patient's pronunciation to ensure that the collected signal has a sufficient signal-to-noise ratio. The design of the microphone array takes into account the acoustic characteristics inside and outside the mouth to adapt to different pronunciation environments and patient conditions.

[0096] The speech data preprocessing unit is used to perform noise reduction, silence detection and normalization processing on the speech signal to remove background noise and other non-pronunciation related sound waves contained in the speech signal;

[0097] The collected original speech signal contains background noise and other non-pronunciation related sound waves. The preprocessing steps include:

[0098] 1. Noise reduction: Use spectral subtraction or deep learning methods to remove background noise.

[0099] 2. Silence detection: Identify and remove silent parts to reduce the amount of data for subsequent processing.

[0100] 3. Normalization: Perform energy normalization on the speech signal to eliminate the influence of different patients’ pronunciation intensities.

[0101] The speech feature extraction unit is used to extract key speech features from the preprocessed speech signal, including spectrum features and long-term average amplitude;

[0102] Extract key features from the preprocessed speech signal for training and evaluation. The following feature extraction methods are used:

[0103] Mel-frequency cepstral coefficient (MFCC): captures the spectral characteristics of speech signals and reflects the pronunciation characteristics of different phonemes.

[0104] Long-term average amplitude (LTA): used to assess the duration and intensity of pronunciation.

[0105] The pronunciation detection unit is used to train the deep neural network model through key speech features, use the trained speech model to detect the patient's pronunciation in real time, identify the phonemes and syllables in the pronunciation, and identify incorrect pronunciation by comparing the model prediction results with the actual pronunciation;

[0106] Use the extracted features to train a deep neural network model, such as a recurrent neural network (RNN) or a long short-term memory network (LSTM), to achieve speech recognition and pronunciation assessment. The model training process includes:

[0107] 1. Data annotation: Collect speech samples with normal pronunciation and annotate the corresponding phonemes or syllables.

[0108] 2. Model training: Use labeled data to train the neural network to learn the mapping relationship between pronunciation features and phonemes.

[0109] The patient's pronunciation is detected in real time using the trained speech model to identify the phonemes and syllables in the pronunciation. The detection process is achieved through the following steps:

[0110] 1. Input the patient’s speech signal into the trained model.

[0111] 2. The model outputs the predicted phoneme sequence.

[0112] 3. Compare the predicted sequence with the standard sequence to detect pronunciation differences.

[0113] Identify mispronunciations by comparing model predictions with actual pronunciations. Use the Dynamic Time Warping (DTW) algorithm to assess pronunciation accuracy and locate mispronunciations of phonemes or syllables.

[0114] The pronunciation feedback training unit is used to generate pronunciation feedback information based on the recognition results of incorrect pronunciations and provide pronunciation training suggestions based on the feedback information.

[0115] Generate intuitive feedback information based on the recognition results of incorrect pronunciations. The feedback includes:

[0116] Phoneme-level error indication: Indicates mispronounced phonemes and provides examples of correct pronunciation.

[0117] Syllable-level correction suggestions: Instruct the patient on how to pronounce the incorrect syllable correctly.

[0118] The patient conducts pronunciation training based on the feedback information. The training process includes:

[0119] 1. Repeat the incorrect pronunciation until the pronunciation is correct.

[0120] 2. The system provides real-time feedback to guide patients to adjust their pronunciation.

[0121] 3. Record pronunciation data during training to evaluate training results.

[0122] Take patient Wang Wu as an example. He had difficulty in pronunciation after surgery, especially with certain specific phonemes. Through the above scheme, Wang Wu's speech signal is first collected and preprocessed. After extracting MFCC and LTA features, these features are used to train the speech model. In the pronunciation detection stage, the model recognizes that Wang Wu has difficulty pronouncing phonemes such as "zh", "ch", and "sh". The system generates specific feedback, points out the wrong phonemes, and provides examples of correct pronunciation. Wang Wu conducts pronunciation training based on the feedback, and the system monitors the pronunciation accuracy in real time and provides instant feedback. After a period of training, Wang Wu's pronunciation accuracy has significantly improved, incorrect pronunciations have decreased, and the pronunciation training effect has been significantly improved. In this way, patients can carry out targeted pronunciation rehabilitation training to improve their quality of life after surgery.

[0123] The swallowing training evaluation module is used to obtain the patient's electromyographic signal change data, jaw swing force data and jaw swing amplitude data, determine whether the patient has completed the swallowing action based on the electromyographic signal change data, and generate a rehabilitation training cycle for the corresponding patient based on the patient's swallowing situation in combination with the jaw swing force data and the jaw swing amplitude data, and guide the patient to perform rehabilitation training.

[0124] In some optional implementations of some embodiments, the swallowing training assessment module includes an electromyographic signal acquisition unit, a jaw swing data acquisition unit, a swallowing action detection unit, and a swallowing training feedback unit;

[0125] The electromyographic signal acquisition unit is used to obtain the electromyographic signal change data of the patient's thyrohyoid muscle and masticatory muscle through the electromyographic sensor;

[0126] Among them, multiple electromyographic sensors are respectively arranged on the patient's neck and side of the jaw, and the electromyographic sensors are used to detect the electromyographic signal change data of the patient's thyrohyoid muscle and chewing muscle in real time, thereby improving the accuracy of the electromyographic sensor in detecting the electromyographic signals of the patient's chewing muscle and thyrohyoid muscle. The electromyographic sensors can be connected to an electromyograph and output the captured electromyographic signals as an electromyogram, thereby intuitively reflecting the changes in the electromyographic signals of the chewing muscle and thyrohyoid muscle of the patient when swallowing.

[0127] The jaw swing data acquisition unit is used to obtain the patient's jaw swing force data through a pressure sensor, and to detect the patient's jaw swing amplitude data through a motion sensor;

[0128] Furthermore, the pressure sensor can be pasted close to the bottom of the patient's jaw to improve the accuracy of detecting the downward swing force of the patient's jaw. The motion sensor can be attached to the patient's chin to facilitate the detection of the swing amplitude of the jaw. Furthermore, the electromyography sensor, pressure sensor and motion sensor are all attached to the corresponding detection positions of the patient through conductive adhesive. The conductive adhesive has good conductive properties, so that the patient's surface electrical signals can be accurately captured.

[0129] The swallowing action detection unit is used to determine whether the patient has completed the swallowing action according to the change of the electromyographic signal;

[0130] By detecting the vibration amplitude of the thyrohyoid muscle and the masticatory muscle, the vibration amplitude of the muscle group is recorded

[0131] The moment of change is the moment when the vibration amplitude of the muscle group changes from an unstable state with a large change amplitude to a stable state with a small change amplitude. When the patient is swallowing, the thyrohyoid muscle and the chewing muscle are in the patient's swallowing process. The patient's tongue tip will press against the top of the mouth, so that the patient's chewing muscle and thyrohyoid muscle and other muscle groups participate in the swallowing action. At this time, the chewing muscle and the thyrohyoid muscle are in a tense state. The chewing muscle pulls the patient's mouth to close and assists the tongue tip to press against the top of the mouth; the thyrohyoid muscle pulls the thyroid cartilage up and close to the hyoid bone, thereby meeting the swallowing action requirements. Since the thyrohyoid muscle and the chewing muscle are in a tense state, when the patient is swallowing, the electromyographic sensor identifies that the vibration amplitude of the muscle group is in a high-frequency and unstable state. When the patient completes the swallowing action, the thyrohyoid muscle and the chewing muscle relax, and the electromyographic sensor can obtain that the sudden change in the vibration amplitude of the muscle group is gentle and stable. The changes in the muscle group can be used as a basis for judging whether the patient has completed the swallowing action.

[0132] The swallowing training feedback unit is used to detect whether the swing force and swing amplitude reach a preset safety threshold. If so, a training cycle corresponding to the patient is generated based on the swing force and swing amplitude of the patient's lower jaw, and the patient is guided to perform rehabilitation training. If not, the electromyographic signal change data is re-detected to see if it meets the changes in the swallowing action.

[0133] Furthermore, when the patient's jaw swings downward, the swinging force of the patient's jaw is detected by the pressure sensor on the auxiliary device; when the patient's jaw swings downward, the swinging amplitude of the patient's jaw is obtained by the motion sensor attached to the patient's jaw position. When the patient swings his jaw downward to assist in swallowing, the motion sensor on the patient's jaw can record the swinging amplitude of the patient's jaw in real time, and feed the swinging amplitude data back to the swallowing training evaluation module. The swallowing training evaluation module obtains the patient's jaw movement parameters when swallowing, so as to detect the patient's swallowing situation in real time, record the moment of mutation of the electromyographic signal of the patient's thyrohyoid muscle, and record the moment of mutation of the electromyographic signal of the patient's chewing muscle. When the time difference of the mutation of the two sets of electromyographic signals is less than the preset time threshold, it is judged that the patient has completed the swallowing movement. Specifically, before the patient undergoes swallowing rehabilitation training, the swallowing training evaluation module obtains the patient's basic physical data. The physical data includes the patient's age and the health status of the patient's body parts, including the health status of the head, neck and throat, so that the swallowing training evaluation module can accurately evaluate the patient's swallowing situation and conduct swallowing rehabilitation training. Furthermore, the preset time threshold ranges from 0.01s to 0.1s. The time threshold can be adjusted according to the patient's physical condition. When the patient completes the swallowing action, the thyrohyoid muscle and the masticatory muscle switch from a tense state to a relaxed state, so that the time intervals of the sudden changes in the electromyographic signals of the thyrohyoid muscle and the masticatory muscle are relatively small. By setting a preset time threshold to detect whether the patient has completed the swallowing action, the accuracy of the swallowing training evaluation module in detecting the patient's swallowing action is improved. According to the patient's physical condition, a safety threshold for the patient's jaw swing force is set, and a safety threshold for the patient's jaw downward swing is set. The safety threshold is set according to the maximum amplitude and maximum force. The safety threshold is set to 70% of the maximum amplitude and maximum force, so as to ensure that the patient can ensure the safety of the jaw movement when performing the jaw swing;

[0134] The target data for patient training is generated by analyzing the patient's swallowing time, jaw swing force and swing amplitude. The target data includes target swallowing time, target swing force and target swing amplitude. The training goal is set according to the target data. The patient's swallowing action and swallowing time are detected in real time during patient training to determine whether the patient has achieved the target data. By setting a fixed-time training cycle and guiding the patient to repeat the swallowing action training through training games, the effectiveness of swallowing rehabilitation training is improved.

[0135] In the second aspect, the present application proposes a method for analyzing oral function rehabilitation training after oral and maxillofacial surgery. Figure 2 As shown, the following steps are included:

[0136] S100: Collect and analyze the data of bite force training and lip closure force training, and correct the wrong training movements through the analysis results to improve the recovery rate of bite force and lip closure force training;

[0137] S200: constructing a facial key point detection network model, performing image recognition and facial motion capture on the patient's face through the facial key point detection network model, obtaining mouth opening degree data, analyzing and comparing the mouth opening degree data collected multiple times, and evaluating the mouth opening training effect according to the comparison results;

[0138] S300: collecting audio data when the patient reads the preset voice, comparing the audio data with the preset voice, judging whether the patient's pronunciation is accurate according to the comparison result, so as to evaluate the pronunciation training effect;

[0139] S400: Obtain the patient's electromyographic signal change data, jaw swing force data and jaw swing amplitude data, determine whether the patient has completed the swallowing action based on the electromyographic signal change data, and generate a rehabilitation training cycle for the patient based on the patient's swallowing situation combined with the jaw swing force data and the jaw swing amplitude data, and guide the patient to perform rehabilitation training.

[0140] In a third aspect, the present application proposes a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the above method are implemented.

[0141] The technicians in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In practical applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated in a processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.

[0142] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0143] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this disclosure.

[0144] In the embodiments provided in the present disclosure, it should be understood that the disclosed apparatus / computer equipment and methods can be implemented in other ways. For example, the apparatus / computer equipment embodiments described above are merely schematic. For example, the division of modules or units is only a logical function division. There may be other division methods in actual implementation. Multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces, indirect coupling or communication connection of the apparatus or unit, which may be electrical, mechanical or other forms.

[0145] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0146] In addition, each functional unit in each embodiment of the present disclosure may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0147] If the integrated module / unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present disclosure implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. The computer program may include computer program code, and the computer program code may be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may include: any entity or device capable of carrying computer program code, recording medium, U disk, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium. It should be noted that the content contained in the computer-readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, the computer-readable medium does not include electric carrier signals and telecommunication signals.

[0148] The above are only preferred implementations of the present invention. It should be pointed out that a number of modifications and improved technical solutions made by those skilled in the art without departing from the technical solution should also be deemed to fall within the scope of protection required by the claims.

Claims

1. A postoperative oral function rehabilitation training and analysis system for oral and maxillofacial surgery, characterized by: It includes closed mouth occlusion training assessment module, mouth opening training assessment module, pronunciation training assessment module and swallowing training assessment module; The closed-mouth bite training evaluation module is used to collect and analyze the data of bite force training and lip closure force training, and to correct incorrect training actions through the analysis results to improve the recovery rate of bite force and lip closure force training; The mouth opening degree training evaluation module is used to construct a facial key point detection network model, perform image recognition and facial motion capture on the patient's face through the facial key point detection network model, obtain mouth opening degree data, analyze and compare the mouth opening degree data collected multiple times, and evaluate the mouth opening training effect according to the comparison results; The pronunciation training evaluation module is used to collect audio data when the patient reads the preset voice, compare the audio data with the preset voice, and judge whether the patient's pronunciation is accurate based on the comparison result, so as to evaluate the pronunciation training effect; The swallowing training evaluation module is used to obtain the patient's electromyographic signal change data, jaw swing force data and jaw swing amplitude data, determine whether the patient has completed the swallowing action based on the electromyographic signal change data, and generate a rehabilitation training cycle for the corresponding patient based on the patient's swallowing situation in combination with the jaw swing force data and the jaw swing amplitude data, and guide the patient to perform rehabilitation training.

2. The system according to claim 1, characterized in that: The closed-mouth occlusion training evaluation module includes an oral data acquisition unit, an oral signal preprocessing unit, an oral data standardization unit, a closed-mouth occlusion data feature extraction unit and a closed-mouth occlusion training effect evaluation unit; The oral data acquisition unit is used to arrange a high-sensitivity pressure sensor in the patient's oral cavity to monitor the force changes during biting and lip closure in real time; The oral signal preprocessing unit is used to collect raw signals containing fluctuations caused by patient movements and environmental noise, and use wavelet transform to denoise the signals to extract real bite and closing force signals; The oral data standardization unit is used to perform standardization processing on the extracted signal using a Z-score standardization method; The closed-mouth occlusal data feature extraction unit is used to extract key features from the preprocessed signal, including: maximum closing force, average closing force and force stability; The closed-mouth occlusion training effect evaluation unit is used to evaluate the closed-mouth occlusion training effect using a statistical method.

3. The system according to claim 2, characterized in that: The mouth opening training evaluation module includes a facial image acquisition unit, a face detection unit, a facial feature extraction unit, a facial motion capture unit and a mouth opening training feedback unit; The facial image acquisition unit is used to acquire the patient's facial image in real time; The face detection unit is used to quickly and accurately locate the face in the collected image using a face detection algorithm based on deep learning, providing accurate input for feature point positioning; The facial feature extraction unit is used to locate key feature points on the patient's face through a key point detection network model, and extract geometric and texture features from the key feature points; The facial motion capture unit is used to capture the patient's facial movements by comparing the coordinate position changes of key feature points in consecutive frames; The mouth opening training feedback unit is used to calculate the mouth opening according to the change of coordinate position: in, and They represent the coordinates of the feature points of the left and right mouth corners respectively. The value of the mouth opening directly reflects the size of the patient's mouth opening; Based on the calculated mouth opening value, the system generates training feedback. If the mouth opening degree does not reach the preset threshold, the system will prompt the patient to increase the mouth opening amplitude. If it exceeds the threshold, the system will prompt the patient to appropriately reduce the amplitude. The training effect is evaluated by comparing the changes in mouth opening degree before and after training.

4. The system according to claim 3, characterized in that: The pronunciation training evaluation module includes a speech signal acquisition unit, a speech data preprocessing unit, a speech feature extraction unit, a pronunciation detection unit and a pronunciation feedback training unit; The voice signal acquisition unit is used to capture the patient's voice signal through a high-sensitivity microphone array to ensure that the voice signal has a sufficient signal-to-noise ratio; The speech data preprocessing unit is used to perform noise reduction, silence detection and normalization processing on the speech signal to remove background noise and other non-pronunciation related sound waves contained in the speech signal; The speech feature extraction unit is used to extract key speech features from the preprocessed speech signal, including spectrum features and long-term average amplitude; The pronunciation detection unit is used to train a deep neural network model through key speech features, use the trained speech model to perform real-time detection of the patient's pronunciation, identify phonemes and syllables in the pronunciation, and identify incorrect pronunciation by comparing the model prediction results with the actual pronunciation; The pronunciation feedback training unit is used to generate pronunciation feedback information according to the recognition result of the incorrect pronunciation, and provide pronunciation training suggestions according to the feedback information.

5. The system according to claim 4, characterized in that: The swallowing training evaluation module includes an electromyographic signal acquisition unit, a jaw swing data acquisition unit, a swallowing action detection unit and a swallowing training feedback unit; The electromyographic signal acquisition unit is used to obtain the electromyographic signal change data of the patient's thyrohyoid muscle and masticatory muscle through the electromyographic sensor; The jaw swing data acquisition unit is used to obtain the patient's jaw swing force data through a pressure sensor, and to detect the patient's jaw swing amplitude data through a motion sensor; The swallowing action detection unit is used to determine whether the patient has completed the swallowing action according to the change of the electromyographic signal; The swallowing training feedback unit is used to detect whether the swing force and swing amplitude reach a preset safety threshold. If so, a training cycle corresponding to the patient is generated based on the swing force and swing amplitude of the patient's lower jaw, and the patient is guided to perform rehabilitation training. If not, the electromyographic signal change data is re-detected to see if it meets the changes in the swallowing action.

6. The system according to claim 5, characterized in that: The face detection algorithm used in the face detection unit is the SSD algorithm or the YOLO algorithm.

7. The system according to claim 6, characterized in that: The key point detection network model used in the facial feature extraction unit is the MHRB-net convolutional neural network model.

8. The system according to claim 7, characterized in that: The network architecture of the deep neural network model used in the pronunciation detection unit is a recurrent neural network or a long short-term memory network.

9. A method for analyzing oral function rehabilitation training after oral and maxillofacial surgery, characterized in that: The following steps are involved: Collect and analyze the data of bite force training and lip closure force training, and correct incorrect training movements through analysis results to improve the recovery rate of bite force and lip closure force training; Construct a facial key point detection network model, use the facial key point detection network model to perform image recognition and facial motion capture on the patient's face, obtain mouth opening data, analyze and compare the mouth opening data collected multiple times, and evaluate the mouth opening training effect based on the comparison results; Collect audio data when the patient reads the preset voice, compare the audio data with the preset voice, and judge whether the patient's pronunciation is accurate based on the comparison results, so as to evaluate the pronunciation training effect; Obtain the patient's electromyographic signal change data, jaw swing force data and jaw swing amplitude data, determine whether the patient has completed the swallowing action based on the electromyographic signal change data, and generate a rehabilitation training cycle for the corresponding patient based on the patient's swallowing situation combined with the jaw swing force data and the jaw swing amplitude data, and guide the patient to perform rehabilitation training.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the method according to any one of claims 9 are implemented.

Images