Method for measuring jaw movement and mandibular joint reaction force

Abstract

The invention discloses a jaw movement and mandibular joint reaction force measuring method, and relates to the technical field of oral biomechanics and medical sensors, and the method comprises the following steps: configuring multiple sensors for a patient, carrying out data acquisition preparation, pasting an optical mark point at a key position of the face of the patient, and starting the multiple sensors for calibration. The method comprises the steps of acquiring bioelectrical signals and mechanical signals, capturing changes of the bioelectrical signals and the mechanical signals, obtaining jaw movement mechanical data, extracting kinematics, dynamics and bioelectrical signal features based on the jaw movement mechanical data, fusing the extracted features by using multi-modal data fusion to generate a comprehensive vector, training historical jaw movement mechanical data by using a convolutional neural network model, and obtaining a training result. According to the method, a multi-sensor data acquisition and convolutional neural network model combined measurement method is adopted, and multi-modal data deep fusion is realized. A multi-sensor data acquisition and convolutional neural network model combined measurement method is adopted.

Description

Technical Field

[0001] The present invention relates to the technical fields of oral biomechanics and medical sensors, and particularly to a method for measuring jaw movement and mandibular joint reaction force. Background Art

[0002] Accurate measurement of mandibular movement and its joint reaction force is of great significance for understanding the pathogenesis of temporomandibular joint diseases, evaluating treatment effects, and formulating personalized treatment plans. Traditional measurement methods such as X-ray films and CT scans can provide structural information and reflect the patient's condition to a certain extent, but there are still some limitations in measuring jaw movement and mandibular joint reaction force.

[0003] On the one hand, the existing technology faces challenges in integrating multiple sensing signals. Especially when dealing with multi-modal data, there is a lack of efficient and accurate data fusion operations, making it difficult to generate comprehensive diagnostic information. On the other hand, traditional prediction models perform poorly when processing complex mandibular movement data, with insufficient accuracy. They cannot automatically identify complex patterns and extract valuable information through deep learning models. Especially when using traditional machine learning for data analysis, their performance is limited by the quality of manually designed features. Summary of the Invention

[0004] In view of the above existing problems, the present invention is proposed.

[0005] Therefore, the present invention provides a method for measuring jaw movement and mandibular joint reaction force to solve the problems of difficult fusion of multi-modal data and insufficient accuracy of prediction models.

[0006] To solve the above technical problems, the present invention provides the following technical solutions: In a first aspect, the present invention provides a method for measuring jaw movement and mandibular joint reaction force, which includes configuring multi-sensors for a patient and preparing for data collection; After completing the data collection preparation, paste optical marker points on key positions of the patient's face, start the multi-sensors and perform calibration; Use the calibrated multi-sensors to capture changes in bioelectric signals and mechanical signals to obtain jaw movement biomechanical data; Based on the obtained jaw movement biomechanical data, extract kinematic, dynamic, and bioelectric signal features; Use multi-modal data fusion to fuse the extracted kinematic, dynamic, and bioelectric signal features to generate a comprehensive vector; Use a convolutional neural network model to train historical jaw movement biomechanical data to obtain a trained convolutional neural network model; Input the fused comprehensive vector into the trained convolutional neural network model to obtain the reaction force of the mandibular joint.

[0007] As a preferred embodiment of the method for measuring jaw movement and mandibular joint reaction force according to the present invention, the following steps are included for preparing data acquisition by configuring multi-sensors for the patient: Integrate highly sensitive pressure sensors into a dental tray to measure the biting force during jaw movement; Attach a small triaxial accelerometer to capture changes in the speed and direction of jaw movement; Install surface electromyogram sensors at both masseter muscles to record the electrical signals generated by muscle activities.

[0008] As a preferred embodiment of the method for measuring jaw movement and mandibular joint reaction force according to the present invention, the following steps are included for pasting optical marker points on key positions of the patient's face and calibrating the multi-sensors after completing data acquisition preparation: After completing data acquisition preparation, use medical adhesive to paste optical marker points on key positions of the patient's face to track jaw movement in real time; The key positions of the patient's face include the cheekbones, forehead, mandibular angles, and chin; After pasting the optical marker points, start the multi-sensors. Calibrate the pressure sensor inside the dental tray to zero under no-load conditions, calibrate the accelerometer to zero in a static state, and calibrate the electromyogram sensor to baseline in the patient's relaxed state to record the electrical signals at rest.

[0009] As a preferred embodiment of the method for measuring jaw movement and mandibular joint reaction force according to the present invention, the following steps are included for capturing changes in bioelectrical signals and mechanical signals using the calibrated multi-sensors to obtain jaw movement biomechanical data: When the patient performs opening and closing of the mouth, lateral movement, protrusion and retrusion, and chewing movements, use the calibrated pressure sensor inside the dental tray to capture changes in the biting force during jaw movement to obtain kinematic data; Use the calibrated accelerometer to capture changes in the movement speed and direction of the chin to obtain kinetic data; Use the calibrated electromyogram sensor to capture the electrical signals generated by the muscle activities of both masseter muscles to obtain bioelectrical signal data; Use a filtering algorithm to preliminarily process the obtained jaw movement biomechanical data to remove high-frequency noise and low-frequency drift.

[0010] As a preferred embodiment of the method for measuring jaw movement and mandibular joint reaction force according to the present invention, the following steps are included for extracting kinematic, kinetic, and bioelectrical signal features based on the obtained jaw movement biomechanical data: Based on the obtained jaw movement biomechanical data, kinematic features are extracted using time-domain statistical analysis and fast Fourier transform methods; Dynamical features are extracted using methods of directly reading pressure sensor data and calculating torque and bite duration; Bioelectrical signal features are extracted using methods of pre-amplification, band-pass filtering, and wavelet transform.

[0011] As a preferred embodiment of the measurement method for jaw movement and mandibular joint reaction force of the present invention, wherein: multi-modal data fusion is used to fuse the extracted kinematic, dynamical, and bioelectrical signal features to generate a comprehensive vector, which specifically includes the following steps, The extracted kinematic, dynamical, and bioelectrical signal features are dimensionally reduced using PCA to generate a dimensionally reduced feature vector; Using the dimensionally reduced feature vector and the corresponding covariance matrix, the Mahalanobis distance of each feature vector to its mean is obtained; The weighted Mahalanobis distance is used to quantify and integrate the kinematic, dynamical, and bioelectrical signal features to generate a comprehensive feature vector.

[0012] As a preferred embodiment of the measurement method for jaw movement and mandibular joint reaction force of the present invention, wherein: a convolutional neural network model is used to train the historical jaw movement biomechanical data to obtain a trained convolutional neural network model, which specifically includes the following steps, The historical jaw movement biomechanical data is normalized, and a convolutional neural network model architecture is constructed; The convolutional neural network model architecture includes an input layer, a convolutional layer, a pooling layer, a fully connected layer, and an output layer; The convolutional neural network model architecture is compiled using a loss function and an Adam optimizer; The compiled convolutional neural network model architecture is trained using the normalized historical jaw movement biomechanical data, and the early stopping method is adopted to prevent overfitting to obtain a trained convolutional neural network model.

[0013] As a preferred embodiment of the measurement method for jaw movement and mandibular joint reaction force of the present invention, wherein: the fused comprehensive vector is input into the trained convolutional neural network model to obtain the reaction force of the mandibular joint, which specifically includes the following steps, The fused comprehensive features are input into the trained convolutional neural network model; Forward propagation is performed on the fused comprehensive features, passing through the convolutional layer, the pooling layer, and the fully connected layer, and finally the prediction result is output to obtain the magnitude of the reaction force.

[0014] In a second aspect, the present invention provides a computer device, including a memory and a processor, where the memory stores a computer program, and: when the computer program is executed by the processor, any step of the method for measuring jaw movement and mandibular joint reaction force as described in the first aspect of the present invention is implemented.

[0015] In a third aspect, the present invention provides a computer-readable storage medium, on which a computer program is stored, and: when the computer program is executed by the processor, any step of the method for measuring jaw movement and mandibular joint reaction force as described in the first aspect of the present invention is implemented.

[0016] The beneficial effects of the present invention are as follows: By proposing a measurement method combining multi-sensor data acquisition and a convolutional neural network model, this method not only uses multi-modal data fusion technology to efficiently fuse data from multiple sensors, including bite force, acceleration, and electromyogram signals, facilitating condition analysis, but also adopts an advanced deep learning model to automatically identify key features through a trained convolutional neural network model, providing strong support for the accurate diagnosis of temporomandibular joint diseases, while improving the data processing efficiency and significantly enhancing the efficiency and accuracy of measuring the mandibular joint reaction force. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the following drawings are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0018] Figure 1 It is a flowchart of the method for measuring jaw movement and mandibular joint reaction force in Embodiment 1.

[0019] Figure 2 It is a flowchart of obtaining the mandibular joint reaction force using jaw movement biomechanical data in Embodiment 1. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0020] To make the above objects, features, and advantages of the present invention more obvious and understandable, the following will provide a detailed description of the specific embodiments of the present invention with reference to the accompanying drawings of the specification.

[0021] Embodiment 1, referring to Figure 1 and Figure 2 , which is the first embodiment of the present invention. This embodiment provides a method for measuring jaw movement and mandibular joint reaction force, including the following steps: S1. Configure multi-sensors for the patient and prepare for data acquisition.

[0022] Specifically, it includes the following steps. To prepare for configuring multiple sensors for a patient to collect data, first select a suitable dental tray and ensure it is comfortable and does not affect the patient's normal occlusion. Integrate a highly sensitive pressure sensor into the dental tray, ensuring that the pressure sensor is fully embedded in the dental tray and its sensing part faces the tooth contact area to accurately capture changes in the biting force and obtain high-quality biting force data; Paste an accelerometer on the chin area of the patient's face, which can directly reflect the changes in the speed and direction of mandibular movement and maximize the reflection of the overall movement characteristics of the mandible. Use medical-grade double-sided tape to paste a small triaxial accelerometer on the chin to ensure that the accelerometer does not shift or fall off during the entire experiment; At the masseter muscle positions on both sides of the patient's face, use a special adhesive to firmly attach surface electromyography sensors to the masseter muscles on both sides, ensuring good contact between the sensors and the skin to obtain clear electrical signals and reduce the influence of external noise.

[0023] S2. After completing the data collection preparation, paste optical marker points on key positions of the patient's face, start the multiple sensors, and perform calibration.

[0024] Specifically, it includes the following steps: After completing the preparation work for data collection, first, precisely paste optical marker points on key positions of the patient's face. These key positions include the cheekbones, forehead, mandibular angles, and chin because these areas can effectively reflect the movement trajectory and posture changes of the mandible. The role of the optical marker points is to track the movement of the mandible in real time and provide high-precision reference data for subsequent kinematic analysis. In specific operations, paste 6 marker clusters (each cluster contains 3 - 4 optical marker points) on both sides of the patient's cheeks, forehead, and mandible to form a plane. Then, arrange four infrared cameras on both sides of the cheeks to ensure that at least two cameras can record the position of each optical marker point. Next, record the positions of all optical marker points in the static state, establish the relative position relationship between the temporomandibular joint cluster and the mandibular cluster. After the preliminary setup, remove the marker clusters on both sides of the cheeks and only retain the marker clusters on the forehead and mandible. This is because the skin on both sides of the cheeks has a large displacement during movement, and the trajectory of the temporomandibular joint during movement can be more accurately determined through the relative position of the mandibular cluster. If the patient has their own CT scan data, the position of the temporomandibular joint can be directly calculated, and thus only 2 marker clusters (forehead and mandible) need to be pasted from the beginning; After pasting the optical marker points, start the multiple sensors and perform detailed calibration operations on each sensor to ensure data accuracy; Zero-calibrate the pressure sensor built into the dental prosthesis in a no-load state. The specific operation is to place the dental prosthesis in the patient's mouth and ensure that the patient does not apply any biting force. At this time, adjust the output value of the pressure sensor to zero to eliminate the initial error. During the calibration process, a dedicated calibration software, such as Biopac Student Lab, is required to monitor the output value of the sensor in real time and ensure the stability of the zero point; Zero-calibrate the accelerometer in a stationary state. The patient keeps the head stationary and calibrate the output value of the accelerometer to ensure that its reading in a non-moving state is zero, thus eliminating the static error. During the calibration process, it is necessary to ensure that the patient's head is in a natural and relaxed state to avoid any minute movement interfering with the calibration result. Use the calibration software Biopac StudentLab to monitor the three-axis output of the accelerometer, including the X, Y, and Z axes, and adjust the output of each axis to zero in a stationary state; Baseline-calibrate the electromyogram sensor in a relaxed state of the patient. The patient keeps the mandibular muscles completely relaxed and record the electrical signal of the electromyogram at this time as the baseline value, so as to accurately capture the change of the electrical signal during muscle activity subsequently. During the calibration process, it is necessary to ensure that the patient is in a quiet environment to avoid external interference affecting the accuracy of the baseline value, and observe whether the baseline signal is stable. Record the baseline value in a completely relaxed state of the patient. If the baseline signal fluctuates greatly, check whether the electrode contact is good and re-calibrate.

[0025] S3. Use the calibrated multi-sensor to capture the changes of bioelectrical signals and mechanical signals and obtain the jaw movement biomechanical data.

[0026] Specifically, it includes the following steps, After calibration is completed, start using the multi-sensor to capture the changes of bioelectrical signals and mechanical signals during the mandibular movement of the patient. When the patient performs a series of standardized movements, including opening and closing the mouth, lateral movement, protrusion and retrusion, and chewing movements, the calibrated pressure sensor built into the dental prosthesis will capture the dynamic changes of the biting force during the mandibular movement in real time. These data reflect the distribution and magnitude of the biting force and are important bases for kinematic analysis; The calibrated accelerometer will capture the changes in the movement speed and direction of the chin area, record the acceleration and movement trajectory of the mandible, so as to obtain kinetic data. The sampling frequency of the accelerometer needs to be set high enough, usually above 1000Hz, to capture the details of fast movements. During the process of collecting data, monitor the output curve of the accelerometer in real time to ensure the integrity and accuracy of the data; The calibrated electromyography sensor synchronously captures the electrical signals generated by the muscle activities of the bilateral masseter muscles, records the changes in the electrical signals during muscle contraction and relaxation, and obtains bioelectrical signal data. The electrodes of the electromyography sensor need to be accurately placed at the muscle belly position of the masseter muscle to ensure the accuracy of signal acquisition. The sampling frequency of the electromyography signal is usually set above 1000 Hz to ensure the capture of the details of muscle activities; During the data acquisition process, in order to ensure the quality of the data, it is necessary to perform preliminary processing on the obtained jaw movement biomechanical data, and use filtering algorithms for processing to remove high-frequency noise caused by electromagnetic interference and equipment vibration, and low-frequency drift caused by baseline drift and sensor temperature drift; In specific operations, check the collected jaw movement biomechanical data to identify abnormal noise or drift phenomena, including using the data visualization software MATLAB to plot the original signal curves of the pressure sensor, accelerometer, and electromyography sensor, and observe whether there is obvious noise or drift. For high-frequency noise, a low-pass filter is usually used for removal. For low-frequency drift, a high-pass filter is usually used for removal to ensure that the filtered jaw movement biomechanical data is smoother and more reliable, providing a high-quality basis for subsequent feature extraction and analysis.

[0027] S4. Extract kinematic, kinetic, and bioelectrical signal features based on the obtained jaw movement biomechanical data.

[0028] Specifically, it includes the following steps: After obtaining the jaw movement biomechanical data, extract features for kinematic data. The kinematic data mainly includes the position, velocity, and acceleration information of the mandibular movement. Perform time-domain statistical analysis on the filtered jaw movement biomechanical data. By calculating the maximum displacement, average movement velocity, and peak acceleration of the mandible during opening and closing movements, extract time-domain features including the movement range, average velocity, and peak acceleration. Use the fast Fourier transform to convert the time-domain signal into a frequency-domain signal and extract frequency-domain features, including the main frequency component and energy distribution, to analyze the periodic characteristics of the mandibular movement; The extraction of dynamic features is mainly based on the bite force data captured by the built-in pressure sensor of the mouthpiece. The raw data of the pressure sensor is directly read to obtain the real-time change curve of the bite force. The dynamic features are further extracted by calculating the torque and bite duration. The torque calculation is based on the distribution of the bite force and the geometric parameters of the mandible, including the calculation of the reaction torque of the mandibular joint by the product of the bite force and the lever arm. The bite duration is determined by analyzing the start and end time of the bite force curve, such as the time period from the start of the bite force to the drop to the baseline. These dynamic features can reflect the strength, stability and duration of the bite, providing an important basis for analyzing the mandibular function. The extraction of bioelectric signal features is mainly based on the masseter muscle electrical signals captured by the electromyography sensor. First, the original electromyographic signals are pre-amplified to improve the signal-to-noise ratio. Then, a bandpass filter is used to remove low-frequency baseline drift and high-frequency noise to retain the effective frequency band of the electromyographic signal. The filtered signal is further analyzed by wavelet transform to extract time-frequency domain features. Wavelet transform can provide time and frequency information at the same time and is suitable for analyzing non-stationary electromyographic signals. Through wavelet transform, features including the start time, duration, frequency components and energy distribution of muscle activity can be extracted, such as analyzing the activation mode and fatigue degree of the masseter muscle during chewing. These bioelectric signal features can reflect the dynamic changes of muscle activity and provide important information for evaluating mandibular muscle function. Through the above operations, key features were extracted from kinematics, kinetics and bioelectric signal data respectively, which laid the foundation for subsequent multimodal data fusion and comprehensive analysis.

[0029] S5. Use multimodal data fusion to fuse the extracted kinematic, dynamic and bioelectric signal features to generate a comprehensive vector.

[0030] The specific operations include the following: After extracting the kinematic, dynamic and bioelectric signal features, PCA principal component analysis is used to reduce the dimensionality of the features. PCA can map high-dimensional data to low-dimensional space through linear transformation while retaining the main information of the data. In the specific operation, the extracted kinematic, dynamic and bioelectric signal features are respectively composed of feature matrices, and then each feature matrix is ​​processed by PCA. First, the covariance matrix of the feature matrix is ​​calculated, and then the covariance matrix is ​​decomposed by eigenvalue to obtain the eigenvalues ​​and corresponding eigenvectors. The eigenvalues ​​are sorted, and the main components are selected according to the needs. The original feature matrix is ​​projected onto these main components to generate the reduced eigenvectors. Using the dimension-reduced feature vectors and the corresponding covariance matrices, calculate the Mahalanobis distance of each feature vector to its mean. The Mahalanobis distance is a distance metric that takes into account the data distribution characteristics and can effectively reflect the distribution of feature vectors in a multi-dimensional space. In specific operations, first calculate the mean vector and covariance matrix of each modal feature vector. For each feature vector, calculate its Mahalanobis distance to the mean vector. The formula is: ; where represents the Mahalanobis distance of the -th sensor feature vector, represents the feature vector of the -th sensor, represents the mean of the feature vector of the -th sensor, represents the transpose operation, represents the -th sensor's covariance matrix; Use the weighted Mahalanobis distance to quantify and integrate the kinematic, dynamic, and bioelectric signal features to generate a comprehensive feature vector. The mathematical expression of the weighted Mahalanobis distance is: ; where represents the fused comprehensive vector, represents the number of sensor types, represents the index of different sensors, represents the -th sensor's weight coefficient, represents the feature vector of the -th sensor, represents the mean of the feature vector of the -th sensor, represents the transpose operation, represents the -th sensor's covariance matrix; Through the above operations, the fusion of multi-modal data is achieved, and a comprehensive feature vector is generated. This process not only reduces the dimension of the data but also integrates the feature information of different modalities, providing high-quality input data for the subsequent training of the convolutional neural network model.

[0031] S6. Use the convolutional neural network model to train the historical jaw movement biomechanical data to obtain a trained convolutional neural network model.

[0032] Specifically, it includes the following operations: Before training the convolutional neural network model using historical jaw movement biomechanical data, the historical jaw movement biomechanical data is normalized to scale the feature data of different modalities to the same scale, avoiding excessive influence on model training due to some features having a large numerical range. In specific operations, the mean and standard deviation of each feature dimension of the historical jaw movement biomechanical data are calculated respectively for normalization, and its mathematical expression is: ; where represents the normalized feature value, represents the original feature value, represents the mean of the feature values, represents the standard deviation of the feature values; After normalization, a convolutional neural network model architecture is constructed. This architecture includes an input layer, a convolutional layer, a pooling layer, a fully connected layer, and an output layer. The input layer receives the normalized historical jaw movement biomechanical data, and its dimension is consistent with that of the comprehensive feature vector. The convolutional layer extracts local features of the data through convolutional kernels. Usually, multiple convolutional kernels are set to capture feature information at different levels. After the convolutional operation, the ReLU activation function is used to introduce non-linearity, and its mathematical expression is: ; where, represents the activation function, and y represents an element in the output features of a certain convolutional layer; The max pooling layer is used to reduce the spatial dimension of the data, reduce the computational amount, and prevent overfitting. The fully connected layer integrates the features extracted by the convolutional layer and the pooling layer and transmits information through multiple layers of neurons. The output layer is designed according to the task requirements. For example, the Softmax function is used for classification tasks and the linear activation function is used for regression tasks, and its mathematical expression is: Regression task: ; where represents the mean squared error loss value, represents the number of samples used to calculate the loss, represents the sample index, represents the th actual mandibular joint reaction force value of the sample, represents the th predicted probability distribution of the mandibular joint reaction force of the sample; Classification task: ; where represents the cross-entropy loss value, represents the number of samples used to calculate the loss, represents the sample index, represents the The actual mandibular joint reaction force value of a sample indicating the predicted probability distribution of the mandibular joint reaction force of the After the construction of the convolutional neural network model architecture is completed, the model is compiled using a loss function and the Adam optimizer. The loss function is used to measure the difference between the model's predicted values and the true values, including mean squared error and cross-entropy loss, which are used for regression tasks and classification tasks respectively. The Adam optimizer is an adaptive learning rate optimization algorithm that can dynamically adjust the learning rate according to gradient information, improving the efficiency and stability of model training. During the compilation process, hyperparameters such as the initial learning rate, batch size, and number of training epochs are set. In specific operations, the API provided by deep learning frameworks such as TensorFlow is used for model compilation to ensure that the model can correctly calculate the loss and update the parameters; The compiled convolutional neural network model is trained using the normalized historical mandibular movement biomechanical data. During the training process, the dataset is divided into a training set and a validation set. The training set is used for learning the model parameters, and the validation set is used to evaluate the generalization ability of the model. To prevent overfitting, the early stopping method is adopted to monitor the loss of the validation set. In specific operations, a tolerance threshold is set. When the validation set loss does not decrease for several consecutive epochs, the training is terminated early and the current optimal model is saved. After the training is completed, the trained convolutional neural network model is obtained. This model can effectively capture the characteristics of the historical mandibular movement biomechanical data and is used for subsequent mandibular joint reaction force prediction tasks.

[0033] S7. Input the fused comprehensive vector into the trained convolutional neural network model to obtain the reaction force of the mandibular joint.

[0034] Specifically, it includes the following steps After inputting the fused comprehensive feature vector into the trained convolutional neural network model, a forward propagation operation is performed on the comprehensive features. Forward propagation is the core calculation process of the convolutional neural network. By passing data layer by layer and calculating the outputs of each layer, the prediction result is finally obtained. The specific operation is as follows; The comprehensive feature vector is first input into the convolutional layer. The convolutional layer extracts the local information of the features through convolutional kernels. Each convolutional kernel slides on the input data, calculates the weighted sum of the local area, and introduces non-linearity through the ReLU activation function. The output feature map of the convolutional layer is then passed to the max pooling layer. The pooling layer reduces the spatial dimension of the data through downsampling operations while retaining the main feature information. The pooled feature map is further passed to the fully connected layer. The fully connected layer flattens the feature map into a one-dimensional vector and performs non-linear transformations through multiple layers of neurons, finally outputting the prediction result, that is, the predicted value of the mandibular joint reaction force. The formula is as follows: ; Among them, represents the predicted value of the mandibular joint reaction force, represents the number of sensor types, represents the index of different sensors, represents the th real-time feature vector of the sensor, represents the mean of the feature vectors of all sensors, represents the transpose operation, represents the covariance matrix, represents the empirical coefficient for adjusting the sensitivity of the regression formula, represents the empirical coefficient to ensure the positive value inside the logarithmic function, represents the empirical coefficient for adjusting the weight of the logarithmic term in the entire loss function, represents the standardized comprehensive feature vector dimension of, represents the standardized comprehensive feature vector in the th element.

[0035] Through the above operations, the fused comprehensive feature vector is input into the trained convolutional neural network model, and the predicted value of the mandibular joint reaction force is obtained through forward propagation. This process makes full use of the feature extraction ability and non-linear modeling ability of the convolutional neural network, can effectively capture the complex patterns of mandibular movement, and achieve high-precision reaction force prediction.

[0036] S8. Based on the reaction force of the mandibular joint, use Vicon Nexus to generate a three-dimensional motion trajectory, and calculate the key indicators of joint angle and torque to obtain three-dimensional reconstruction data.

[0037] Specifically, it includes the following steps: After obtaining the reaction force of the mandibular joint, first use Vicon Nexus to generate a three-dimensional motion trajectory of mandibular movement. Vicon Nexus can perform motion capture based on optical marker points and can record the position changes of the marker points in three-dimensional space with high precision. In specific operations, first paste the optical marker points at the key positions on the patient's face. After starting Vicon Nexus, the position information of the marker points is captured in real time by multiple high-speed infrared cameras and converted into three-dimensional coordinate data. Vicon Nexus will automatically identify the trajectory of each marker point and generate a three-dimensional motion trajectory of mandibular movement. These trajectory data include the displacement, velocity, and acceleration information of the marker points in the X, Y, and Z directions, providing a basis for subsequent joint angle and torque calculations; Based on the generated three-dimensional motion trajectory, the key indicators of the angle and torque of the mandibular joint are calculated. The calculation of the joint angle is usually based on the geometric relationship of the marker points. The geometric model of the mandible can be constructed through the coordinates of the zygomatic bone, mandibular angle and chin marker points, and the angle change of the mandible relative to the skull can be calculated. In the specific operation, vector operations are used to calculate the joint angle. For example, by calculating the angle between the vector from the mandibular angle to the zygomatic bone and the vector from the mandibular angle to the chin, the opening angle of the mandible is obtained. Similarly, the lateral movement and the angle of extension and retraction can be calculated. The calculation of joint torque is based on the reaction force and the geometric parameters of the joint. For example, the torque of the mandibular joint is calculated by multiplying the reaction force and the lever arm, such as the distance from the mandibular angle to the joint center. These angle and torque data can reflect the biomechanical characteristics of mandibular movement and provide an important basis for functional evaluation. Based on the above calculation results, 3D reconstruction data is obtained. The 3D reconstruction data includes the 3D trajectory of mandibular movement, the curve of joint angle change, and the torque distribution diagram. These data can be displayed and analyzed through visualization software such as the tools provided by Vicon Nexus, such as generating a 3D animation of mandibular movement, intuitively displaying the angle and torque changes during the movement, drawing the curves of joint angle and torque changes over time, analyzing the stability and symmetry of the movement, generating a torque distribution diagram, and evaluating the distribution of the bite force. The 3D reconstruction data not only provides an intuitive reference for clinical diagnosis and treatment, but also provides high-quality data support for in-depth research on the biomechanical mechanism of mandibular movement. Through the above operations, Vicon Nexus is used to generate a three-dimensional motion trajectory based on the reaction force, and key indicators such as joint angle and torque are calculated to finally obtain three-dimensional reconstruction data. This process combines motion capture technology, geometric modeling and biomechanical analysis, and can comprehensively and accurately reflect the functional status of mandibular movement.

[0038] S9 Based on the preliminary estimation of static muscle force obtained from literature data, in order to further obtain the muscle contraction force and temporomandibular joint (TMJ) reaction force and torque in individualized and dynamic processes, a fine mandibular musculoskeletal mechanical model was constructed, and dynamic calculations were performed in combination with three-dimensional motion capture data and inverse dynamics analysis. The specific steps are as follows: Step 1: Constructing the mandibular musculoskeletal mechanical model

[0039] Extract three-dimensional structural parameters of the mandible and related masticatory muscles from CT scan images. According to anatomical measurement data, obtain the origin and insertion coordinates, muscle length, and moment arm length, etc. of the key masticatory muscles [including the main muscles during mandibular closure (masseter, temporalis, medial pterygoid, lateral pterygoid), and the main muscles during mandibular opening (digastric, geniohyoid, mylohyoid, stylohyoid)]. Use data such as muscle volume and physiological cross-sectional area in the public literature (e.g., Van Eijden et al., 1997), combined with the intrinsic muscle tension value (37 N / cm²), to estimate the maximum contractile force of each muscle. This model provides a static anatomical basis for subsequent kinetic calculations.

[0040] The maximum isometric contractile force of each muscle is calculated by the following formula: F i,max =σ⋅PCSA i Where, PCSA i =V i / L i , V i is the muscle volume, L i is the muscle fiber length, and the intrinsic muscle strength σ (sigma) is taken as 37 N / cm². The muscle volume is derived from anatomical literature data or CT reconstruction measurements. The muscle force direction is calculated from the origin and insertion coordinates of the muscle.

[0041] Step two: Obtain actual motion data By setting facial marker points, use a three-dimensional motion capture system (such as Vicon) to record the mandibular movement trajectories of the subjects during natural mouth opening, closing, and chewing. Simultaneously use a dental sensor to measure the mandibular bite force. The above data are used to drive the musculoskeletal dynamics model to achieve individualized simulation.

[0042] Step three: Muscle dynamic mechanics modeling and optimization solution

[0043] Import the musculoskeletal model constructed in step one and the mandibular movement trajectory data obtained in step two into the MATLAB platform for analysis and processing. First, calculate the spatial position and attitude changes of the mandible at each time point based on the motion capture marker point data, and then establish a kinematic model of the mandibular rigid body. Based on the known bite force input and muscle attachment point information, construct a static equilibrium equation system.

[0044] During the process of solving muscle forces, the Non-negative Least Squares (NNLS) or Static Optimization algorithm is used to solve the redundant muscle system. The optimization goal is to minimize the sum of the squares of the activation levels of each muscle to achieve a reasonable distribution of forces. The above methods can obtain the actual contraction forces of each chewing muscle at each moment and are used for subsequent calculation of the reaction forces of the temporomandibular joint.

[0045] Step Four: Calculate the resultant muscle force and the reaction force of the temporomandibular joint

[0046] After solving the contraction forces of each muscle at each moment, the vector forces of all activated muscles are summed to obtain the total resultant muscle force during mandibular movement , (where a i is the muscle activation level, and \(\vec{u}\) i is the muscle direction vector).

[0047] Based on the actual measured value of the biting force \(F_b\), combined with the known muscle attachment points and joint position relationships, applying the principle of static equilibrium, through the three-force system (resultant muscle force , occlusal point reaction force , temporomandibular joint reaction force ), the reaction force and moment of the temporomandibular joint are solved. The formulas are as follows: F m +F b +R TMJ =0 (force equilibrium equation) (moment equilibrium equation)

[0048] S9. Combine the three-dimensional reconstruction data and the reaction force of the temporomandibular joint to generate a diagnostic report.

[0049] Specifically, it includes the following steps: After obtaining the three-dimensional reconstruction data and the reaction force of the temporomandibular joint, these data are integrated and standardized. The three-dimensional reconstruction data includes the three-dimensional trajectory of mandibular movement, the joint angle change curve, and the moment distribution map, while the reaction force of the temporomandibular joint is a mechanical index predicted by a convolutional neural network model. To combine these two types of data, first, their time series need to be aligned to ensure that the movement data and reaction force data at each time point can correspond; In specific operations, timestamps are used to synchronize the data, and interpolation methods are employed to fill in any potential missing data. The data is then normalized, for example, by normalizing joint angles and torques to the same numerical range for subsequent analysis and visualization. Based on the integrated data, key features are extracted and a diagnostic report is generated. The content of the diagnostic report typically includes the following parts: Kinematics analysis: Based on three-dimensional motion trajectories and joint angle change curves, the range, speed, symmetry, and stability of mandibular movement are analyzed. For example, indicators such as mouth opening degree, lateral movement range, and protrusion / retraction range are calculated, and curves showing the change of angles over time are plotted to evaluate whether the movement is normal or has abnormal patterns. Mechanical analysis: Based on the mandibular joint reaction force and torque distribution maps, the intensity, distribution, and variation law of the biting force are analyzed. For example, the maximum biting force, average biting force, and peak values and distributions of torques are calculated to evaluate the load status and functional adaptability of the mandibular joint. Comprehensive evaluation: Combining the results of kinematics and mechanical analyses, the functional status of mandibular movement is comprehensively evaluated. For example, it is observed whether there are problems such as excessive joint load, movement asymmetry, and poor muscle coordination, and clinical diagnostic opinions are put forward. Finally, the analysis results are organized into a complete diagnostic report, which usually includes the following parts: Patient information: Includes basic information such as name, gender, age, and examination date. Examination results: Details of the analysis results of three-dimensional reconstruction data and reaction forces are described, including kinematic indicators, mechanical indicators, and comprehensive evaluation conclusions. Diagnostic opinions: Based on the analysis results, clinical diagnostic opinions are put forward, such as temporomandibular joint disorders, occlusal abnormalities, and muscle dysfunction. Correction plan: Based on the clinical diagnostic opinions, the symmetry of mandibular joint angles and torques during mandibular movement is analyzed for the patient. By optimizing the mandibular opening angle and muscle strength, correct alignment and symmetry are achieved to restore the function of the temporomandibular joint. Specific correction measures can include wearing splints or night guards, and receiving orthodontic treatments such as braces to achieve proper balance and alignment of the occlusion.

[0050] Through the above operations, the three-dimensional reconstruction data and mandibular joint reaction forces are combined to generate a detailed diagnostic report. This process not only provides comprehensive diagnostic basis for clinicians but also offers scientific support for the formulation of patient treatment plans.

[0051] This embodiment also provides a computer device, which is applicable to the measurement method of jaw movement and mandibular joint reaction force, and includes: a memory and a processor; the memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions to implement the measurement method of jaw movement and mandibular joint reaction force proposed in the above embodiment.

[0052] The computer device can be a terminal. The computer device includes a processor, a memory, a communication interface, a display screen, and an input device connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be implemented through WIFI, a carrier network, NFC (Near Field Communication), or other technologies. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen. The input device of the computer device can be a touch layer covered on the display screen, or a button, a trackball, or a touchpad set on the shell of the computer device, or an external keyboard, a touchpad, or a mouse, etc.

[0053] This embodiment also provides a storage medium, on which a computer program is stored. When the program is executed by a processor, it implements the measurement method of jaw movement and mandibular joint reaction force proposed in the above embodiment; the storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as Static Random Access Memory (SRAM for short), Electrically Erasable Programmable Read-Only Memory (EEPROM for short), Erasable Programmable Read Only Memory (EPROM for short), Programmable Read-Only Memory (PROM for short), Read-Only Memory (ROM for short), magnetic memory, flash memory, a magnetic disk, or an optical disc.

[0054] In summary, the present invention provides a measurement method based on multi-sensor data acquisition and a convolutional neural network model. This method not only uses multi-modal data fusion technology to efficiently fuse data from multiple sensors, including bite force, acceleration, and electromyogram signals, facilitating disease analysis, but also adopts an advanced deep learning model to automatically identify key features through a trained convolutional neural network model, providing strong support for the accurate diagnosis of temporomandibular joint diseases. While improving the data processing efficiency, it significantly enhances the efficiency and accuracy of measuring the reaction force of the jaw joint.

[0055] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit them. Although the present invention has been described in detail with reference to the preferred embodiments, those of ordinary skill in the art should understand that the technical solutions of the present invention can be modified or equivalently replaced without departing from the spirit and scope of the technical solutions of the present invention, and they should all be covered within the scope of the claims of the present invention.

Claims

1. A method for measuring jaw movement and the reaction force of the mandibular joint, characterized in that: Including, Configure multiple sensors for the patient and prepare for data collection; After completing the data collection preparation, paste optical marker points at key positions on the patient's face, start the multiple sensors and perform calibration; Use the calibrated multiple sensors to capture the changes in bioelectric signals and mechanical signals and obtain jaw movement biomechanical data; Based on the obtained jaw movement biomechanical data, extract kinematic, dynamic and bioelectric signal features; Use multi-modal data fusion to fuse the extracted kinematic, dynamic and bioelectric signal features to generate a comprehensive vector; Use a convolutional neural network model to train the historical jaw movement biomechanical data to obtain a trained convolutional neural network model; Input the fused comprehensive vector into the trained convolutional neural network model to obtain the reaction force of the temporomandibular joint.

2. The measurement method of jaw movement and mandibular joint reaction force according to claim 1, characterized in that: The step of configuring multiple sensors for the patient and preparing for data collection specifically includes the following steps: Integrate a highly sensitive pressure sensor into a dental tray to measure the biting force during mandibular movement; Use a small triaxial accelerometer sticker to capture the changes in the speed and direction of mandibular movement; Use surface electromyography sensors installed at both masseter muscles to record the electrical signals generated by muscle activities.

3. The method for measuring jaw movement and mandibular joint reaction force according to claim 2, wherein: After completing the data collection preparation, paste optical marker points at key positions on the patient's face, start the multiple sensors and perform calibration, specifically including the following steps: After completing the data collection preparation, use a medical adhesive to paste the optical marker points at key positions on the patient's face to track the mandibular movement in real time; The key positions on the patient's face include the cheekbones, forehead, mandibular angles and chin; After pasting the optical marker points, start the multiple sensors. Calibrate the pressure sensor inside the dental tray to zero under no-load conditions, calibrate the accelerometer to zero in a stationary state, and calibrate the baseline of the electromyography sensor in the relaxed state of the patient to record the electrical signals at rest.

4. The method for measuring jaw movement and mandibular joint reaction force according to claim 3, wherein: The step of using the calibrated multiple sensors to capture the changes in bioelectric signals and mechanical signals and obtain jaw movement biomechanical data specifically includes the following steps: When the patient performs opening and closing, lateral movement, protrusion and retrusion, and chewing movements, use the calibrated pressure sensor inside the dental tray to capture the changes in the biting force during mandibular movement and obtain kinematic data; Use the calibrated accelerometer to capture the changes in the movement speed and direction of the chin area and obtain dynamic data; Use the calibrated electromyography sensor to capture the electrical signals generated by the muscle activities of both masseter muscles and obtain bioelectric signal data; Perform preliminary processing on the obtained jaw movement biomechanical data using a filtering algorithm to remove high-frequency noise and low-frequency drift.

5. The method for measuring jaw movement and temporomandibular joint reaction force according to claim 4, wherein: The step of extracting kinematic, dynamic and bioelectric signal features based on the obtained jaw movement biomechanical data specifically includes the following steps: Based on the obtained jaw movement biomechanical data, use time-domain statistical analysis and fast Fourier transform methods to extract kinematic features; Use the method of directly reading the pressure sensor data and calculating the torque and bite duration to extract dynamic features; Use the methods of pre-amplification, band-pass filtering and wavelet transform to extract bioelectric signal features.

6. The method for measuring jaw movement and mandibular joint reaction force according to claim 5, characterized in that: The use of multi-modal data fusion to fuse the extracted kinematic, kinetic, and bioelectrical signal features to generate a comprehensive vector specifically includes the following steps: Perform dimensionality reduction on the extracted kinematic, kinetic, and bioelectrical signal features using PCA to generate a feature vector after dimensionality reduction; Use the feature vector after dimensionality reduction and the corresponding covariance matrix to obtain the Mahalanobis distance of each feature vector to its mean; Use the weighted Mahalanobis distance to quantify and integrate the kinematic, kinetic, and bioelectrical signal features to generate a comprehensive feature vector.

7. The method for measuring jaw movement and mandibular joint reaction force according to claim 6, characterized in that: The use of a convolutional neural network model to train historical jaw movement biomechanical data to obtain a trained convolutional neural network model specifically includes the following steps: Perform normalization on the historical jaw movement biomechanical data and construct the architecture of the convolutional neural network model; The architecture of the convolutional neural network model includes an input layer, a convolutional layer, a pooling layer, a fully connected layer, and an output layer; Use a loss function and an Adam optimizer to compile the architecture of the convolutional neural network model; Train the compiled architecture of the convolutional neural network model using the normalized historical jaw movement biomechanical data and adopt the early stopping method to prevent overfitting to obtain a trained convolutional neural network model.

8. The method for measuring jaw movement and mandibular joint reaction force according to claim 7, characterized in that: The step of inputting the fused comprehensive vector into the trained convolutional neural network model to obtain the reaction force of the temporomandibular joint specifically includes the following steps: Input the fused comprehensive features into the trained convolutional neural network model; Perform forward propagation on the fused comprehensive features, passing through the convolutional layer, the pooling layer, and the fully connected layer, and finally output the prediction result to obtain the magnitude of the reaction force.

9. A computer device, comprising a memory and a processor, the memory storing a computer program, characterized in that: When the processor executes the computer program, it implements the steps of the method for measuring jaw movement and temporomandibular joint reaction force according to any one of claims 1 to 8.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by the processor, it implements the steps of the method for measuring jaw movement and temporomandibular joint reaction force according to any one of claims 1 to 8.

Images