Artificial intelligence-based knee joint state identification method and system

Abstract

The invention discloses a knee bone joint state recognition method and system based on artificial intelligence, and relates to the technical field of image analysis, and the method comprises the steps: obtaining multi-source medical image data of a knee joint region of a patient, carrying out the spatial registration and standardization preprocessing, and constructing a three-dimensional structure model of a knee joint after unifying a coordinate system; establishing a mapping relation between joint movement and an anatomical structure by combining knee vibration and behavior characteristic data collected by a patient in a specific movement state, extracting joint key geometric and mechanical parameters from the three-dimensional structure model, and fusing the joint key geometric and mechanical parameters with synchronously collected vibration behavior characteristics; and performing state recognition on the fused multi-source features based on an intelligent recognition model, and outputting a health state label of the knee joint. According to the invention, more accurate and more comprehensive knee joint health state identification can be realized, the accuracy and reliability of an identification result are effectively improved, and the timeliness of disease discovery and the scientificity of intervention are enhanced.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image analysis, in particular to a knee joint state recognition method and system based on artificial intelligence. BACKGROUND

[0002] With the development of medical image acquisition technology and intelligent recognition method, the early identification of knee joint disease gradually evolves towards precision and intelligence. At present, MRI and CT have high resolution and clinical value in the detection of structural lesions such as cartilage degeneration and osteophyte formation, and can provide structural basis information for diseases such as KOA. At the same time, with the help of wearable devices such as inertial measurement unit (IMU), foot pressure pad and joint vibration sensor (VAG), clinicians can collect gait, force, vibration and other biomechanical data of patients in dynamic state to reflect their joint movement performance and stability. In the recognition method, artificial intelligence models such as convolutional neural network (CNN), time convolution network (TCN) and attention mechanism model have been widely used in two-dimensional image recognition and behavior data analysis, which has improved the objectivity and automation level of knee joint health assessment. Related research has begun to try multi-modal fusion and three-dimensional reconstruction method, realizing the comprehensive modeling from image anatomy structure to behavior dynamics, providing technical support for digital orthopedics, individualized treatment and remote rehabilitation.

[0003] However, the existing knee joint state recognition method mostly stays in the two-dimensional image or local three-dimensional modeling stage, and fails to build a complete three-dimensional structure model of the knee joint under a unified registration coordinate system, resulting in a lack of spatial correspondence between image data and behavior data, and failing to realize effective mapping between structure and behavior. Secondly, the existing recognition model mostly relies on single-source static features or low-dimensional behavior indicators, ignoring the coupling relationship between vibration signals and local structural mechanical response, and failing to accurately capture the local degradation trend under dynamic load, which limits the sensitivity and interpretability of state recognition. These problems make it difficult for the current method to balance the structural authenticity and dynamic response consistency, and the recognition result has limitations such as one-sidedness and dependence on artificial experience. SUMMARY

[0004] In view of the above existing problems, the present application is proposed.

[0005] Therefore, the present application provides a knee joint state recognition method and system based on artificial intelligence, which solves the problems of lack of unified spatial reference in image and behavior data fusion and insufficient coupling modeling capability of structural response and dynamic behavior in existing knee joint recognition technology.

[0006] To solve the above technical problems, the present application provides the following technical solutions:

[0007] In a first aspect, the present application provides a knee joint state recognition method based on artificial intelligence, which comprises,

[0008] Multi-source medical image data of a patient's knee joint region are acquired, spatial registration and standardization preprocessing are performed, and a three-dimensional structure model of the knee joint is constructed after unifying the coordinate system;

[0009] Combining knee vibration and behavior feature data collected from the patient under a specific motion state, a mapping relationship between joint motion and anatomical structure is established, key geometric and mechanical parameters of the joint are extracted from the three-dimensional structure model, and are fused with the synchronously collected vibration behavior features;

[0010] Based on the intelligent recognition model, the fused multi-source features are subjected to state recognition, and a health state label of the knee joint is outputted, the recognition results are summarized to generate a standard report, and are synchronously uploaded to a remote management platform.

[0011] As a preferred scheme of the knee joint state recognition method based on artificial intelligence, wherein: the three-dimensional structure model of the knee joint constructed after unifying the coordinate system refers to that, under the unified registration space, the aligned MRI image and CT image constitute a multi-modal input sample, the structural features of the main tissues of the knee joint are extracted through the trained double-encoder U-Net structure, high-precision semantic segmentation is realized, and a structured mask map is outputted, the main tissues of the knee joint include femur, tibia, cartilage and meniscus;

[0012] The three-dimensional mask map corresponding to each type of tissue is subjected to boundary smoothing processing, the smoothed mask map is subjected to region growing operation to fill the defects in the structure, and the mask of each type of tissue is subjected to three-dimensional morphological closing operation, the repaired mask maps of various types of tissues are recombined into an optimized mask map with four channels, and each channel represents the spatial distribution structure of femur, tibia, cartilage and meniscus respectively;

[0013] The Marching Cubes algorithm is used to perform three-dimensional modeling on the mask map of each type of tissue, four triangular facet mesh models are generated, and the corresponding relationship between the triangular mesh and the original image voxel is established, and the voxel index mapping table of each mesh point in the image is obtained.

[0014] As a preferred scheme of the knee joint state recognition method based on artificial intelligence, wherein: the combination of the knee vibration and behavior feature data collected from the patient under a specific motion state, and the establishment of the mapping relationship between the joint motion and the anatomical structure comprise:

[0015] A complete three-dimensional structure model M is generated based on four triangular facet mesh models, a boundary surface index map is constructed synchronously, a tetrahedral element is discretized to the three-dimensional structure model M using a Delaunay subdivision algorithm to generate an initial finite element mesh, a cartilage region is identified by organizing structure annotation information, local mesh encryption processing is performed, and a unified tissue response annotation map Z is constructed according to the spatial position relationship of each structure in the medical image, so that each grid element is given a corresponding tissue mechanical property label in a spatial index manner;

[0016] According to the unified registered T2 magnetic resonance image sequence, combined with the voxel index mapping table, the spatial position of each grid node in the three-dimensional structure is accurately mapped to the voxel coordinates on the original T2 image, and the T2 relaxation time value T2(x) of each grid node corresponding position is extracted;

[0017] An empirical estimation formula established by regression analysis in advance is used to calculate the initial tissue response ability Q of each point. If there is no effective T2 image, the patient's basic information will be automatically called to interpolate and estimate the missing T2 value. A response parameter set corresponding to the grid node is constructed, each point records the corresponding tissue response ability value, and the tissue type label corresponding to the data structure is consistent, so as to form a tissue response parameter atlas. The initial physiological response ability parameter corresponding to each grid node is converted into the material property value required for finite element solving, and a material parameter set corresponding to the structure node position is constructed. Taking the three-dimensional structure model M as the carrier, the core information is fused, and a unified structure-behavior coupling input data set is constructed;

[0018] The sensor data from the IMU and the plantar pressure pad are time-aligned and data-cleaned. Based on the IMU data installed on the proximal end of the patient's knee, combined with the body biological parameters, the equivalent external force and external moment at the knee joint are inversely calculated according to the classical rigid body dynamics formula;

[0019] The structure-behavior coupling input data set is called, the node metadata is extracted, the key stress action area is identified, and the node set with the tissue type label of the tibial surface cartilage area is selected as the standard force application area node set of the external force input. For each sampling time point t, the knee joint external force F(t) and external moment M(t) obtained by inversion are applied to the standard force application area node set, a node neighborhood weighted loading strategy is adopted, and spatial consistency verification is performed by comparing with the gait posture sequence and the bottom pressure center;

[0020] The acceleration sensor arranged on the surface of the patient's patella records the vibration signal corresponding to each time point, representing the overall vibration performance of the joint during movement, and the vibration performance and structure response are mapped to obtain a one-to-one correspondence matrix of the structure point response vibration characteristics.

[0021] As a preferred scheme of the knee joint state recognition method based on artificial intelligence, wherein: the joint key geometric and mechanical parameters are extracted from the three-dimensional structure model, and the vibration behavior characteristics are fused, the behavior response key structure region node set is extracted from the marked acceleration sensor node group and its neighborhood region, which is the starting point of the joint key geometric and mechanical parameter analysis, a plurality of geometric shape characteristics are calculated based on the node set, the vibration characteristics are calculated based on the structure point response The vibration characteristics are correspondingly matched with the matrix, the dynamic mechanical characteristics of the corresponding nodes at each time point are extracted, and the aligned acceleration sensor signal characteristic data in the matrix is called to fuse the static geometric characteristics, the dynamic mechanical characteristics and the acceleration sensor signal characteristics into a structure-behavior joint input vector, and finally a structure-behavior fusion feature matrix is generated.

[0022] As a preferred scheme of the knee joint state recognition method based on artificial intelligence, wherein: the state recognition is performed on the fused multi-source characteristics based on the intelligent recognition model, and the health state label of the knee joint is outputted, which means that a multi-channel time sequence convolution network combined with a channel attention mechanism is used as the model backbone architecture, the model is trained using the patient's historical fusion characteristics to obtain a trained model, the structure-behavior fusion feature matrix is used as the input, and the health state prediction label at each time is outputted, and the dominant label in the time dimension is counted as the final diagnosis result.

[0023] As a preferred scheme of the knee joint state recognition method based on artificial intelligence, wherein: the recognition result is summarized to generate a standard report, which means that the knee joint health state label obtained by the model recognition is summarized together with the patient's basic information and the structure-behavior feature analysis result to generate a standardized diagnosis report.

[0024] As a preferred scheme of the knee joint state recognition method based on artificial intelligence, wherein: the multi-source medical image data of the patient's knee joint region is obtained, which means that the soft tissue and bone structure images of the knee joint are respectively acquired by MRI and CT scanning, and recorded as MRI original image set and CT original image set, the MRI simultaneously contains T2 Mapping data, during the specified open-chain and closed-chain knee joint activities of the patient, the IMU sensor is used to record the angular velocity and linear acceleration data, the foot pressure insole is used to collect the foot reaction force, and the VAG acceleration sensor is used to synchronously collect the vibration signal to form complete dynamic mechanical data.

[0025] In a second aspect, the application provides a knee joint state recognition system based on artificial intelligence, which comprises,

[0026] A multi-source data acquisition module is configured to acquire MRI and CT image data and IMU and VAG sensor multi-source dynamic behavior data and construct a structure and behavior unified input data basis;

[0027] A three-dimensional structure modeling module is configured to perform semantic segmentation and three-dimensional modeling on the registered MRI and CT images to generate a knee joint multi-tissue structure grid model in a unified coordinate system;

[0028] A structure-behavior coupling analysis module is configured to perform coupling analysis on the behavior data of the patient and the structure model, construct a structure point response and vibration feature correspondence matrix, and generate a structure-behavior fusion feature matrix;

[0029] An intelligent identification and health assessment module is configured to perform state identification on the structure-behavior fusion feature matrix based on a multi-channel time sequence convolution network model and output a health state label at each time point.

[0030] A report generation module is configured to generate a standard diagnosis report and synchronously upload the report to a remote health management platform.

[0031] In a third aspect, the present application provides a computer device comprising a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, any step of the artificial intelligence-based knee joint state identification method according to the first aspect of the present application is implemented.

[0032] In a fourth aspect, the present application provides a computer readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, any step of the artificial intelligence-based knee joint state identification method according to the first aspect of the present application is implemented.

[0033] The present application has the following beneficial effects: the present application constructs a complete three-dimensional structure model of a knee joint in a unified coordinate system and fuses multi-source behavior data acquired under a specific motion state, effectively solving the problem of lack of spatial consistency between image data and dynamic behavior data in the prior art, significantly improving the accuracy of structure-behavior coupling modeling, relying on a one-to-one mapping mechanism of structure point response and vibration features, establishing a closed loop correlation between dynamic mechanical response and actual behavior features, effectively overcoming the problem that single static features or low-dimensional behavior features in traditional methods cannot fully reflect joint functional degeneration, and combining a multi-channel time sequence convolution network and an attention mechanism, the present application can realize fine identification of high-dimensional features and accurate determination of the health state of a knee joint, improving the sensitivity and interpretability of the model in early lesion identification. BRIEF DESCRIPTION OF DRAWINGS

[0034] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings needed to be used in the embodiments will be briefly introduced as follows. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative effort on the basis of these drawings.

[0035] Fig. 1 Flowchart of the knee joint state recognition method based on artificial intelligence in Embodiment 1.

[0036] Fig. 2 Structure diagram of the knee joint state recognition system based on artificial intelligence in Embodiment 1.

[0037] Fig. 3 Flowchart of three-dimensional structure modeling and mesh generation in Embodiment 1.

[0038] Fig. 4 Flowchart of structure-behavior coupling and feature fusion in Embodiment 1. DETAILED DESCRIPTION

[0039] In order to make the above objectives, features and advantages of the present application more apparent and comprehensible, the specific embodiments of the present application will be described in detail below with reference to the accompanying drawings.

[0040] In the following description, many specific details are set forth in order to provide a thorough understanding of the present application. However, the present application can be practiced in other ways different from those described herein without departing from the scope of the present application, and those skilled in the art can make similar generalizations without departing from the scope of the present application. Therefore, the present application is not limited to the specific embodiments disclosed below.

[0041] Secondly, the "one embodiment" or "embodiment" referred to herein means that the specific features, structures or characteristics can be included in at least one implementation of the present application. "In one embodiment" appearing in different places in the specification does not mean the same embodiment, nor is it an independent or alternative embodiment that excludes other embodiments.

[0042] Embodiment 1, refer to Figs. 1-4 , the first embodiment of the present application, the embodiment provides a knee joint state recognition method based on artificial intelligence, comprising the following steps:

[0043] S1, acquiring multi-source medical image data of the knee joint region of the patient, and performing spatial registration and standardization preprocessing, constructing a three-dimensional structure model of the knee joint after unifying the coordinate system;

[0044] Specifically, multi-source medical image data of the knee joint region of a patient is acquired, and spatial registration and standardization preprocessing are performed. The soft tissue and bone structure images of the knee joint are collected by MRI (including T2 Mapping) and CT scanning respectively, and recorded as the MRI original image set and the CT original image set. During the specified open-chain (such as leg lifting) and closed-chain (such as deep squatting) knee joint movement of the patient, the IMU sensor is used to record the angular velocity and linear acceleration data, the foot pressure insole is used to collect the foot reaction force, and the VAG acceleration sensor is used to synchronously collect the vibration signal, forming complete dynamic mechanical data.

[0045] T2 Mapping means that in addition to the conventional MRI sequence, a special quantitative imaging technique called T2 Mapping is included in the MRI (magnetic resonance imaging) examination. In MRI, T2 is a physical property of the tissue, called transverse relaxation time, which reflects the speed of the magnetization vector of water molecules in the tissue in the transverse plane (perpendicular to the main magnetic field direction) after being excited by a radio frequency pulse. Different soft tissues (such as normal articular cartilage, damaged cartilage, ligaments, meniscus, etc.) have different T2 values. For example, healthy articular cartilage has low T2 value due to its ordered arrangement of water content and collagen fibers, while when the cartilage is in early degeneration or damage, the water content increases and the structure is disordered, and the T2 value increases.

[0046] T2 mapping is a quantitative MRI technique that does not provide contrast images like conventional MRI, but by acquiring multiple images with different echo times (TE), then calculating each pixel point to generate a pixel-level T2 value distribution map, on which different colors or gray scales represent different T2 values (usually in milliseconds ms), doctors or researchers can objectively and quantitatively evaluate the biochemical composition and microscopic structure changes of soft tissues by observing the changes of T2 values.

[0047] To ensure the synchronization and spatial unity of multi-source data, all devices are connected to a unified time synchronization module, and all sensor outputs are processed by a unified coordinate conversion matrix to complete the spatio-temporal consistency alignment and form a standardized multi-modal data set.

[0048] Non-local mean filtering is performed on the MRI images to reduce noise, histogram equalization and Gamma correction are performed on the CT images to enhance the clarity of bone tissue, and a rigid registration algorithm based on mutual information maximization is used to register the MRI to the CT coordinate system. Then, a free deformation algorithm is used for fine registration of the cartilage region, and finally a standardized image data pair in a unified space is obtained.

[0049] The application establishes a unified standardized data set of structure image and dynamic behavior by collecting MRI (including T2 Mapping), CT, multi-source sensor (IMU, foot pressure pad, VAG) and other data of the knee joint area, enhances the spatial accuracy and time sequence consistency of state recognition, T2 Mapping provides physiological markers of cartilage health, CT enhances bone structure recognition, behavior data reflects actual stress and vibration response, through multi-modal fusion, an artificial intelligence model can comprehensively understand the structure-function relationship, realize more accurate and early recognition of knee joint degeneration, and effectively improve the scientificity, reliability and clinical application value of the recognition result.

[0050] Further, constructing a three-dimensional structure model of the knee joint after unifying the coordinate system comprises: under the unified registration space, the aligned MRI image and the CT image constitute a multi-modal input sample, the structure features of the main tissues of the knee joint are extracted through the trained double-encoder U-Net structure, high-precision semantic segmentation is realized, and a structured mask image is output.

[0051] The main tissues of the knee joint include femur, tibia, cartilage and meniscus.

[0052] The registered MRI image (which has been subjected to non-local mean filtering denoising) and the CT image (which has been subjected to histogram equalization and Gamma correction enhancement) are correspondingly paired according to position and voxel, to form a double-channel image sample, and the sample structure is a point-by-point alignment of three-dimensional MRI images and CT images at the voxel level.

[0053] In order to improve the training and inference stability of the neural network, standardization processing is performed on the two image channels, specifically, the pixel value of each image channel is subtracted by the mean value of all pixels in the channel, and then divided by the standard deviation of the channel pixels, to realize the normalized distribution with mean value of 0 and variance of 1;

[0054] The standardized MRI and CT images are stacked according to the channel dimension to form a four-dimensional tensor, the dimension of which is (height x width x slice number x channel number), and the channel number is 2, corresponding to the MRI and CT image channels respectively;

[0055] The dual-encoder U-Net model is used as the backbone neural network structure, which is composed of two parallel encoders and a shared decoder. The first encoder is used to process MRI images, focusing on extracting texture and morphological features of soft tissues (especially cartilage and meniscus). The second encoder processes CT images, focusing on extracting density and edge features of bony structures (such as femur and tibia). After each encoder extracts multi-scale features, the feature maps are spliced in the middle fusion layer, and the attention mechanism module (such as SE module or CBAM module) is used to selectively emphasize the channel features useful for the final segmentation. The decoder of the U-Net performs step-by-step upsampling and splicing of the fused feature maps and the corresponding encoder shallow features, and finally outputs a semantic segmentation result with the same size and resolution as the input image.

[0056] The training data of the dual-encoder U-Net model uses MRI and CT registered images in the OAI (Osteoarthritis Initiative) or similar medical image public dataset, and uses manually labeled tissue structures as supervised labels. The labeled structures include four types of tissues: femur, tibia, cartilage, and meniscus. During training, the cross-entropy loss function is combined with the Dice coefficient loss function for optimization. The latter is used to handle the imbalance between classes, which encourages the model to improve the recognition accuracy of small volume structures (such as meniscus). The Adam or AdamW optimizer is used during training, and the learning rate is recommended to be set to 0.0001 initially. The batch size is set according to the memory capacity (usually 4 or 8), and the training rounds are recommended to be between 50 and 100. During training, data augmentation strategies such as affine transformation, mirror flipping, or Gaussian noise injection can be applied to enhance the model's generalization ability. After training is complete, the model weights are saved for subsequent patient data inference. The training data and patient data are not cross-used to avoid overfitting or data leakage risks.

[0057] After the model training is completed and the weight parameters are fixed, the constructed registered image tensor can be input into the trained dual-encoder U-Net model for forward inference. The model automatically identifies the main anatomical structures of the knee joint region and outputs a semantic segmentation map. The output segmentation map is a four-channel mask map, each channel representing the voxel position of the femur, tibia, cartilage, and meniscus. The pixel value is 0 or 1, indicating whether the voxel belongs to the tissue category. The final semantic mask map maintains the same spatial coordinates and resolution as the original image, providing a geometric basis for direct participation in three-dimensional structure modeling.

[0058] For each type of tissue, including femur, tibia, cartilage and meniscus, the corresponding three-dimensional mask map is subjected to boundary smoothing processing, and the embodiment adopts a three-dimensional Gaussian filtering algorithm, that is, for each voxel position in the mask map, the voxel values in the neighborhood range are weighted and averaged according to the Gaussian distribution, the continuity of the boundary is strengthened, and the sharp corners are eliminated;

[0059] The region growing operation is performed on the smoothed mask map to fill the defects on the structure, and the specific method is as follows: in the mask of each type of tissue, the center of the maximum intensity connected region is selected as the seed point, a growth threshold is set, the threshold is generally plus or minus 10% of the average gray value of the seed region, and the seed point is expanded outward in the three-dimensional voxel map, and the adjacent voxels similar to the seed value are added until the expansion cannot continue. This method can effectively close the incomplete edge structure or hollow area. The three-dimensional morphological closing operation is performed on each type of tissue mask, that is, the structure is first expanded and then eroded. The expansion operation is used to fill small holes, and the erosion is used to restore the overall boundary profile. The morphological closing operation can further enhance the structural integrity of the segmented region;

[0060] The repaired mask maps of each type of tissue are recombined into an optimized four-channel mask map, and each channel represents the spatial distribution structure of femur, tibia, cartilage and meniscus respectively;

[0061] The Marching Cubes algorithm is used to perform three-dimensional modeling on the mask map of each type of tissue. The algorithm is a classic surface reconstruction method and is widely used in medical image processing. Its working principle is to find the local isosurface in the three-dimensional binary voxel map, and form a continuous surface mesh by splicing triangular facets. Here, for each type of tissue, the voxel value of 1 indicates the presence of tissue, and the value of 0 indicates no tissue. In this embodiment, the reconstruction isosurface threshold is set to 0.5 based on experience. For the four types of tissue, femur, tibia, cartilage and meniscus, the algorithm is executed respectively to generate four triangular facet mesh models. Each mesh model is composed of a large number of triangular facets, describing the external surface shape of the tissue;

[0062] At the same time of generating the three-dimensional mesh, the corresponding relationship between the triangular mesh and the original image voxels needs to be established. For this purpose, the three-dimensional physical coordinates of each triangular facet vertex are converted into voxel index positions according to the voxel resolution and coordinate origin of the original image. For example, if the image resolution is 0.5mm per voxel in X, Y and Z directions, and the image space origin is (0, 0, 0), then the three-dimensional point (10mm, 20mm, 30mm) corresponds to the voxel index (20, 40, 60). In this way, a voxel index mapping table of each mesh point in the image is established. The four types of three-dimensional models generated and their corresponding voxel index mapping tables are packaged to form a unified data structure.

[0063] Combining the advantages of MRI in soft tissue imaging and CT in bone tissue imaging, the modeling accuracy of key tissues such as femur, tibia, cartilage and meniscus is significantly improved; the dual-encoder U-Net structure and attention mechanism are used to enhance the semantic segmentation performance, effectively improving the identification accuracy of small volume tissues (such as meniscus); the multi-level image post-processing process (such as three-dimensional Gaussian filtering, region growing and morphological closing operation) is introduced to ensure the continuity of the model boundary and the integrity of the structure; finally, through Marching Cubes modeling and voxel mapping table construction, high-precision correspondence between image information and three-dimensional grid model is realized, laying a solid foundation for subsequent mechanical simulation, structure-behavior mapping and health status recognition.

[0064] S2, combining the knee vibration and behavior characteristic data collected from the patient in a specific motion state, a mapping relationship between joint motion and anatomical structure is established, key geometric and mechanical parameters of the joint are extracted from the three-dimensional structure model, and are fused with the simultaneously collected vibration behavior characteristics;

[0065] Specifically, combining the knee vibration and behavior characteristic data collected from the patient in a specific motion state, a mapping relationship between joint motion and anatomical structure is established, including:

[0066] Topological repair operations are performed on the triangular facet models of each tissue, including deleting duplicate faces, closing open boundaries, eliminating non-manifold structures, etc., to ensure that each tissue model is a closed surface body. The contact area between the cartilage and the hard tissues such as femur and tibia is detected by a registration algorithm, and a shared boundary is generated on the contact interface, so that there is no overlap or gap between adjacent tissues, forming a spatially continuous connection relationship. A unique tissue label code is assigned to each triangular facet, and a mapping table from facet to voxel coordinates is established to achieve a one-to-one correspondence between each surface grid element and the original MRI / CT image data in spatial position. The repaired triangular facet models of each tissue are combined into a complete three-dimensional structure model M, and a boundary surface index map is simultaneously constructed to identify the contact interface between tissues and its topological relationship. This combined model maintains clear and distinct tissue boundaries while having overall spatial closure and continuity;

[0067] Delaunay tetrahedral subdivision algorithm is used to perform volume grid subdivision on the surface of each tissue structure in the three-dimensional structure model M, converting the original voxel-level surface representation into high-quality tetrahedral finite element grid structure. This algorithm uses the maximum and minimum angle optimization principle to ensure the numerical stability of the grid and the convergence of subsequent calculations. After subdivision, to ensure the calculation accuracy of key structure areas in stress simulation, the cartilage area is identified through tissue structure labeling information, and local grid encryption processing is performed with an encryption ratio of three times that of other areas to ensure accurate capture of local strain and vibration response characteristics in this area during subsequent stress calculation.

[0068] According to the spatial position relationship of each structure in the medical image, a unified tissue response label map Z is constructed, and each grid cell is given a corresponding tissue mechanical property label in a spatial indexing manner, including different types of structure response classifications such as bone tissue, articular cartilage, meniscus, etc.

[0069] According to the unified registration T2 magnetic resonance image sequence, combined with the voxel index mapping table, the spatial position of each grid node in the three-dimensional structure is accurately mapped to the voxel coordinates on the original T2 image, and the T2 relaxation time value T2(x) of the corresponding position of each grid node is extracted, which reflects the water content and collagen fiber arrangement state of the tissue position, and is one of the important parameters for reflecting the health status of cartilage in the clinic;

[0070] In order to convert the image physiological information provided by the T2 image into the quantitative parameters required for structure response modeling, an empirical estimation formula established by regression analysis in advance is used to calculate the initial tissue response ability Q of each point, which is specifically:

[0071] The T2 value of each node is input into the empirical estimation model:

[0072]

[0073] In the formula, T2(x) is the MRI T2 relaxation time at point x, and a and b are empirical parameters fitted from clinical statistical data, which are used to model the mapping relationship between different tissue T2 values and their response capabilities;

[0074] If the node has no effective T2 image (such as boundary missing or artifact area), the patient's basic information (such as age, body mass index, gender, etc.) will be automatically called, and a linear regression relationship will be constructed using the existing training model to estimate the missing T2 value, so as to ensure the continuity and integrity of the tissue response parameters in the spatial dimension. After the response capability estimation of all nodes with effective T2 values is completed, a response parameter set corresponding to the grid nodes will be constructed, each point will record its tissue response capability value, and the tissue response capability value will be consistent with its corresponding tissue type label in the data structure, thereby forming a tissue response parameter atlas;

[0075] According to the tissue response atlas T(x) obtained by T2 magnetic resonance image inversion, the initial physiological response capability parameters (T2 relaxation time) corresponding to each grid node are converted into material property values required for finite element solving:

[0076] For tissue regions covered by T2 data such as cartilage, a pre-trained linear regression estimation model is used to map T2(x) to material parameters such as elastic modulus E(x), damping coefficient D(x), Poisson's ratio v(x), etc.

[0077] For the boundary or artifact area missing T2 data, the individual parameters of the patient (such as age, gender, BMI) are substituted into the regression model to complete the response capacity estimation, ensuring the continuity and closure of the material field in the full spatial dimension, completing the assignment of the multi-parameter material attribute field corresponding to the spatial position x of each grid node, and constructing a material parameter set corresponding to the structure node position one by one, taking the structure model M as the carrier, integrating the core information, and constructing a unified structure-behavior coupling input data set:

[0078] The spatial coordinate position of each node;

[0079] The tissue type label to which the node belongs;

[0080] The material attribute values E(x), v(x), D(x), etc. corresponding to the node;

[0081] To ensure that multi-source behavior data can be used for mechanical modeling, first, the sensor data from the IMU and the plantar pressure pad need to be time-aligned and data-cleaned, and the specific process is as follows:

[0082] Set a uniform sampling period (such as sampling once every 10 milliseconds);

[0083] Perform linear interpolation on IMU data (acceleration, angular velocity) to align it with GRF (plantar vertical and horizontal pressure) time axis, use a noise filter (such as a fifth-order Butterworth low-pass filter) to remove high-frequency interference, ensure the physical continuity of the motion trajectory and the contact force curve, and convert all data to a unified coordinate system to maintain consistency with the three-dimensional structure model space;

[0084] After data preprocessing, based on the IMU data installed on the proximal end of the patient's knee, combined with the body biological parameters (including the equivalent mass, moment of inertia, and segment length of the tibiofibular segment), the equivalent external force and external moment at the knee joint are inversely calculated according to the classical rigid body dynamics formula:

[0085] F(t) = m x a(t), M(t) = I x dω(t) / dt

[0086] Where a(t) is the linear acceleration measured by the sensor, ω(t) is the angular velocity, dω / dt represents the angular acceleration, m and I are the mass and moment of inertia of the limb segment, which are estimated by the patient's height and weight through the biomechanical model, and F(t) and M(t) are the equivalent external force and external moment acting on the proximal reference point of the knee joint (the midpoint of the upper edge of the tibial plateau) at each time;

[0087] To ensure that the external force and moment obtained by the motion behavior data inversion can be accurately loaded into the three-dimensional structure model, and to avoid local stress abnormalities, the standard force application area and boundary loading strategy need to be clearly defined in the model. The operation process is as follows: call the structure-behavior coupling input data set, extract the node metadata containing the node spatial coordinates, tissue label and material attribute information, identify the key force action area, according to the anatomical positioning standard in sports biomechanics, select the node set of the tissue type label "tibial surface cartilage" area as the standard force application area node set of the external force input, for each sampling time point t, apply the inversion obtained knee joint external force F(t) and external moment M(t) to the standard force application area node set, in order to avoid local stress singularity caused by single node loading point load, the node neighborhood weighted loading strategy is adopted, that is, according to the geometric distance, grid area or volume between nodes, the weight is divided, the external force is equivalent to the target node and its neighborhood node, forming a small range of continuous loading area, so as to improve the numerical stability and physiological rationality of the boundary condition, all loading paths and the unified coordinate system of the structure model are strictly aligned, and the spatial consistency is verified by comparing with the gait posture sequence and the bottom pressure center, to ensure the physical effectiveness of the boundary input;

[0088] At the same time, the acceleration sensor (VAG) arranged on the surface of the patient's patellar region records the vibration signal corresponding to each time point, representing the overall vibration performance of the joint during movement, in order to establish the mapping between vibration performance and structure response, the following operations are performed:

[0089] Call the structure-behavior coupling input data set, locate the structure node group closest to the spatial position of the VAG sensor installation point, and confirm that its tissue label is "patellar surface cartilage" or "patellar edge cartilage", to ensure that the vibration response it reflects has anatomical significance;

[0090] For each sampling time point t, construct the structure response vector S(t) = [maximum stress, maximum strain, main frequency, stress change rate, etc.], and construct the vibration observation vector V(t) = [acceleration peak value, RMS, main frequency, etc.];

[0091] Accurately align the structure response vector and the vibration observation vector on the time axis, and establish a one-to-one correspondence matrix of "structure point response Vibration characteristics".

[0092] This invention constructs a high-dimensional input system that couples structure and behavior by integrating 3D structural modeling, MRI physiological indicators, motion behavior data, and vibration signals, achieving a precise mapping between joint motion and anatomical structure. This method enhances the anatomical continuity of the model through topology repair and shared boundary construction, ensures the accuracy of stress simulation through Delaunay subdivision and local mesh refinement, improves physiological realism by inverting material properties using T2 images, introduces IMU and plantar pressure data to invert external forces and moments, and employs a multi-node weighted loading method to ensure physiological rationality of boundary conditions. By aligning VAG signals with structural responses over time, a "structural response ↔ vibration feature" mapping matrix is ​​established, effectively improving the physical interpretability of the features. The final constructed structure-behavior fusion feature matrix provides a temporally consistent, high-dimensional, and accurate input foundation for the state recognition model, significantly improving the accuracy, interpretability, and clinical applicability of knee joint health status recognition.

[0093] Furthermore, key geometric and mechanical parameters of the joint are extracted from the 3D structural model and fused with synchronously acquired vibration behavior features. Specifically, a set of key structural region nodes for behavioral response is extracted from the marked VAG node group and its neighborhood, serving as the starting point for the analysis of key geometric and mechanical parameters of the joint. Based on the node set, multiple geometric morphological features are calculated, including Gaussian curvature, mean curvature, local shape index, normal vector deviation, principal axis length to surface area-volume ratio, to accurately characterize the local structural morphology of the joint. After completing the static geometric feature extraction, the structural point response is then analyzed. A vibration feature-to-one correspondence matrix is ​​generated to extract the dynamic mechanical features of the corresponding nodes at each time point, including maximum principal stress, equivalent stress, maximum displacement amplitude, stress response frequency, local damping response index, and material heterogeneity index. Aligned VAG signal feature data (such as peak acceleration, root mean square value, dominant frequency, kurtosis, skewness, etc.) in the matrix are also called to ensure a one-to-one correspondence between each structural response and behavioral feature in time and space. Static geometric features, dynamic mechanical features, and VAG signal features are fused into a structure-behavior joint input vector Z(t). All time slices t are concatenated to finally generate a structure-behavior fusion feature matrix. This realizes a complete closed-loop input construction process that starts from structural morphology and material response and integrates actual behavioral vibration performance, providing a high-dimensional, time-consistent, and accurate feature foundation for subsequent model training and state recognition.

[0094] By fusing the three-dimensional structure model and the VAG behavior vibration data, a structure-behavior fusion feature matrix is constructed, which significantly improves the accuracy and clinical adaptability of the knee joint state recognition. On the one hand, the static geometric features comprehensively depict the local joint shape changes, which are convenient for identifying structural lesions. On the other hand, the dynamic mechanical features reflect the stress and deformation state in the actual movement process, which can capture hidden mechanical abnormalities. Through one-to-one mapping of structure response and vibration features, the deep coupling of multi-modal data in time and space is realized, the joint input vector constructed has high dimension, strong representation and time sequence consistency, which greatly enhances the sensitivity and generalization ability of the downstream recognition model to early degeneration or mild abnormalities, and improves the practicability and stability of the intelligent recognition system.

[0095] S3, based on the intelligent recognition model, the fused multi-source features are recognized to output the health state label of the knee joint, the recognition results are summarized to generate a standard report, and the report is uploaded to a remote management platform;

[0096] Specifically, based on the intelligent recognition model, the fused multi-source features are recognized to output the health state label of the knee joint. A multi-channel time convolution network combined with a channel attention mechanism is used as the model backbone architecture, including an input layer, a time convolution module (1D CNN), a channel attention fusion (SE-Block / CBAM), and a fully connected layer + softmax classification head. The structure geometry flow, material attribute flow, and vibration behavior flow are processed respectively, and the channel weighted fusion is performed through the attention mechanism in the middle layer to output the standard knee joint health state label (such as KL classification 0-4), as shown in Table 1:

[0097] Table 1: Example table of knee joint health state label

[0098]

[0099] In the model training phase, the patient's historical fusion features are used as input, the cross-entropy loss function is combined with the class balance factor to construct the training target, the Adam optimizer is used for parameter update, and the cross-validation and early stopping mechanism are used to avoid overfitting. To improve the generalization ability of the model on time series data, data enhancement operations such as window slicing and time axis disturbance are performed on the input data before training. After training, the weight parameter file is output, and the accuracy, recall rate and F1 score of the model on the validation set are evaluated to ensure that the recognition performance meets the standard, and the trained model is obtained;

[0100] Using the structure-behavior fusion feature matrix as input, the health state prediction label at each time is output, and the dominant label in the time dimension is counted as the final diagnosis result (the dominant label refers to the most number of classes in a specified time period).

[0101] By fusing the multi-modal feature flow of structure and behavior, and introducing a channel attention mechanism, an intelligent recognition model with time series modeling capability and modal differentiation capability is constructed, which can ensure the recognition accuracy while realizing the automatic output of the KL standard health state. Its core advantages are: 1) fusion of structure + behavior dual-domain signals to enhance the robustness of degeneration detection; 2) introduction of attention mechanism to optimize multi-channel fusion and improve the recognition ability of mild and moderate lesions; 3) the recognition label has medical standardization comparability, which is convenient for subsequent intervention decision and follow-up management.

[0102] Further, the recognition results are summarized to generate a standard report, and are uploaded to a remote management platform. The knee joint health status label obtained by the model recognition, patient basic information, and structure-behavior feature analysis results are summarized together to automatically generate a standardized diagnosis report, and are uploaded to a remote management platform in PDF and structured data dual formats, realizing data archiving, visual display, and subsequent retrieval support.

[0103] The embodiment also provides a knee joint state recognition system based on artificial intelligence, comprising:

[0104] A multi-source data acquisition module is configured to acquire MRI and CT image data and IMU and VAG sensor multi-source dynamic behavior data, and construct a unified input data basis of structure and behavior;

[0105] A three-dimensional structure modeling module is configured to perform semantic segmentation and three-dimensional modeling on the registered MRI and CT images, and generate a knee joint multi-tissue structure grid model in a unified coordinate system;

[0106] A structure-behavior coupling analysis module is configured to perform coupling analysis on the behavior data and the structure model of the patient, construct a corresponding matrix of structure point response and vibration characteristics, and generate a structure-behavior fusion feature matrix;

[0107] An intelligent recognition and health assessment module is configured to perform state recognition on the structure-behavior fusion feature matrix based on a multi-channel time series convolution network model, and output a health status label at each time point;

[0108] A report generation module is configured to generate a standard diagnosis report and synchronously upload it to a remote health management platform.

[0109] The embodiment also provides a computer device suitable for the knee joint state recognition method based on artificial intelligence, comprising a memory and a processor; the memory is configured to store computer executable instructions, and the processor is configured to execute the computer executable instructions to implement the knee joint state recognition method based on artificial intelligence proposed in the above embodiment.

[0110] The computer device can be a terminal, which includes a processor, a memory, a communication interface, a display screen and an input device connected by a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operating system and the computer program in the non-volatile storage medium to run. The communication interface of the computer device is used to communicate with external terminals in a wired or wireless manner. The wireless manner can be achieved by WIFI, operator network, NFC (Near Field Communication) or other technologies. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen. The input device of the computer device can be a touch layer overlaid on the display screen, or a key, trackball or touchpad arranged on the shell of the computer device, or an external keyboard, touchpad or mouse, etc.

[0111] The embodiment also provides a storage medium having a computer program stored thereon, the program being executed by a processor to implement the method for recognizing the knee joint state based on artificial intelligence proposed in the above embodiment. The storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as a static random access memory (SRAM), an electrically erasable programmable read-only memory (EEPROM), an erasable programmable read-only memory (EPROM), a programmable read-only memory (PROM), a read-only memory (ROM), a magnetic storage, a flash memory, a magnetic disk or an optical disk.

[0112] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present application but not limit the present application. Although the present application has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present application can be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present application, which should be covered in the scope of the claims of the present application.

Claims

1. A method for recognizing the state of a knee joint based on artificial intelligence, characterized in that: include, Multi-source medical image data of the patient's knee joint region were acquired, and spatial registration and standardization preprocessing were performed. After unifying the coordinate system, a three-dimensional structural model of the knee joint was constructed. By combining knee vibration and behavioral characteristic data collected from patients under specific movement states, a mapping relationship between joint movement and anatomical structure is established. Key geometric and mechanical parameters of the joint are extracted from the three-dimensional structural model and fused with the synchronously collected vibration and behavioral characteristics. Based on the intelligent recognition model, the system performs state recognition on the fused multi-source features, outputs a health status label for the knee joint, summarizes the recognition results to generate a standard report, and uploads it to the remote management platform simultaneously.

2. The knee joint state recognition method based on artificial intelligence as described in claim 1, characterized in that: The unified coordinate system is used to construct a three-dimensional structural model of the knee joint. Under the unified registration space, the aligned MRI images and CT images are used to form multimodal input samples. Through the trained dual encoder U-Net structure, the structural features of the main tissues of the knee joint are extracted to achieve high-precision semantic segmentation and output a structured mask map. The main tissues of the knee joint include the femur, tibia, cartilage and meniscus. For each type of tissue, the boundary of the 3D mask is smoothed. The smoothed mask is then subjected to a region growth operation to fill structural defects. A 3D morphological closing operation is performed on each type of tissue mask. The repaired tissue masks are then recombined into a four-channel optimized mask, with each channel representing the spatial distribution structure of the femur, tibia, cartilage, and meniscus. The Marching Cubes algorithm is used to perform 3D modeling of the mask image for each type of tissue, generating four triangular mesh models and establishing the correspondence between the triangular mesh and the voxels of the original image, thus obtaining a voxel index mapping table for each mesh point in the image.

3. The knee joint state recognition method based on artificial intelligence as described in claim 2, characterized in that: The process of establishing a mapping relationship between joint movement and anatomical structure by combining knee vibration and behavioral characteristic data collected from patients under specific movement states includes: A complete three-dimensional structural model M is generated based on four triangular mesh models. A boundary surface index map is constructed simultaneously. The three-dimensional structural model M is discretized into tetrahedral elements using the Delaunay subdivision algorithm to generate an initial finite element mesh. The cartilage region is identified through tissue structure annotation information and local mesh refinement is performed. Based on the spatial positional relationship of each structure in the medical image, a unified tissue response annotation map Z is constructed. Each mesh unit is assigned a corresponding tissue mechanical property label in a spatial index manner. Based on the uniformly registered T2 magnetic resonance image sequence, combined with the voxel index mapping table, the spatial position of each grid node in the three-dimensional structure is accurately mapped to the voxel coordinates on the original T2 image, and the T2 relaxation time value T2(x) of each grid node at the corresponding position is extracted. Using an empirical estimation formula established in advance through regression analysis, the initial tissue response capability Q of each point is calculated. If there is no valid T2 image for a node, the patient's basic human information will be automatically retrieved to interpolate and estimate the missing T2 value. A set of response parameters corresponding to each grid node will be constructed. The corresponding tissue response capability value of each point will be recorded and kept consistent with its corresponding tissue type label in the data structure, thereby forming a tissue response parameter map. The initial physiological response capability parameter corresponding to each grid node will be converted into the material property value required for finite element solution. A set of material parameters corresponding to the structural node position will be constructed. Using the three-dimensional structural model M as the carrier, core information will be integrated to construct a unified structure-behavior coupled input dataset. Time alignment and data cleaning were performed on sensor data from IMU and plantar pressure pad. Based on IMU data installed near the patient's knee, combined with somatic biological parameters, the equivalent external force and torque at the knee joint were inversely calculated using classical rigid body dynamics formulas. The structure-behavior coupled input dataset is called, node metadata is extracted, key force application areas are identified, and the set of nodes labeled as tibial cartilage region is selected as the standard force application area node set for external force input. For each sampling time point t, the inverted knee joint external force F(t) and external torque M(t) are applied to the standard force application area node set. The node neighborhood weighted loading strategy is adopted, and spatial consistency is verified by comparison with gait posture sequence and bottom pressure center. Accelerometers placed on the surface of the patient's patellar region record vibration signals at each time point, representing the overall vibration performance of the joint during movement. A mapping is established between the vibration performance and the structural response to obtain the structural point response. A one-to-one correspondence matrix of vibration characteristics.

4. The knee joint state recognition method based on artificial intelligence as described in claim 3, characterized in that: The extraction of key geometric and mechanical parameters of the joints from the 3D structural model and their fusion with synchronously acquired vibration behavior features refers to extracting a set of key structural region nodes for behavioral response from the marked accelerometer node group and its neighborhood, serving as the starting point for the analysis of key geometric and mechanical parameters of the joints. Multiple geometric morphological features are calculated based on the node set, and the structural point response is then analyzed. The vibration feature is mapped one-to-one to the matrix. The dynamic mechanical features of the corresponding node at each time point are extracted. The accelerometer signal feature data that has been aligned in the matrix are called. The static geometric features, dynamic mechanical features and accelerometer signal features are fused into a structure-behavior joint input vector, and finally the structure-behavior fusion feature matrix is ​​generated.

5. The knee joint state recognition method based on artificial intelligence as described in claim 4, characterized in that: The intelligent recognition model performs state recognition on the fused multi-source features and outputs a health status label for the knee joint. It uses a multi-channel temporal convolutional network combined with a channel attention mechanism as the backbone architecture of the model. The model is trained with the patient's historical fusion features to obtain a trained model. The structure-behavior fusion feature matrix is ​​used as input to output the predicted health status label at each time step, and the dominant label in the time dimension is statistically analyzed as the final diagnostic result.

6. The knee joint state recognition method based on artificial intelligence as described in claim 5, characterized in that: The process of summarizing the identification results to generate a standard report refers to summarizing the knee joint health status labels obtained by the model with the patient's basic information and the results of the structural-behavioral feature analysis to generate a standardized diagnostic report.

7. The knee joint state recognition method based on artificial intelligence as described in claim 6, characterized in that: The acquisition of multi-source medical image data of the patient's knee joint region refers to the acquisition of soft tissue and bony structure images of the knee joint through MRI and CT scans, respectively, and recording them as MRI raw image sets and CT raw image sets. MRI also includes T2 mapping data. During the patient's specified open-chain and closed-chain knee joint activities, IMU sensors are used to record angular velocity and linear acceleration data, plantar pressure insoles collect plantar reaction forces, and VAG accelerometers simultaneously collect vibration signals to form complete dynamic mechanical data.

8. An artificial intelligence-based knee joint state recognition system, based on the artificial intelligence-based knee joint state recognition method according to any one of claims 1 to 7, characterized in that: include, The multi-source data acquisition module is used to acquire MRI, CT image data and multi-source dynamic behavior data from IMU and VAG sensors, and to build a unified input data foundation for structure and behavior. The 3D structural modeling module is used to perform semantic segmentation and 3D modeling on the registered MRI and CT images, and generate a multi-tissue structure mesh model of the knee joint in a unified coordinate system. The structure-behavior coupling analysis module is used to couple and analyze the patient's behavioral data with the structural model, construct the correspondence matrix between structural point response and vibration characteristics, and generate a structure-behavior fusion feature matrix. The intelligent recognition and health assessment module is used to perform state recognition on the structure-behavior fusion feature matrix based on a multi-channel temporal convolutional network model, and output the health status label at each time point. The report generation module is used to generate standard diagnostic reports and upload them synchronously to the remote health management platform.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that: When the processor executes the computer program, it implements the steps of the artificial intelligence-based knee joint state recognition method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by the processor, it implements the steps of the artificial intelligence-based knee joint state recognition method according to any one of claims 1 to 7.

Images