A knee joint image information fusion method based on coordinate association and related equipment

Abstract

The application provides a knee joint image information fusion method based on coordinate association and related equipment, and is applied to the technical field of data processing. The application obtains a training data set and target user data; processes the target user data to generate attribute information of the target user; processes a knee joint sagittal sequence and a knee joint coronal proton density sequence based on knee joint sequence target coordinate information to generate a knee joint image set; pre-processes the training data set to generate a training data set with identification information; processes the training data set with identification information based on a preset processing rule to generate a training set and a validation set; obtains an initial knee joint image fusion model matched with the identification information; trains the initial knee joint image fusion model based on the training set and the validation set to generate a target knee joint image fusion model; and processes the knee joint image set to generate knee joint wear information of the target user.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, and in particular to a method and related equipment for fusion of knee joint image information based on coordinate association. Background Technology

[0002] Current knee injury detection models rely solely on single MRI sequences as input, leading to issues such as low accuracy, blurred lesion boundaries, and inconsistencies in the identification of the same lesion across different patient positions. While MRI, due to its non-invasive nature and high-quality knee imaging, has become the standard imaging method for diagnosing meniscus injuries, the thick-slice, multi-positional nature of knee MRI images makes interpretation time-consuming and complex, requiring frequent image switching and multi-angle viewing for diagnosis. Furthermore, the often localized and ambiguous nature of meniscus injuries further complicates diagnosis.

[0003] In recent years, deep learning models have far surpassed traditional image analysis methods in many fields. Their end-to-end nature enables them to automatically learn image features and collaboratively optimize with downstream tasks. However, due to the complexity of modalities and the scarcity of public datasets, there is relatively little work on deep learning in knee MRI modalities. Existing technologies for knee MRI applications are mainly limited to cartilage segmentation and cartilage injury detection, and can be divided into two types. One type uses a single sagittal or coronal sequence as input. This method cannot establish the feature relationship between coronal and sagittal images within the model, which affects classification accuracy. The other type extracts features separately from the two sequences and then directly concatenates the features of the two positions in the final classification layer for fusion. This cross-sequence information fusion is too simplistic and crude, and it loses multi-scale cross-sequence features.

[0004] It should be noted that the information disclosed in the background section above is only used to enhance the understanding of the background of this disclosure, and therefore may include information that does not constitute prior art known to those skilled in the art. Summary of the Invention

[0005] The purpose of this application is to provide a method and related equipment for fusion of knee joint image information based on coordinate association, which at least to some extent overcomes the problems existing in the prior art. It extracts and effectively fuses features from sagittal and coronal planes at different spatial scales, achieving cross-sequence information fusion at various scales to enable more effective feature learning. By utilizing coronal and sagittal information from MRI images for lesion identification, the model can fully utilize information from different angles and scales, thereby improving the accurate identification capability of knee joint wear. Other features and advantages of this application will become apparent from the following detailed description, or may be learned in part by practice of the invention.

[0006] According to one aspect of this application, a method for fusion of knee joint image information based on coordinate association is provided, comprising: acquiring a training dataset and target user data; processing the target user data to generate attribute information of the target user, wherein the attribute information of the target user includes a sagittal sequence of the knee joint, a coronal proton density sequence of the knee joint, and target coordinate information of the knee joint sequence matching the target user, wherein the target coordinate information of the knee joint sequence includes coordinate information corresponding to the sagittal sequence of the knee joint and coordinate information corresponding to the coronal proton density sequence of the knee joint; processing the sagittal sequence of the knee joint and the coronal proton density sequence of the knee joint based on the target coordinate information of the knee joint sequence to generate a knee joint image set, wherein the knee joint image set includes half-images corresponding to different knee joint images. The training dataset is preprocessed to generate a training dataset with labeled information, wherein the labeled information is used to characterize physiological factors affecting meniscus injury. The labeled training dataset is then processed according to preset processing rules to generate a training set and a validation set. An initial knee joint image fusion model matching the labeled information is obtained, wherein the initial knee joint image fusion model is set based on target user data received within a preset time period. The initial knee joint image fusion model is trained based on the training set and the validation set to generate a target knee joint image fusion model. The knee joint image set is then processed based on the target knee joint image fusion model to generate knee joint wear information for the target user.

[0007] In one embodiment of this application, processing the target user data to generate target user attribute information includes: performing data cleaning processing on the target user data to generate knee joint information of the target user; processing the knee joint information of the target user based on a preset image extraction model to generate knee joint image information of the target user; processing the knee joint image information of the target user to generate knee joint material parameter values ​​of the target user; and generating target user attribute information based on the knee joint material parameter values ​​of the target user.

[0008] In one embodiment of this application, preprocessing the training dataset to generate a training dataset with identification information includes: extracting features from the training dataset to determine an original feature library; dividing the original feature library into various feature datasets to generate a training dataset and a validation set; using a classifier to predict the various validation sets in the original feature library and determining the prediction results; using a preset algorithm to train on the various training datasets in the original feature library to obtain the validation set class prediction results; and generating training samples with identification information based on the prediction results and the validation set class prediction results.

[0009] In one embodiment of this application, feature extraction is performed on the training dataset to determine the original feature library, including: processing standard material information based on preset processing rules to generate standardized features, wherein the standardized features are meniscus features with clearly defined and known material composition; processing the standardized features based on preset feature filtering and dimensionality reduction rules to generate original features; and generating the original feature library based on several original features.

[0010] In one embodiment of this application, training the initial knee joint image fusion model based on the training set and the validation set to generate a target knee joint image fusion model includes: extracting multiple sets of data from the training set, wherein each set of data contains a preset number of data samples, and at least one data sample includes identification information; training the initial knee joint image fusion model based on the data samples in the multiple sets of data to generate a trained knee joint image fusion model; processing the trained knee joint image fusion model based on the validation set to generate a validation result; if the data sample containing identification information in the validation result indicates that the physiological factors affecting meniscus injury are in an abnormal state, then the trained knee joint image fusion model is used as the target knee joint image fusion model.

[0011] In one embodiment of this application, the knee joint image set is processed based on the target knee joint image fusion model to generate knee joint wear information of the target user. This includes: processing the knee joint image set based on the target knee joint image fusion model to generate knee joint image coding features; and processing the knee joint image coding features based on the target joint wear warning model to generate correlation values ​​of different knee joint images. The correlation values ​​of different knee joint images are used to characterize the data sharing information between different knee joint images from different perspectives. If the correlation value of a knee joint image is higher than a preset correlation value, it indicates that the target user's knee joint is in an abnormal state.

[0012] In one embodiment of this application, the knee joint image set is processed based on the target knee joint image fusion model to generate knee joint wear information of the target user, including: the target joint wear warning model includes a calculation formula for obtaining the correlation value of different knee joint images, the calculation formula being: Q = X·W Q K = X·W K V = X·W V Where Attention(Q,K) represents the association value between different knee joint images, d k The softmax function, representing the dimension of the feature vector, is used to normalize the similarity score, K. TV represents the transpose of the key vector, V represents the value vector, Q represents the query vector, K represents the key vector, and W represents the query vector. Q W K W V Let X be the weight matrix corresponding to each of them, and let X be the input sequence.

[0013] Another aspect of this application provides a knee joint image information fusion device based on coordinate association, characterized in that it includes: an acquisition module for acquiring a training dataset and target user data; acquiring an initial knee joint image fusion model matching identification information, wherein the initial knee joint image fusion model is set based on target user data received within a preset time period; and a processing module for processing the target user data to generate attribute information of the target user, wherein the attribute information of the target user includes a knee joint sagittal sequence, a knee joint coronal proton density sequence, and knee joint sequence target coordinate information matching the target user, the knee joint sequence target coordinate information including coordinate information corresponding to the knee joint sagittal sequence and coordinate information corresponding to the knee joint coronal proton density sequence; based on The knee joint sequence target coordinate information is processed to generate a knee joint image set by processing the sagittal and coronal proton density sequences of the knee joint. This knee joint image set includes meniscus location information corresponding to different knee joint images and knee joint material parameter values ​​for the target user. The training dataset is preprocessed to generate a training dataset with labeled information, which characterizes physiological factors affecting meniscus injury. The labeled training dataset is then processed based on preset processing rules to generate a training set and a validation set. The initial knee joint image fusion model is trained using the training set and the validation set to generate a target knee joint image fusion model. Finally, the knee joint image set is processed based on the target knee joint image fusion model to generate knee joint wear information for the target user.

[0014] According to another aspect of this application, an electronic device is characterized by comprising: a first processor; and a memory for storing executable instructions of the first processor; wherein the first processor is configured to execute the above-described coordinate-association-based knee joint image information fusion method by executing the executable instructions.

[0015] According to another aspect of this application, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a second processor, implements the above-described method for fusion of knee joint image information based on coordinate association.

[0016] According to another aspect of this application, a computer program product is provided, comprising a computer program, characterized in that, when the computer program is executed by a third processor, it implements the above-described method for fusion of knee joint image information based on coordinate association.

[0017] This application provides a method and related equipment for fusion of knee joint image information based on coordinate association. The method involves a server acquiring a training dataset and target user data; processing the target user data to generate attribute information for the target user, including a sagittal knee joint sequence, a coronal knee joint proton density sequence, and target coordinate information for the knee joint sequence. The target coordinate information for the knee joint sequence includes coordinates corresponding to both the sagittal and coronal knee joint sequences; and processing the sagittal and coronal knee joint proton density sequences based on the target coordinate information to generate a knee joint image set, which includes different knee joint image pairs. The model obtains the corresponding meniscus location information and the knee joint material parameter values ​​of the target user; preprocesses the training dataset to generate a training dataset with labeled information, which is used to characterize the physiological factors affecting meniscus injury; processes the labeled training dataset according to preset processing rules to generate a training set and a validation set; obtains an initial knee joint image fusion model matching the labeled information, which is set based on the target user data received within a preset time period; trains the initial knee joint image fusion model based on the training set and validation set to generate a target knee joint image fusion model; processes the knee joint image set based on the target knee joint image fusion model to generate the knee joint wear information of the target user. Features from sagittal and coronal planes are extracted and effectively fused at different spatial scales, achieving cross-sequence information fusion at various scales to achieve more effective feature learning. Lesion identification is performed using information from the coronal and sagittal planes of MRI images, enabling the model to fully utilize information from different angles and scales, thereby improving the accurate identification ability of knee joint wear.

[0018] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0019] Figure 1 The flowchart illustrates a method for fusing knee joint image information based on coordinate association, provided in an embodiment of this application.

[0020] Figure 2 A schematic diagram of the structure of a knee joint image information fusion device based on coordinate association provided in an embodiment of this application is shown;

[0021] Figure 3 This illustration shows a schematic diagram of the structure of an electronic device according to an embodiment of this application;

[0022] Figure 4 A schematic diagram of a storage medium provided in one embodiment of this application is shown. Detailed Implementation

[0023] The preferred embodiments of the present invention will be described below with reference to the accompanying drawings. It should be understood that the preferred embodiments described herein are for illustration and explanation only and are not intended to limit the present invention.

[0024] The following is combined with Figure 1 This application describes a coordinate-association-based knee joint image information fusion method according to exemplary embodiments thereof. It should be noted that the following application scenarios are shown only to facilitate understanding of the spirit and principles of this application, and the embodiments of this application are not limited in any way. Rather, the embodiments of this application can be applied to any applicable scenario.

[0025] In one embodiment, this application also proposes a method and related equipment for fusion of knee joint image information based on coordinate association. Figure 1 A schematic flowchart illustrating a method for fusing knee joint image information based on coordinate association according to an embodiment of this application is shown. Figure 1 As shown, this method is applied to a server and includes:

[0026] S101, Obtain the training dataset and target user data.

[0027] In one embodiment, target user data is collected, including the patient's basic information, clinical diagnosis, surgical records, treatment process, rehabilitation status, etc. This invention does not limit this, and the applicant can select and set it according to actual needs.

[0028] The training dataset consists of knee joint information from other users matched with the target user data, identifying the target user group, such as patients with anterior cruciate ligament tears, fractures, or knee injuries. These users will be the focus of model training and analysis. Key information, such as injury type, surgical details, postoperative recovery, and complications, is extracted from the target users' electronic medical records. Based on the target users' electronic medical record information, similar electronic medical record data is matched from a larger patient group to form the training dataset.

[0029] S102, process the target user data to generate the target user's attribute information.

[0030] In one embodiment, the attribute information of the target user includes a sagittal sequence of the knee joint, a coronal proton density sequence of the knee joint, and target coordinate information of the knee joint sequence that matches the target user. The target coordinate information of the knee joint sequence includes coordinate information corresponding to the sagittal sequence of the knee joint and coordinate information corresponding to the coronal proton density sequence of the knee joint.

[0031] Sagittal sequences are a standard view in MRI scans, providing an image of the knee joint along the body's long axis. This sequence is suitable for evaluating the anterior cruciate ligament, posterior cruciate ligament, meniscus, and articular cartilage of the knee. Coronal proton density (PD) sequences are another MRI scan view, providing an image along the width of the knee joint. This sequence helps to visualize the collateral ligaments, patella, and surrounding soft tissues of the knee. Target coordinate information refers to the coordinate points in MRI images used to locate and analyze specific structures. This coordinate information is crucial for accurately measuring and assessing the anatomical structures and pathological changes of the knee joint.

[0032] The coordinate information corresponding to the sagittal sequence of the knee joint includes identifying and locating the coordinate points of key knee joint structures in the sagittal images, such as specific areas of ligaments, menisci, and cartilage; the coordinate information corresponding to the coronal proton density sequence of the knee joint includes identifying and locating the coordinate points of key knee joint structures in the coronal images, such as specific areas of the collateral ligaments and patella.

[0033] Using this coordinate information, doctors and researchers can perform precise image analysis and measurements, such as measuring meniscus thickness, assessing the extent of cartilage damage, or determining ligament integrity. This detailed image and coordinate information is crucial for the clinical diagnosis of knee joint diseases and the planning of treatments (such as surgery). It helps doctors better understand a patient's specific condition and develop personalized treatment plans. Through this detailed knee joint sequence target coordinate information, medical professionals can more accurately assess the condition of the knee joint, providing patients with more precise diagnoses and more effective treatment options.

[0034] Data cleaning is performed on the target user data to generate knee joint information. The collected knee joint data is cleaned to remove invalid, erroneous, or irrelevant information, ensuring data quality. The cleaned data is used to generate basic knee joint information for the target user, including basic anatomical structures and potential injuries. Based on a pre-defined image extraction model, the knee joint information is processed to generate knee joint images. The model extracts detailed images of the knee joint from MRI or CT scans, including information on its various components such as bones, cartilage, ligaments, and menisci.

[0035] The knee joint image information of the target user is processed to generate knee joint material parameter values. In the process of knee joint image analysis and material parameter quantification, high-quality MRI or CT scans are first used to acquire knee joint images. Image preprocessing, including noise reduction and contrast enhancement, lays the foundation for subsequent analysis. The analysis includes cartilage thickness measurement, bone mineral density assessment, ligament size and morphology analysis, and detailed examination of the meniscus and joint space. Furthermore, assessing synovial fluid and soft tissue damage, as well as joint range of motion, are important components of the diagnostic process. Further, biomechanical modeling and multimodal image fusion techniques allow for a more comprehensive understanding of the knee joint's structure and function. The application of computer-aided diagnostic systems improves the accuracy and efficiency of diagnosis. Finally, the quantified parameters are combined with clinical data to generate detailed reports, assisting physicians in diagnostic and treatment decisions.

[0036] Based on the target user's knee joint material parameter values, the system generates the target user's attribute information. Detailed attribute information is generated based on these values, which is crucial for assessing knee health. Features such as the degree of cartilage degeneration and ligament damage are extracted from images and material parameters and analyzed in depth. This process allows medical professionals to gain a deeper understanding of the patient's knee condition, providing more accurate diagnoses and personalized treatments, while improving the success rate of surgeries and treatments. Furthermore, it provides a scientific basis for clinical decisions regarding knee injury and disease management.

[0037] S103, based on the target coordinate information of the knee joint sequence, process the sagittal sequence and coronal proton density sequence of the knee joint to generate a set of knee joint images.

[0038] In one implementation, the knee joint image set includes meniscus location information corresponding to different knee joint images and knee joint material parameter values ​​for the target user. In the knee joint image analysis process, the proton density sequences in sagittal and coronal planes are first precisely processed using the target coordinate information of the knee joint sequence to generate a detailed knee joint image set. This image set pays particular attention to the location information of the meniscus, which is a key area for assessing knee joint injury. Simultaneously, material parameter values ​​of the knee joint, including cartilage thickness, bone density, and ligament size, are extracted through image analysis; these parameters are crucial for personalized diagnosis and treatment planning.

[0039] Furthermore, these images and parameter values ​​are used to support clinical decisions, such as treatment planning and surgical design optimization. Integrating this data with other patient clinical information creates a comprehensive medical record, assisting physicians in continuous patient monitoring and treatment adjustments, providing a scientific basis for the diagnosis and treatment of knee joint diseases, and improving the quality and efficiency of medical services.

[0040] In another implementation, to obtain the real-world coordinates of each point in the sagittal and coronal proton density sequences of the knee joint, the applicant needs to use the following fields from the image DICOM file:

[0041] Field Name effect InstanceNumber This indicates which image is in a sequence. ImagePositionPatient This indicates the position of the top-left corner of the current image in real space. ImageOrientationPatient It records the angular changes of the point coordinate system in the current image. PixelSpacing It records the ratio between the x and y coordinates in the current image and the true coordinates. Spacing Between Slices Indicates the distance between two images in real space.

[0042] The applicant assumes that the pixel coordinates within the current image are pixel[x,y]. Then, its coordinates in real space are:

[0043] real[x,y,z]=pixel[x,y]×ImageOrientationPatient×spacing+

[0044] ImagePositionPatienteal;

[0045] All multiplications in the formula are matrix multiplications, where:

[0046]

[0047] S104, preprocess the training dataset to generate a training dataset with identification information.

[0048] In one implementation, features are extracted from the training dataset to determine an original feature library. These features include anatomical parameters of the knee joint, movement habits, and past sports injuries. Using the original feature library, the features are divided into different datasets to generate a training dataset for model training and a validation set for model evaluation. A classifier is used to predict the values ​​in each validation set of the original feature library to determine the prediction results. A pre-defined algorithm is used to train the training dataset, learning the relationship between features and meniscus injuries, and obtaining the prediction results for the validation sets. An appropriate machine learning algorithm, such as logistic regression, support vector machine, random forest, or neural network, is selected. These algorithms can handle classification or regression tasks. The training dataset is prepared, including labeled examples containing various physiological and environmental features related to meniscus injuries. The pre-defined algorithm is used to train the training dataset, enabling the model to learn how to identify patterns and trends of meniscus injuries from the features.

[0049] Based on the prediction results and the validation set prediction results, training samples with labeled information are generated. This labeled information is used to identify and characterize key physiological factors that may affect meniscus injury, providing important input features for the model. Labeled information plays a central role in the knee joint image fusion model for meniscus injury, providing key input features by identifying crucial physiological factors. These factors include knee joint anatomy, biomechanical load, biomarkers, age, sex, injury history, type of exercise, body morphology, genetic factors, lifestyle habits, psychological state, environmental conditions, and medical history. This comprehensive information helps assess an individual's susceptibility to and risk of meniscus injury.

[0050] By analyzing this information, healthcare professionals can more accurately predict the likelihood of meniscus injuries and develop personalized preventative measures and treatment plans for patients. For example, considering biomechanics can reveal the risk of injury due to overuse or improper technique; while genetic factors and biomarkers may indicate differences in the biological response of individuals to injury. Furthermore, lifestyle habits and psychological factors also influence an individual's health status and injury risk.

[0051] Integrating this multi-dimensional information into a knee joint imaging fusion model not only improves the model's accuracy but also helps physicians better understand the complexity of meniscus injuries, thus providing more comprehensive clinical decision support. This personalized medical approach helps optimize patient treatment outcomes, reduce unnecessary interventions, and improve patient satisfaction and quality of life.

[0052] In another implementation, the standard material information is processed based on preset processing rules to generate standardized features, where the standardized features are meniscus characteristics with clearly defined and known material composition. Specifically, standard material information is collected, including normal and abnormal MRI or CT scan images of the meniscus. Features related to meniscus injury and degeneration are selected, such as the shape, size, signal intensity, location, and relationship of the meniscus to other knee joint structures. The selected features are quantified, for example, by measuring the thickness of the meniscus, assessing signal homogeneity, or identifying potential tears and injuries, ensuring that the feature data are comparable between different samples and scans. Standardization techniques are used to eliminate differences caused by different scanning conditions.

[0053] The extracted standardized features are stored in a feature library to provide a foundation for subsequent model training and data analysis. The accuracy and reliability of the features are verified to ensure they accurately reflect the state of the meniscus. The data in the feature library is used to train a machine learning model, learning the relationship between features and meniscus injury. Based on expertise and data characteristics, feature selection rules are formulated to identify the features most influential on the prediction target. Dimensionality reduction techniques, such as Principal Component Analysis (PCA), Linear Discriminant Analysis (LDA), or t-distributed random neighborhood embedding (t-SNE), are applied to reduce the number of features while retaining important information. Combining the feature selection and dimensionality reduction results, a set of original features is generated that effectively represents the data and reduces complexity. The generated original features are aggregated to construct the original feature library, providing a foundation for subsequent model training and data analysis. The original feature library is validated to ensure that the selected features accurately reflect the research target, such as the characteristics of meniscus injury.

[0054] S105, The training dataset with identification information is processed based on preset processing rules to generate a training set and a validation set.

[0055] In one implementation, features are extracted from a training dataset containing identification information to determine an original feature library. The original feature library is then processed using a neural network model to generate forged feature data, which differs from the feature data in the original feature library. The original feature library is then divided to generate a training dataset and a validation set, wherein at least one training dataset and one validation set are used, and the validation set includes the forged feature data. By generating forged feature data, preparation is made for subsequent validation of the initial knee joint image fusion model. That is, the forged feature data is actually anomalous data; however, if the initial knee joint image fusion model fails to identify the current data as anomalous, it indicates that the current model's detection results are inaccurate.

[0056] S106, Obtain an initial knee joint image fusion model that matches the identification information.

[0057] In one implementation, the initial knee joint image fusion model is set based on target user data received within a preset time period. That is, the knee joint image fusion model is trained by selecting training samples related to the target user data to avoid excessive error in the results generated by the model.

[0058] S107, The initial knee joint image fusion model is trained based on the training set and the validation set to generate the target knee joint image fusion model.

[0059] In one implementation, multiple sets of data are extracted from a training set, each set containing a predetermined number of data samples, wherein at least one data sample includes identification information. An initial knee joint image fusion model is trained based on the data samples from these multiple sets to generate a trained knee joint image fusion model. In developing an accurate knee joint image fusion model, firstly, data sets containing a predetermined number of samples are extracted from the training set, ensuring that each set contains at least one sample with key identification information. This identification information may indicate a patient's specific physiological state or infection risk factors. Then, these data sets are used to systematically train the initial knee joint image fusion model to form a trained model.

[0060] The trained knee joint image fusion model was processed using a validation set to generate validation results. Subsequently, an independent validation set was used to validate the trained model again, generating validation results. When analyzing the validation results, particular attention was paid to samples with labeled information to determine whether the model could accurately identify physiological factors affecting meniscus injuries. If the model could effectively predict abnormal states—that is, samples with abnormal physiological factors—then the trained model was considered the target knee joint image fusion model. If the data samples containing labeled information in the validation results indicated abnormal physiological factors affecting postoperative joint infection, then the trained knee joint image fusion model was used as the target knee joint image fusion model.

[0061] Machine learning models, such as classification, regression, or deep learning models, are trained using training data. The model's performance is then evaluated on a validation set. The model parameters and structure are adjusted and optimized. The trained model is then used to predict the risk of postoperative complications such as infection in target users. Based on the model's predictions, personalized medical advice, such as preventative measures and treatment plans, is provided to doctors.

[0062] Furthermore, model optimization and iteration are ongoing processes that require continuous adjustments based on clinical feedback and validation results to improve the model's accuracy and usability. After the model is applied in clinical practice, its performance is continuously monitored and regularly evaluated to ensure it adapts to changes in the healthcare environment. Throughout this process, maintaining data privacy and ethical standards is crucial to ensuring the safe and compliant use of patient data. Through this process, healthcare professionals can build an effective early warning system to help identify infection risks in advance, optimize preventative measures, improve the quality of postoperative recovery for patients, and simultaneously reduce the burden on the healthcare system.

[0063] S108, Based on the target knee joint image fusion model, the knee joint image set is processed to generate knee joint wear information of the target user.

[0064] In one implementation, a set of knee joint images is processed based on a target knee joint image fusion model to generate knee joint image coding features. The target knee joint image fusion model is used to process the set of knee joint images, extract key visual information, and generate coding features from the knee joint images. These features can represent the morphological features and potential signs of wear of the knee joint.

[0065] Based on the target joint wear early warning model, the encoding features of knee joint images are processed to generate correlation values ​​for different knee joint images. The correlation values ​​of different knee joint images are used to characterize the data sharing information between different knee joint images from different perspectives. That is, it is used to determine whether the corresponding knee joint images have a risk of infection from multiple perspectives. If the correlation value of the knee joint image is higher than the preset correlation value, it indicates that the target user's knee joint is in an abnormal state.

[0066] First, the model processes knee joint images to generate coded features that capture morphological changes in the knee joint. Next, correlation values ​​are calculated between different knee joint images to quantify image similarity and data sharing. A preset correlation value threshold is used to determine whether the image correlation is abnormal. When the correlation value exceeds this threshold, it indicates that the target user's knee joint may be in an abnormal state, triggering an early warning signal to assist medical professionals in making further diagnostic and treatment decisions. Furthermore, the model includes mechanisms for continuous monitoring and optimization based on feedback to ensure the accuracy of predictions and relevance to clinical applications.

[0067] Specifically, the connection between different knee joint images is strengthened by calculating the similarity between features from two different viewpoints. This is achieved by calculating the inner product of two feature vectors, thus obtaining a similarity score (i.e., association value). Then, by normalizing these scores, the attention weight of each feature vector to the other viewpoint can be obtained. Assuming the applicant has two feature vectors, denoted as Q and K, representing image features from different viewpoints respectively, and the target joint wear warning model includes a formula for calculating the association value of different knee joint images. The formula is as follows: Q = X·W Q K = X·W K V = X·W V Where Attention(Q,K) represents the association value between different knee joint images, d k K represents the dimension of the feature vector, the softmax function is used to normalize the similarity score. T V represents the transpose of the key vector, V represents the value vector, used for weighted summation to generate the final attention feature representation, Q represents the query vector, K represents the key vector, and W represents the value vector. Q W K WV Let X be the weight matrix corresponding to each of them, and let X be the input sequence.

[0068] In this application, the server acquires the training dataset and target user data; the target user data is cleaned to generate knee joint information; the knee joint information is processed based on a preset image extraction model to generate knee joint image information; the knee joint image information is processed to generate knee joint material parameter values; the target user's attribute information is generated based on the knee joint material parameter values, wherein the target user's attribute information includes a knee joint sagittal sequence, a knee joint coronal proton density sequence, and knee joint sequence target coordinate information matching the target user, the knee joint sequence target coordinate information including coordinate information corresponding to the knee joint sagittal sequence and coordinate information corresponding to the knee joint coronal proton density sequence; the knee joint sagittal sequence and knee joint coronal proton density sequence are processed based on the knee joint sequence target coordinate information to generate a knee joint image set, wherein the knee joint image set includes meniscus position information corresponding to different knee joint images and knee joint material parameter values ​​of the target user.

[0069] Standard material information is processed according to preset processing rules to generate standardized features, where standardized features are meniscus features with clearly defined and known material composition. These standardized features are then processed using preset feature selection and dimensionality reduction rules to generate original features. An original feature library is generated based on these original features. The original feature library is then divided into various feature datasets to generate training and validation datasets. A classifier is used to predict the results on each validation dataset of the original feature library. A preset algorithm is used to train the training datasets of the original feature library to obtain validation set prediction results. Based on the prediction results and validation set prediction results, training samples with labeled information are generated, where the labeled information characterizes the physiological factors affecting meniscus injury. The training data with labeled information is then processed according to preset processing rules. The system processes the data to generate a training set and a validation set; it obtains an initial knee joint image fusion model that matches the identification information, wherein the initial knee joint image fusion model is set based on target user data received within a preset time period; it extracts multiple sets of data from the training set, wherein each set of data contains a preset number of data samples, wherein at least one data sample includes identification information; it trains the initial knee joint image fusion model based on the data samples in the multiple sets of data to generate a trained knee joint image fusion model; it processes the trained knee joint image fusion model based on the validation set to generate validation results; if the data sample containing identification information in the validation results indicates that the physiological factors affecting meniscus injury are in an abnormal state, then the trained knee joint image fusion model is used as the target knee joint image fusion model.

[0070] The knee joint image set is processed based on the target knee joint image fusion model to generate knee joint image coding features. The knee joint image coding features are then processed based on the target joint wear warning model to generate correlation values ​​between different knee joint images. These correlation values ​​characterize data sharing information between different knee joint images from different viewpoints. If the correlation value of a knee joint image is higher than a preset correlation value, it indicates that the target user's knee joint is in an abnormal state. The target joint wear warning model includes a calculation formula for obtaining the correlation values ​​of different knee joint images. The calculation formula is as follows: Q = X·W Q K = X·W K V = X·W V Where Attention(Q,K) represents the association value between different knee joint images, d k The softmax function, representing the dimension of the feature vector, is used to normalize the similarity score, K. T V represents the transpose of the key vector, V represents the value vector, Q represents the query vector, K represents the key vector, and W represents the query vector. Q W K W V Let X be the input sequence and X be the corresponding weight matrix. Features from sagittal and coronal planes are extracted and effectively fused at different spatial scales, achieving cross-sequence information fusion at various scales for more effective feature learning. Lesion identification is performed using information from coronal and sagittal MRI images, enabling the model to fully utilize information from different angles and scales, thereby improving the accuracy of knee joint wear identification.

[0071] In one implementation, such as Figure 2 As shown, this application also provides a knee joint image information fusion device based on coordinate association, comprising:

[0072] The acquisition module 201 is used to acquire training dataset and target user data; acquire an initial knee joint image fusion model that matches the identification information, wherein the initial knee joint image fusion model is set based on the target user data received within a preset time period;

[0073] Processing module 202 is used to process the target user data to generate target user attribute information, wherein the target user attribute information includes a knee joint sagittal sequence, a knee joint coronal proton density sequence, and knee joint sequence target coordinate information matching the target user, wherein the knee joint sequence target coordinate information includes coordinate information corresponding to the knee joint sagittal sequence and coordinate information corresponding to the knee joint coronal proton density sequence; and processes the knee joint sagittal sequence and knee joint coronal proton density sequence based on the knee joint sequence target coordinate information to generate a knee joint image set, wherein the knee joint image set includes... The system includes meniscus location information corresponding to different knee joint images and knee joint material parameter values ​​of the target user; the training dataset is preprocessed to generate a training dataset with labeled information, wherein the labeled information is used to characterize physiological factors affecting meniscus injury; the labeled training dataset is processed according to preset processing rules to generate a training set and a validation set; the initial knee joint image fusion model is trained based on the training set and the validation set to generate a target knee joint image fusion model; the knee joint image set is processed based on the target knee joint image fusion model to generate knee joint wear information of the target user.

[0074] This application provides an electronic device, such as... Figure 3 As shown, the electronic device 3 includes a first processor 300, a memory 301, a bus 302, and a communication interface 303. The first processor 300, the communication interface 303, and the memory 301 are connected through the bus 302. The memory 301 stores a computer program that can run on the first processor 300. When the first processor 300 runs the computer program, it executes the knee joint image information fusion method based on coordinate association provided in any of the foregoing embodiments of this application.

[0075] The memory 301 may include high-speed random access memory (RAM) or non-volatile memory, such as at least one disk storage device. Communication between this system network element and at least one other network element is achieved through at least one communication interface 303 (which can be wired or wireless), such as the Internet, wide area network, local area network, or metropolitan area network.

[0076] Bus 302 can be an ISA bus, PCI bus, or EISA bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. Memory 301 is used to store programs. After receiving an execution instruction, the first processor 300 executes the program. The knee joint image information fusion method based on coordinate association disclosed in any of the foregoing embodiments of this application can be applied to the first processor 300, or implemented by the first processor 300.

[0077] The first processor 300 may be an integrated circuit chip with signal processing capabilities. In implementation, each step of the above method can be completed by the integrated logic circuitry in the hardware of the first processor 300 or by instructions in software form. The first processor 300 may be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it may also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), an off-the-shelf programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor may be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this application may be executed by a hardware decoding processor, or executed by a combination of hardware and software modules in the decoding processor. The software modules may reside in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. The storage medium is located in memory 301. The first processor 300 reads the information in memory 301 and, in conjunction with its hardware, completes the steps of the above method.

[0078] The electronic devices provided in the above embodiments of this application and the knee joint image information fusion method based on coordinate association provided in the embodiments of this application are based on the same inventive concept and have the same beneficial effects as the methods adopted, run or implemented by the applications stored therein.

[0079] This application provides a computer-readable storage medium, such as... Figure 4 As shown, the computer-readable storage medium 401 stores a computer program, which is read and run by the second processor 402 to implement the knee joint image information fusion method based on coordinate association as described above.

[0080] The computer-readable storage medium provided in the above embodiments of this application and the knee joint image information fusion method based on coordinate association provided in the embodiments of this application are based on the same inventive concept and have the same beneficial effects as the methods adopted, run or implemented by the application stored therein.

[0081] This application provides a computer program product, including a computer program, which is executed by a third processor to implement the method described above.

[0082] The computer program products provided in the above embodiments of this application and the knee joint image information fusion method based on coordinate association provided in the embodiments of this application are based on the same inventive concept and have the same beneficial effects as the methods adopted, run or implemented by the applications stored therein.

[0083] It should be noted that in this application, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0084] The various embodiments in this application are described in a related manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the embodiments for evaluating the coordinate-association-based knee joint image information fusion method, electronic device, electronic device, and readable storage medium are basically similar to the coordinate-association-based knee joint image information fusion method embodiments described above, and therefore the descriptions are relatively simple. Relevant parts can be referred to in the descriptions of the coordinate-association-based knee joint image information fusion method embodiments described above.

[0085] While this application discloses the above information, it is not limited thereto. Any person skilled in the art can make various modifications and alterations without departing from the spirit and scope of this application; therefore, the scope of protection of this application shall be determined by the scope defined in the claims.

Claims

1. A method for fusing knee joint image information based on coordinate association, characterized in that, include: Obtain the training dataset and target user data; The target user data is processed to generate target user attribute information, wherein the target user attribute information includes a knee joint sagittal sequence, a knee joint coronal proton density sequence, and knee joint sequence target coordinate information that match the target user. The knee joint sequence target coordinate information includes coordinate information corresponding to the knee joint sagittal sequence and coordinate information corresponding to the knee joint coronal proton density sequence. Based on the target coordinate information of the knee joint sequence, the sagittal sequence and coronal proton density sequence of the knee joint are processed to generate a set of knee joint images. The set of knee joint images includes meniscus position information corresponding to different knee joint images and knee joint material parameter values ​​of the target user. The training dataset is preprocessed to generate a training dataset with labeling information, wherein the labeling information is used to characterize the physiological factors affecting meniscus injury; The training dataset with identification information is processed based on preset processing rules to generate a training set and a validation set; Obtain an initial knee joint image fusion model that matches the identification information, wherein the initial knee joint image fusion model is set based on target user data received within a preset time period; The initial knee joint image fusion model is trained based on the training set and the validation set to generate the target knee joint image fusion model. The knee joint image set is processed based on the target knee joint image fusion model to generate knee joint wear information of the target user. This includes processing the knee joint image set based on the target knee joint image fusion model to generate knee joint image coding features; and processing the knee joint image coding features based on the target joint wear warning model to generate correlation values ​​of different knee joint images. The correlation values ​​of different knee joint images are used to characterize the data sharing information between different knee joint images from different perspectives. If the correlation value of a knee joint image is higher than a preset correlation value, it indicates that the target user's knee joint is in an abnormal state.

2. The method as described in claim 1, characterized in that, The target user data is processed to generate the target user's attribute information, including: The target user data is cleaned to generate the target user's knee joint information; The knee joint information of the target user is processed based on a preset image extraction model to generate knee joint image information of the target user; The knee joint image information of the target user is processed to generate material parameter values ​​of the target user's knee joint; Generate the target user's attribute information based on the target user's knee joint material parameter values.

3. The method as described in claim 1, characterized in that, The training dataset is preprocessed to generate a training dataset with labeling information, including: Feature extraction is performed on the training dataset to determine the original feature library; Based on the original feature library, divide each feature dataset to generate a training dataset and a validation set; The original feature library is divided into various validation sets using a classifier for prediction, and the prediction results are determined. The original feature library is divided into various training datasets for training using a preset algorithm, and the validation set class prediction results are obtained. Based on the prediction results and the validation set prediction results, training samples with labeling information are generated.

4. The method as described in claim 3, characterized in that, Feature extraction is performed on the training dataset to determine the original feature library, including: The standard material information is processed based on preset processing rules to generate standardized features, wherein the standardized features are meniscus features with clearly defined and known material composition. The standardized features are processed based on preset feature filtering and dimensionality reduction rules to generate original features; An original feature library is generated based on several original features.

5. The method as described in claim 3, characterized in that, The initial knee joint image fusion model is trained based on the training set and the validation set to generate a target knee joint image fusion model, including: Multiple sets of data are extracted from the training set, wherein each set of data contains a preset number of data samples, and at least one data sample includes identification information. The initial knee joint image fusion model is trained based on data samples from multiple sets of data groups to generate a trained knee joint image fusion model. The trained knee joint image fusion model is processed based on the validation set to generate validation results; If the data sample containing the identification information in the verification result is in an abnormal state of physiological factors affecting meniscus injury, then the trained knee joint image fusion model will be used as the target knee joint image fusion model.

6. The method as described in claim 1, characterized in that, The knee joint image set is processed based on the target knee joint image fusion model to generate knee joint wear information for the target user, and the process also includes: The target joint wear early warning model includes a calculation formula for obtaining correlation values ​​of different knee joint images, and the calculation formula is as follows: in, This represents the correlation value between different knee joint images. The softmax function, which represents the dimension of the feature vector, is used to normalize the similarity score. Let V represent the transpose of the key vector, Q represent the value vector, Q represent the query vector, and K represent the key vector. , Let X be the weight matrix corresponding to each of them, and let X be the input sequence.

7. A knee joint image information fusion device based on coordinate association, characterized in that, For implementing the method of claim 1, the apparatus includes: The acquisition module is used to acquire training datasets and target user data; and to acquire an initial knee joint image fusion model that matches the identification information, wherein the initial knee joint image fusion model is set based on the target user data received within a preset time period. The processing module is used to process the target user data to generate attribute information of the target user. The attribute information includes a sagittal knee joint sequence, a coronal knee joint proton density sequence, and target coordinate information of the knee joint sequence matching the target user. The target coordinate information of the knee joint sequence includes coordinate information corresponding to the sagittal knee joint sequence and coordinate information corresponding to the coronal knee joint proton density sequence. Based on the target coordinate information of the knee joint sequence, the sagittal knee joint sequence and the coronal knee joint proton density sequence are processed to generate a knee joint image set. The knee joint image set includes... The training dataset is preprocessed to generate a training dataset with labeled information, wherein the labeled information is used to characterize physiological factors affecting meniscus injury. The labeled training dataset is then processed according to preset processing rules to generate a training set and a validation set. Based on the training set and the validation set, the initial knee joint image fusion model is trained to generate a target knee joint image fusion model. Based on the target knee joint image fusion model, the knee joint image set is processed to generate knee joint wear information for the target user.

8. An electronic device, characterized in that, include: First processor; and memory for storing executable instructions of the first processor; The first processor is configured to execute the knee joint image information fusion method based on coordinate association as described in any one of claims 1 to 6 by executing the executable instructions.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the second processor, it implements the knee joint image information fusion method based on coordinate association as described in any one of claims 1 to 6.

Images