Knee joint image information fusion method based on coordinate association and related equipment

Abstract

The invention provides a knee joint image information fusion method based on coordinate association and related equipment, and is applied to the technical field of data processing. The method comprises the steps of obtaining a training data set and target user data; processing the target user data to generate attribute information of the target user; processing the knee joint sagittal view sequence and the knee joint coronal view proton density sequence based on the knee joint sequence target coordinate information to generate a knee joint image set; preprocessing the training data set to generate a training data set with identification information; processing the training data set with the identification information based on a preset processing rule to generate a training set and a verification set; acquiring an initial knee joint image fusion model matched with the identification information; training the initial knee joint image fusion model based on the training set and the verification set to generate a target knee joint image fusion model; and processing the knee joint image set to generate knee joint wear information of the target user.

Description

Technical Field

[0001] The present invention relates to the technical field of data processing, and in particular to a coordinate association-based knee joint image information fusion method and related equipment. Background Art

[0002] The current detection model for knee joint damage only uses a single MRI sequence image as the model input, resulting in low recognition accuracy, blurred lesion boundaries, and the same lesion cannot be matched in different body positions. MRI has become the standard imaging method for diagnosing meniscus injuries due to its advantages of being non-invasive and having high-quality knee joint imaging. However, knee joint MRI images have the characteristics of thick layers and multiple body positions, which makes the film reading process time-consuming and complicated, requiring frequent image switching and diagnosis combined with multi-angle views. In addition, meniscus injuries are often localized and blurred, which further increases the difficulty of diagnosing meniscus injuries.

[0003] In recent years, deep learning models have far surpassed traditional image analysis methods in many fields. Their end-to-end characteristics give them the ability to automatically learn image features and coordinate optimization with downstream tasks. However, due to the complexity of the modality and the scarcity of public datasets, there are relatively few deep learning works in the knee MRI modality. The existing technology for the application of knee MRI is mainly limited to cartilage segmentation and cartilage injury detection. There are two main types. One is to use a single sagittal or coronal sequence as input. This method cannot establish the feature relationship between coronal and sagittal images within the model, resulting in a reduction in classification accuracy. The other method is to extract features from two sequences separately, and only directly concat the features of the two positions as fusion in the final classification layer. This cross-sequence information fusion is too simple and crude, and will lose multi-scale cross-sequence features.

[0004] It should be noted that the information disclosed in the above background technology section is only used to enhance the understanding of the background of the present disclosure, and therefore may include information that does not constitute the prior art known to ordinary technicians in the field. Summary of the invention

[0005] The purpose of the present application is to provide a method and related equipment for knee joint image information fusion based on coordinate association, which at least overcomes the problems existing in the prior art to a certain extent, extracts and effectively fuses the features of the sagittal and coronal planes at different spatial scales, and realizes cross-sequence information fusion at various scales to achieve more effective feature learning. Lesions are identified by using the coronal and sagittal information of MRI images, so that the model can make full use of information at different angles and scales, thereby improving the ability to accurately identify knee joint wear. Other features and advantages of the present application will become apparent from the detailed description below, or will be learned in part through the practice of the present invention.

[0006] According to one aspect of the present application, a method for fusion of knee joint image information based on coordinate association is provided, comprising: acquiring a training data set and target user data; processing the target user data to generate attribute information of the target user, wherein the attribute information of the target user comprises a knee joint sagittal sequence, a knee joint coronal proton density sequence and a knee joint sequence target coordinate information that matches the target user, and the knee joint sequence target coordinate information comprises coordinate information corresponding to the knee joint sagittal sequence and coordinate information corresponding to the knee joint coronal proton density sequence; processing the knee joint sagittal sequence and the knee joint coronal proton density sequence based on the knee joint sequence target coordinate information to generate a knee joint image set, wherein the knee joint image set comprises semi-rectangular ridges corresponding to different knee joint images; meniscus position information and knee joint material parameter values ​​of the target user; preprocessing the training data set to generate a training data set with identification information, wherein the identification information is used to characterize the physiological factors affecting meniscus injury; processing the training data set with identification information based on preset processing rules to generate a training set and a verification set; obtaining an initial knee joint image fusion model that matches the identification information, wherein the initial knee joint image fusion model is set based on the target user data received within a preset time period; training the initial knee joint image fusion model based on the training set and the verification set to generate a target knee joint image fusion model; processing the knee joint image set based on the target knee joint image fusion model to generate knee joint wear information of the target user.

[0007] In one embodiment of the present application, the target user data is processed to generate attribute information of the target user, including: performing data cleaning processing on the target user data to generate knee joint information of the target user; processing the knee joint information of the target user based on a preset image extraction model to generate knee joint image information of the target user; processing the knee joint image information of the target user to generate knee joint material parameter values ​​of the target user; and generating the attribute information of the target user based on the knee joint material parameter values ​​of the target user.

[0008] In one embodiment of the present application, the training data set is preprocessed to generate a training data set with identification information, including: extracting features from the training data set to determine an original feature library; dividing each feature data set according to the original feature library to generate a training data set and a verification set; using a classifier to predict each verification set divided from the original feature library to determine a prediction result; using a preset algorithm to train each training data set divided from the original feature library to obtain a verification set class prediction result; and generating a training sample with identification information based on the prediction result and the verification set class prediction result.

[0009] In one embodiment of the present application, feature extraction is performed on the training data set to determine an original feature library, including: processing standard material information based on preset processing rules to generate standardized features, wherein the standardized features are meniscus features with clear and known material composition; processing the standardized features based on preset feature screening and dimensionality reduction rules to generate original features; and generating an original feature library based on a number of original features.

[0010] In one embodiment of the present application, the initial knee joint image fusion model is trained based on the training set and the validation set to generate a target knee joint image fusion model, including: extracting multiple data groups from the training set, wherein each data group contains a preset number of data samples, wherein at least one data sample includes identification information; training the initial knee joint image fusion model based on the data samples in the multiple data groups to generate a trained knee joint image fusion model; processing the trained knee joint image fusion model based on the validation set to generate a verification result; if the data sample containing the identification information in the verification result indicates that the physiological factors affecting meniscus injury are in an abnormal state, then the trained knee joint image fusion model is used as the target knee joint image fusion model.

[0011] In one embodiment of the present application, the knee joint image set is processed based on the target knee joint image fusion model to generate knee joint wear information of the target user, including: processing the knee joint image set based on the target knee joint image fusion model to generate knee joint image coding features; processing the knee joint image coding features based on the target joint wear warning model to generate association values ​​of different knee joint images, wherein the association values ​​of different knee joint images are used to characterize data sharing information between different knee joint images at different perspectives: if the association value of the knee joint image is higher than the preset association value, it characterizes that the knee joint of the target user is in an abnormal state.

[0012] In one embodiment of the present application, the knee joint image set is processed based on the target knee joint image fusion model to generate knee joint wear information of the target user, including: the target joint wear warning model includes a calculation formula for obtaining the correlation value of different knee joint images, and the calculation formula is: Q=X·W Q ; K = X·W K ; V = X·W V ; Among them, Attention(Q,K) represents the association value of different knee joint images, d k The softmax function is used to normalize the similarity score, K Trepresents the transpose of the key vector, V represents the value vector, Q represents the query vector, K represents the key vector, and W Q , W K , W V are the weight matrices corresponding to them respectively, and X is the input sequence.

[0013] Another aspect of the present application is a knee joint image information fusion device based on coordinate association, characterized in that it includes: an acquisition module, used to acquire a training data set and target user data; acquire an initial knee joint image fusion model that matches the identification information, wherein the initial knee joint image fusion model is set based on the target user data received within a preset time period; a processing module, used to process the target user data to generate attribute information of the target user, wherein the attribute information of the target user includes a knee joint sagittal sequence, a knee joint coronal proton density sequence and a knee joint sequence target coordinate information that matches the target user, and the knee joint sequence target coordinate information includes coordinate information corresponding to the knee joint sagittal sequence and coordinate information corresponding to the knee joint coronal proton density sequence; based on The knee joint sequence target coordinate information processes the knee joint sagittal sequence and the knee joint coronal proton density sequence to generate a knee joint image set, wherein the knee joint image set includes meniscus position information corresponding to different knee joint images and knee joint material parameter values ​​of the target user; the training data set is preprocessed to generate a training data set with identification information, wherein the identification information is used to characterize the physiological factors affecting meniscus injury; the training data set with identification information is processed based on preset processing rules to generate a training set and a verification set; the initial knee joint image fusion model is trained based on the training set and the verification set to generate a target knee joint image fusion model; the knee joint image set is processed based on the target knee joint image fusion model to generate knee joint wear information of the target user.

[0014] According to another aspect of the present application, an electronic device is characterized in that it includes: a first processor; and a memory for storing executable instructions of the first processor; wherein the first processor is configured to execute the above-mentioned coordinate association-based knee joint image information fusion method by executing the executable instructions.

[0015] According to another aspect of the present application, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a second processor, the above-mentioned coordinate association-based knee joint image information fusion method is implemented.

[0016] According to another aspect of the present application, a computer program product is provided, including a computer program, characterized in that when the computer program is executed by a third processor, the above-mentioned coordinate association-based knee joint image information fusion method is implemented.

[0017] The present application provides a coordinate association-based knee joint image information fusion method and related equipment, wherein a server obtains a training data set and target user data; the target user data is processed to generate attribute information of the target user, wherein the attribute information of the target user includes a knee joint sagittal sequence, a knee joint coronal proton density sequence and a knee joint sequence target coordinate information that match the target user, and the knee joint sequence target coordinate information includes coordinate information corresponding to the knee joint sagittal sequence and coordinate information corresponding to the knee joint coronal proton density sequence; based on the knee joint sequence target coordinate information, the knee joint sagittal sequence and the knee joint coronal proton density sequence are processed to generate a knee joint image set, wherein the knee joint image set includes different knee joint image pairs. The corresponding meniscus position information and the target user's knee joint material parameter value are obtained; the training data set is preprocessed to generate a training data set with identification information, wherein the identification information is used to characterize the physiological factors affecting meniscus injury; the training data set with identification information is processed based on the preset processing rules to generate a training set and a validation set; an initial knee joint image fusion model matching the identification information is obtained, wherein the initial knee joint image fusion model is set based on the target user data received within a preset time period; the initial knee joint image fusion model is trained based on the training set and the validation set to generate a target knee joint image fusion model; the knee joint image set is processed based on the target knee joint image fusion model to generate the target user's knee joint wear information. The sagittal and coronal features are extracted and effectively fused at different spatial scales, and cross-sequence information fusion is realized at each scale to achieve more effective feature learning. The coronal and sagittal information of the MRI image is used for lesion identification, so that the model can make full use of information at different angles and scales, thereby improving the ability to accurately identify knee joint wear.

[0018] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 A flowchart of a method for fusion of knee joint image information based on coordinate association provided by an embodiment of the present application is shown;

[0020] Figure 2 A schematic structural diagram of a knee joint image information fusion device based on coordinate association provided by an embodiment of the present application is shown;

[0021] Figure 3 A schematic diagram of the structure of an electronic device provided by an embodiment of the present application is shown;

[0022] Figure 4 A schematic diagram of a storage medium provided in an embodiment of the present application is shown. DETAILED DESCRIPTION

[0023] The preferred embodiments of the present invention are described below in conjunction with the accompanying drawings. It should be understood that the preferred embodiments described herein are only used to illustrate and explain the present invention, and are not used to limit the present invention.

[0024] Combine the following Figure 1 To describe the coordinate association-based knee joint image information fusion method according to an exemplary embodiment of the present application. It should be noted that the following application scenarios are only shown to facilitate understanding of the spirit and principles of the present application, and the embodiments of the present application are not limited in this regard. On the contrary, the embodiments of the present application can be applied to any applicable scenario.

[0025] In one implementation, the present application also proposes a coordinate association-based knee joint image information fusion method and related equipment. Figure 1 The flowchart of a method for fusion of knee joint image information based on coordinate association according to an embodiment of the present application is schematically shown. Figure 1 As shown, the method is applied to a server, comprising:

[0026] S101, obtaining a training data set and target user data.

[0027] In one implementation, target user data is collected, including the patient's basic information, clinical diagnosis, surgical records, treatment process, rehabilitation status, etc. The present invention is not limited to this, and the applicant can select and set it according to actual needs.

[0028] The training data set is the knee joint information of other users that matches the target user data. The target user group is determined, such as patients with anterior cruciate ligament tears, fractures, and knee injuries. These users will be the focus of model training and analysis. Key information is extracted from the target user's electronic medical record, such as injury type, surgical details, postoperative recovery, complications, etc. Based on the target electronic medical record information, similar electronic medical record data is matched from a larger patient group to form a training data set.

[0029] S102: Process the target user data to generate attribute information of the target user.

[0030] In one embodiment, the attribute information of the target user includes a knee joint sagittal sequence, a knee joint coronal proton density sequence and a knee joint sequence target coordinate information that match the target user, and the knee joint sequence target coordinate information includes coordinate information corresponding to the knee joint sagittal sequence and coordinate information corresponding to the knee joint coronal proton density sequence.

[0031] The sagittal sequence is a standard view in MRI scans that provides images of the knee joint along the long axis of the body. This sequence is suitable for evaluating the anterior cruciate ligament, posterior cruciate ligament, meniscus, and articular cartilage of the knee. The coronal proton density (PD) sequence is another MRI scan view that provides images along the width of the knee joint. This sequence helps to observe the collateral ligaments, patella, and surrounding soft tissues of the knee joint. Target coordinate information refers to the coordinate points used to locate and analyze specific structures in MRI images. This coordinate information is crucial for accurately measuring and evaluating the anatomical structure and pathological changes of the knee joint.

[0032] The coordinate information corresponding to the sagittal sequence of the knee joint includes identifying and locating the coordinate points of the key structures of the knee joint in the sagittal image, such as specific areas of the ligaments, meniscus and cartilage; the coordinate information corresponding to the coronal proton density sequence of the knee joint includes identifying and locating the coordinate points of the key structures of the knee joint in the coronal image, such as specific areas of the collateral ligaments and patella.

[0033] Using this coordinate information, doctors and researchers can perform precise image analysis and measurements, such as measuring the thickness of the meniscus, assessing the degree of cartilage damage, or determining the integrity of the ligaments. These detailed images and coordinate information are essential for clinical diagnosis of knee diseases and planning treatments (such as surgery). They can help doctors better understand the patient's specific condition and develop a personalized treatment plan. With this detailed knee sequence target coordinate information, medical professionals can more accurately assess the condition of the knee joint and provide patients with more precise diagnoses and more effective treatment plans.

[0034] The target user's data is cleaned and processed to generate the target user's knee joint information. The collected target user's knee joint data is cleaned to remove invalid, erroneous or irrelevant information to ensure data quality. The cleaned data is used to generate the target user's knee joint basic information, including basic anatomical structure and possible injuries. The target user's knee joint information is processed based on the preset image extraction model to generate the target user's knee joint image information. The knee joint information is processed using the preset image extraction model to extract detailed images of the knee joint from MRI or CT scans. The extracted image information includes various components of the knee joint, such as bones, cartilage, ligaments and meniscus.

[0035] The target user's knee joint image information is processed to generate the target user's knee joint material parameter values. In the process of knee joint image analysis and material parameter quantification, the knee joint image is first acquired through high-quality MRI or CT scanning, followed by image preprocessing, including denoising and contrast enhancement, to lay the foundation for subsequent analysis. The analysis covers cartilage thickness measurement, bone density assessment, ligament size and morphology analysis, and detailed examination of the meniscus and joint space. In addition, the assessment of joint fluid and soft tissue damage, as well as joint mobility, are important components of the diagnostic process. Further, biomechanical modeling and multimodal image fusion technology can provide a more comprehensive understanding of the structure and function of the knee joint. The application of computer-aided diagnosis system improves the accuracy and efficiency of diagnosis. Finally, the quantified parameters are combined with clinical data to generate a detailed report to assist doctors in making diagnosis and treatment decisions.

[0036] Generate the target user's attribute information based on the target user's knee joint material parameter values. Generate the target user's detailed attribute information based on the knee joint material parameter values. This information is crucial for evaluating the health of the knee joint. Extract features from images and material parameters, such as the degree of cartilage degeneration, ligament damage, etc., and conduct in-depth analysis. Through this process, medical professionals can have a deeper understanding of the patient's knee joint condition, provide patients with more accurate diagnosis and personalized treatment, and improve the success rate of surgery and treatment. At the same time, this also provides a scientific basis for clinical decision-making in knee injury and disease management.

[0037] S103, processing a knee joint sagittal sequence and a knee joint coronal proton density sequence based on the knee joint sequence target coordinate information to generate a knee joint image set.

[0038] In one embodiment, the knee joint image set includes the meniscus position information corresponding to different knee joint images and the knee joint material parameter values ​​of the target user. In the process of knee joint image analysis, the sagittal and coronal proton density sequences are first accurately processed using the knee joint sequence target coordinate information to generate a detailed knee joint image set. This image set pays special attention to the position information of the meniscus, which is a key area for evaluating knee joint injuries. At the same time, the material parameter values ​​of the knee joint are extracted through image analysis, including cartilage thickness, bone density, and ligament size, which are crucial for personalized diagnosis and treatment plans.

[0039] Furthermore, these images and parameter values ​​are used to support clinical decision-making, such as the formulation of treatment plans and optimization of surgical designs. Integrating these data with other clinical information of the patient can form a comprehensive medical record, assisting doctors in continuous patient monitoring and treatment adjustments, providing a scientific basis for the diagnosis and treatment of knee diseases, and improving the quality and efficiency of medical services.

[0040] In another embodiment, in order to obtain the coordinates of each point in the sagittal sequence of the knee joint and the coronal proton density sequence of the knee joint in the real world, the applicant needs to use the following fields in the image d icom file:

[0041] Field Name effect InstanceNumber Indicates the number of images in a sequence. ImagePositionPatient Indicates the position of the upper left corner of the current image in real space ImageOrientationPatient Records the angle changes of the point coordinate system in the current image PixelSpacing Records the proportional relationship between the x, y coordinates in the current image and the real coordinates SpacingBetweenSlices Represents the distance between two images in real space

[0042] The applicant assumes that the pixel coordinate point in the current image is pixel[x,y], then its coordinates in the real space are:

[0043] real[x,y,z]=pixel[x,y]×ImageOrientationPatient×spacing+

[0044] ImagePositionPatienteal;

[0045] The multiplications in the formula are all matrix multiplications, where:

[0046]

[0047] S104, preprocessing the training data set to generate a training data set with identification information.

[0048] In one embodiment, feature extraction is performed on the training data set to determine the original feature library, and features are extracted from the training data set to form the original feature library, which includes anatomical parameters of the knee joint, movement habits, previous sports injuries, etc. The original feature library is used to divide the features into different data sets to generate a training data set for model training and a validation set for model evaluation. The classifier is used to divide the original feature library into each validation set for prediction to determine the prediction result; the training data set is trained using a preset algorithm to learn the relationship between the features and meniscus injury, and the prediction result of the validation set is obtained. An appropriate machine learning algorithm is selected, such as logistic regression, support vector machine, random forest or neural network, which can handle classification or regression tasks, and a training data set is prepared, including labeled examples, which contain various physiological and environmental features related to meniscus injury. The training data set is trained using a preset algorithm so that the model learns how to identify the pattern and trend of meniscus injury from the features.

[0049] Based on the prediction results and the prediction results of the validation set, training samples with identification information are generated. The identification information is used to identify and characterize the key physiological factors that may affect meniscus injury and provide important input features for the model. Identification information plays a core role in the knee joint image fusion model of meniscus injury. It provides key input features for the model by identifying key physiological factors. These factors include anatomical characteristics of the knee joint, sports mechanical load, biomarkers, age, gender, previous injury history, type of exercise, body shape, genetic factors, lifestyle, psychological state, environmental conditions and medical history. This comprehensive information helps to assess an individual's susceptibility to meniscus injury and risk of injury.

[0050] By analyzing this marker information, medical professionals can more accurately predict the likelihood of meniscus injury and develop personalized prevention measures and treatment plans for patients. For example, considering sports mechanics can reveal the risk of injury caused by overuse or improper sports techniques; while genetic factors and biomarkers may indicate differences in individual biological responses to injuries. In addition, lifestyle habits and psychological factors can also affect an individual's health status and risk of injury.

[0051] Integrating this multi-dimensional information into the knee image fusion model can not only improve the accuracy of the model, but also help doctors better understand the complexity of meniscus injuries, thereby providing more comprehensive clinical decision support. This personalized medical approach helps optimize patient treatment outcomes, reduce unnecessary interventions, and improve patient satisfaction with treatment and quality of life.

[0052] In another embodiment, the standard material information is processed based on preset processing rules to generate standardized features, wherein the standardized features are meniscus features with clear and known material composition. Specifically, standard material information is collected, including normal and abnormal MRI or CT scan images of the meniscus, and features related to meniscus injury and degeneration are selected, such as the shape, size, signal strength, position and relationship of the meniscus with other knee joint structures, and the selected features are quantified, for example, by measuring the thickness of the meniscus, evaluating signal uniformity or identifying potential tears and injuries, ensuring that the feature data is comparable between different samples and scans, and eliminating differences caused by different scanning conditions through standardization technology.

[0053] The extracted standardized features are stored in the feature library to provide a basis for subsequent model training and data analysis, verify the accuracy and reliability of the features to ensure that they can truly reflect the status of the meniscus, use the data in the feature library to train the machine learning model, and learn the relationship between the features and meniscus injury. According to professional knowledge and data characteristics, feature screening rules are formulated to identify the features that have the greatest impact on the prediction target. Dimensionality reduction techniques such as principal component analysis (PCA), linear discriminant analysis (LDA), or t-distributed random neighbor embedding (t-SNE) are applied to reduce the number of features while retaining important information. Combine the feature screening and dimensionality reduction results to generate a set of original features that can effectively represent the data and reduce complexity. The generated original features are collected to construct the original feature library to provide a basis for subsequent model training and data analysis. The original feature library is verified to ensure that the selected features can accurately reflect the research objectives, such as the characteristics of meniscus injury.

[0054] S105, processing the training data set with identification information based on preset processing rules to generate a training set and a verification set.

[0055] In one implementation, feature extraction is performed on a training data set with identification information to determine an original feature library; the original feature library is processed based on a neural network model to generate forged feature data, wherein the forged feature data is different from the feature data in the original feature library; the original feature library is divided to generate a training data set and a verification set, wherein the number of the training data set and the verification set is at least one, and the verification set includes the forged feature data. By generating forged feature data, preparation is made for subsequent verification of the initial knee joint image fusion model, that is, the forged feature data is actually abnormal data, but if the initial knee joint image fusion model does not identify the current data as abnormal, it indicates that the detection result of the current model is inaccurate.

[0056] S106, obtaining an initial knee joint image fusion model that matches the identification information.

[0057] In one embodiment, the initial knee joint image fusion model is set based on the target user data received within a preset time period, that is, the knee joint image fusion model is trained based on the screening of training samples related to the target user data to avoid excessive errors in the results generated by the model.

[0058] S107, training the initial knee joint image fusion model based on the training set and the validation set to generate a target knee joint image fusion model.

[0059] In one embodiment, multiple data groups are extracted from a training set, wherein each data group contains a preset number of data samples, wherein at least one data sample includes identification information. The initial knee joint image fusion model is trained based on the data samples in the multiple data groups to generate a trained knee joint image fusion model. In the process of developing an accurate knee joint image fusion model, firstly, a data group containing a preset number of samples is extracted from the training set to ensure that each group contains at least one sample with key identification information. These identification information may indicate a patient's specific physiological state or infection risk factors. Then, the initial knee joint image fusion model is systematically trained using these data groups to form a trained model.

[0060] The trained knee joint image fusion model is processed based on the validation set to generate validation results. Subsequently, the trained model is validated using an independent validation set to generate validation results. When analyzing the validation results, special attention is paid to samples with identification information to determine whether the model can accurately identify the physiological factors that affect meniscus injury. If the model can effectively predict abnormal states, that is, samples with abnormal physiological factors, then the trained model is considered to be the target knee joint image fusion model. If the data sample containing identification information in the validation results indicates that the physiological factors that affect postoperative joint infection are in an abnormal state, the trained knee joint image fusion model is used as the target knee joint image fusion model.

[0061] Use training set data to train machine learning models, such as classification models, regression models, or deep learning models, evaluate the performance of the model on the validation set, adjust model parameters and structure, and optimize them. Use the trained model to predict the risk of complications such as postoperative infection for target users, and provide doctors with personalized medical advice, such as preventive measures and treatment plans, based on the model's prediction results.

[0062] In addition, the optimization and iteration of the model is an ongoing process that requires continuous adjustment based on clinical feedback and verification results to improve the accuracy and practicality of the model. After the model is applied to clinical practice, its performance is continuously monitored and regularly evaluated to ensure that the model can adapt to changes in the medical environment. Throughout the process, it is crucial to maintain data privacy and ethical standards to ensure the security and compliance of patient data. Through this process, medical professionals can build an effective early warning system to help identify infection risks in advance, optimize preventive measures, improve the quality of postoperative recovery of patients, and reduce the burden on the medical system.

[0063] S108, processing the knee joint image set based on the target knee joint image fusion model to generate knee joint wear information of the target user.

[0064] In one embodiment, a knee joint image set is processed based on a target knee joint image fusion model to generate knee joint image coding features, the knee joint image set is processed using the target knee joint image fusion model to extract key visual information, and coding features are generated from the knee joint images, which can represent the morphological characteristics and potential signs of wear of the knee joint.

[0065] The encoded features of the knee joint image are processed based on the target joint wear warning model to generate association values ​​of different knee joint images, wherein the association values ​​of different knee joint images are used to characterize the data sharing information between different knee joint images at different perspectives, that is, to judge whether the corresponding knee joint image has the risk of infection from multiple perspectives; if the association value of the knee joint image is higher than the preset association value, it indicates that the knee joint of the target user is in an abnormal state.

[0066] First, the model processes knee joint images to generate encoded features that capture the morphological changes of the knee joint. Then, the correlation values ​​between different knee joint images are calculated to quantify the similarity and data sharing information between images. The preset correlation value threshold is used to determine whether the image correlation is abnormal. When the correlation value exceeds this threshold, it indicates that the target user's knee joint may be in an abnormal state, triggering an early warning signal to assist medical professionals in making further diagnosis and treatment decisions. In addition, the model also includes a mechanism for continuous monitoring and optimization based on feedback to ensure the accuracy of the prediction and the relevance of clinical applications.

[0067] Specifically, the connection between different knee joint images is strengthened by calculating the similarity between the features of two different perspectives. This can be achieved by calculating the inner product of two feature vectors to obtain a similarity score (i.e., association value). Then, by normalizing these scores, the attention weight of each feature vector to another perspective can be obtained. Assume that the applicant has two feature vectors, representing image features from different perspectives, denoted as Q and K, representing query and key respectively. The target joint wear warning model includes a calculation formula for obtaining the association values ​​of different knee joint images, and the calculation formula is Q=X·W Q ; K = X·W K ; V = X·W V ; Among them, Attention(Q,K) represents the association value of different knee joint images, d k represents the dimension of the feature vector, the softmax function is used to normalize the similarity score, K T represents the transpose of the key vector, V represents the value vector, which is used for weighted summation to generate the final attention feature representation, Q represents the query vector, K represents the key vector, and W Q , W K , WV are the weight matrices corresponding to them respectively, and X is the input sequence.

[0068] In the present application, a server obtains a training data set and target user data; performs data cleaning processing on the target user data to generate knee joint information of the target user; processes the knee joint information of the target user based on a preset image extraction model to generate knee joint image information of the target user; processes the knee joint image information of the target user to generate knee joint material parameter values ​​of the target user; generates attribute information of the target user based on the knee joint material parameter values ​​of the target user, wherein the attribute information of the target user includes a knee joint sagittal sequence, a knee joint coronal proton density sequence and knee joint sequence target coordinate information matching the target user, and the knee joint sequence target coordinate information includes coordinate information corresponding to the knee joint sagittal sequence and coordinate information corresponding to the knee joint coronal proton density sequence; processes the knee joint sagittal sequence and the knee joint coronal proton density sequence based on the knee joint sequence target coordinate information to generate a knee joint image set, wherein the knee joint image set includes meniscus position information corresponding to different knee joint images and the knee joint material parameter values ​​of the target user.

[0069] Based on the preset processing rules, the standard material information is processed to generate standardized features, wherein the standardized features are meniscus features with clear and known material composition; based on the preset feature screening and dimensionality reduction rules, the standardized features are processed to generate original features; based on several original features, an original feature library is generated; each feature data set is divided according to the original feature library to generate a training data set and a verification set; a classifier is used to predict each verification set divided by the original feature library to determine the prediction results; a preset algorithm is used to train each training data set divided in the original feature library to obtain a verification set class prediction result; based on the prediction results and the verification set class prediction results, a training sample with identification information is generated, wherein the identification information is used to characterize the physiological factors affecting meniscus injury; based on the preset processing rules, the training data with identification information is The method comprises the steps of: processing a set to generate a training set and a validation set; obtaining an initial knee joint image fusion model that matches the identification information, wherein the initial knee joint image fusion model is set based on the target user data received within a preset time period; extracting multiple data groups from the training set, wherein each data group contains a preset number of data samples, wherein at least one data sample includes identification information; training the initial knee joint image fusion model based on the data samples in the multiple data groups to generate a trained knee joint image fusion model; processing the trained knee joint image fusion model based on the validation set to generate a verification result; if the data sample containing the identification information in the verification result indicates that the physiological factor affecting the meniscus injury is in an abnormal state, then using the trained knee joint image fusion model as the target knee joint image fusion model.

[0070] The knee joint image set is processed based on the target knee joint image fusion model to generate knee joint image coding features; the knee joint image coding features are processed based on the target joint wear warning model to generate association values ​​of different knee joint images, wherein the association values ​​of different knee joint images are used to characterize data sharing information between different knee joint images at different viewing angles: if the association value of the knee joint image is higher than the preset association value, it indicates that the knee joint of the target user is in an abnormal state; the target joint wear warning model includes a calculation formula for obtaining the association values ​​of different knee joint images, and the calculation formula is: Q=X·W Q ; K = X·W K ; V = X·W V ; Among them, Attention(Q,K) represents the association value of different knee joint images, d k The softmax function is used to normalize the similarity score, K T represents the transpose of the key vector, V represents the value vector, Q represents the query vector, K represents the key vector, and W Q , W K , W V The weight matrices corresponding to them are respectively, and X is the input sequence. The sagittal and coronal features are extracted and effectively fused at different spatial scales, and cross-sequence information fusion is achieved at each scale to achieve more effective feature learning. The coronal and sagittal information of MRI images is used for lesion identification, so that the model can make full use of information at different angles and scales, thereby improving the ability to accurately identify knee joint wear.

[0071] In one embodiment, if Figure 2 As shown, the present application also provides a knee joint image information fusion device based on coordinate association, comprising:

[0072] The acquisition module 201 is used to acquire the training data set and the target user data; acquire the initial knee joint image fusion model that matches the identification information, wherein the initial knee joint image fusion model is set based on the target user data received within a preset time period;

[0073] The processing module 202 is used to process the target user data to generate attribute information of the target user, wherein the attribute information of the target user includes a knee joint sagittal sequence, a knee joint coronal proton density sequence and a knee joint sequence target coordinate information that matches the target user, and the knee joint sequence target coordinate information includes coordinate information corresponding to the knee joint sagittal sequence and coordinate information corresponding to the knee joint coronal proton density sequence; based on the knee joint sequence target coordinate information, the knee joint sagittal sequence and the knee joint coronal proton density sequence are processed to generate a knee joint image set, wherein the knee joint image set includes The invention discloses a method for preparing a knee joint image fusion model comprising meniscus position information corresponding to different knee joint images and knee joint material parameter values ​​of a target user; preprocessing the training data set to generate a training data set with identification information, wherein the identification information is used to characterize physiological factors affecting meniscus injury; processing the training data set with identification information based on preset processing rules to generate a training set and a verification set; training the initial knee joint image fusion model based on the training set and the verification set to generate a target knee joint image fusion model; processing the knee joint image set based on the target knee joint image fusion model to generate knee joint wear information of a target user.

[0074] The present application embodiment provides an electronic device, such as Figure 3 As shown, the electronic device 3 includes a first processor 300, a memory 301, a bus 302 and a communication interface 303. The first processor 300, the communication interface 303 and the memory 301 are connected via the bus 302. The memory 301 stores a computer program that can be run on the first processor 300. When the first processor 300 runs the computer program, the coordinate association-based knee joint image information fusion method provided in any of the aforementioned embodiments of the present application is executed.

[0075] The memory 301 may include a high-speed random access memory (RAM), and may also include a non-volatile memory, such as at least one disk memory. The communication connection between the system network element and at least one other network element is realized through at least one communication interface 303 (which may be wired or wireless), and the Internet, wide area network, local area network, metropolitan area network, etc. may be used.

[0076] The bus 302 may be an ISA bus, a PCI bus, or an EISA bus, etc. The bus may be divided into an address bus, a data bus, a control bus, etc. The memory 301 is used to store a program, and the first processor 300 executes the program after receiving an execution instruction. The coordinate-association-based knee joint image information fusion method disclosed in any implementation of the embodiment of the present application may be applied to the first processor 300, or implemented by the first processor 300.

[0077] The first processor 300 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the hardware integrated logic circuit or software instructions in the first processor 300. The above-mentioned first processor 300 can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a readily available programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The methods, steps and logic block diagrams disclosed in the embodiments of the present application can be implemented or executed. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. The steps of the method disclosed in the embodiments of the present application can be embodied as a hardware decoding processor to be executed, or a combination of hardware and software modules in the decoding processor to be executed. The software module can be located in a mature storage medium in the field such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory 301, and the first processor 300 reads the information in the memory 301 and completes the steps of the above method in combination with its hardware.

[0078] The electronic device provided by the above-mentioned embodiments of the present application and the coordinate association-based knee joint image information fusion method provided by the embodiments of the present application are based on the same inventive concept and have the same beneficial effects as the methods adopted, run or implemented by the application programs stored therein.

[0079] The present application embodiment provides a computer-readable storage medium, such as Figure 4 As shown, the computer-readable storage medium stores 401 a computer program, and when the computer program is read and executed by the second processor 402, the above-mentioned coordinate association-based knee joint image information fusion method is implemented.

[0080] The computer-readable storage medium provided in the above-mentioned embodiments of the present application and the coordinate association-based knee joint image information fusion method provided in the embodiments of the present application are based on the same inventive concept and have the same beneficial effects as the methods adopted, run or implemented by the application programs stored therein.

[0081] An embodiment of the present application provides a computer program product, including a computer program, wherein the computer program is executed by a third processor to implement the method described above.

[0082] The computer program product provided by the above-mentioned embodiments of the present application and the coordinate association-based knee joint image information fusion method provided by the embodiments of the present application are based on the same inventive concept and have the same beneficial effects as the methods adopted, run or implemented by the application programs stored therein.

[0083] It should be noted that, in this application, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or still includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise one..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0084] Each embodiment in the present application is described in a related manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the evaluation of the coordinate-association-based knee joint image information fusion method, electronic device, electronic device, and readable storage medium embodiment, since it is basically similar to the above-mentioned coordinate-association-based knee joint image information fusion method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the coordinate-association-based knee joint image information fusion method embodiment.

[0085] Although the present application is disclosed as above, the present application is not limited thereto. Any person skilled in the art can make various changes and modifications without departing from the spirit and scope of the present application. Therefore, the protection scope of the present application shall be subject to the scope defined by the claims.

Claims

1. A method for knee joint image information fusion based on coordinate association, characterized in that: include: Obtain training data sets and target user data; Processing the target user data to generate attribute information of the target user, wherein the attribute information of the target user includes a knee joint sagittal sequence, a knee joint coronal proton density sequence and knee joint sequence target coordinate information that match the target user, and the knee joint sequence target coordinate information includes coordinate information corresponding to the knee joint sagittal sequence and coordinate information corresponding to the knee joint coronal proton density sequence; Processing the knee joint sagittal sequence and the knee joint coronal proton density sequence based on the knee joint sequence target coordinate information to generate a knee joint image set, wherein the knee joint image set includes meniscus position information corresponding to different knee joint images and knee joint material parameter values ​​of the target user; Preprocessing the training data set to generate a training data set with identification information, wherein the identification information is used to characterize physiological factors that affect meniscus injury; Processing the training data set with identification information based on preset processing rules to generate a training set and a verification set; Acquire an initial knee joint image fusion model that matches the identification information, wherein the initial knee joint image fusion model is set based on target user data received within a preset time period; Training the initial knee joint image fusion model based on the training set and the validation set to generate a target knee joint image fusion model; The knee joint image set is processed based on the target knee joint image fusion model to generate knee joint wear information of the target user.

2. The method according to claim 1, characterized in that The target user data is processed to generate attribute information of the target user, including: Performing data cleaning processing on the target user data to generate knee joint information of the target user; Processing the target user's knee joint information based on a preset image extraction model to generate the target user's knee joint image information; Processing the knee joint image information of the target user to generate a knee joint material parameter value of the target user; The attribute information of the target user is generated based on the knee joint material parameter value of the target user.

3. The method according to claim 1, characterized in that Preprocessing the training data set to generate a training data set with identification information includes: Extracting features from the training data set to determine an original feature library; Divide each feature data set according to the original feature library to generate a training data set and a verification set; Use the classifier to divide the original feature library into various verification sets for prediction and determine the prediction results; Use the preset algorithm to divide each training data set in the original feature library for training, and obtain the prediction results of the validation set class; Based on the prediction results and the prediction results of the validation set, training samples with identification information are generated.

4. The method according to claim 3, characterized in that Performing feature extraction on the training data set to determine an original feature library includes: Processing the standard material information based on preset processing rules to generate standardized features, wherein the standardized features are meniscus features with clear and known material composition; Processing the standardized features based on preset feature screening and dimension reduction rules to generate original features; A primitive feature library is generated based on a number of primitive features.

5. The method according to claim 3, characterized in that The initial knee joint image fusion model is trained based on the training set and the validation set to generate a target knee joint image fusion model, including: Extracting a plurality of data groups from the training set, wherein each data group includes a preset number of data samples, wherein at least one data sample includes identification information; Training the initial knee joint image fusion model based on the data samples in the plurality of data groups to generate a trained knee joint image fusion model; Processing the trained knee joint image fusion model based on the verification set to generate a verification result; If the data sample containing identification information in the verification result indicates that the physiological factors affecting meniscus injury are in an abnormal state, the trained knee joint image fusion model is used as the target knee joint image fusion model.

6. The method according to claim 2, characterized in that The knee joint image set is processed based on the target knee joint image fusion model to generate knee joint wear information of the target user, including: Processing the knee joint image set based on the target knee joint image fusion model to generate knee joint image coding features; The knee joint image coding features are processed based on the target joint wear warning model to generate association values ​​of different knee joint images, wherein the association values ​​of different knee joint images are used to characterize data sharing information between different knee joint images at different viewing angles: If the correlation value of the knee joint image is higher than a preset correlation value, it indicates that the knee joint of the target user is in an abnormal state.

7. The method according to claim 6, characterized in that The knee joint image set is processed based on the target knee joint image fusion model to generate knee joint wear information of the target user, further comprising: The target joint wear warning model includes a calculation formula for obtaining the correlation value of different knee joint images, and the calculation formula is: Q=X·W Q ; K=X·W K ; V=X·W V ; Among them, Attention(Q,K) represents the association value of different knee joint images, d k The softmax function is used to normalize the similarity score, K T represents the transpose of the key vector, V represents the value vector, Q represents the query vector, K represents the key vector, and W Q , W K , W V are the weight matrices corresponding to them respectively, and X is the input sequence.

8. A knee joint image information fusion device based on coordinate association, characterized in that: The device comprises: An acquisition module, used to acquire a training data set and target user data; acquire an initial knee joint image fusion model that matches the identification information, wherein the initial knee joint image fusion model is set based on the target user data received within a preset time period; A processing module is used to process the target user data to generate attribute information of the target user, wherein the attribute information of the target user includes a knee joint sagittal sequence, a knee joint coronal proton density sequence and a knee joint sequence target coordinate information that matches the target user, and the knee joint sequence target coordinate information includes coordinate information corresponding to the knee joint sagittal sequence and coordinate information corresponding to the knee joint coronal proton density sequence; based on the knee joint sequence target coordinate information, the knee joint sagittal sequence and the knee joint coronal proton density sequence are processed to generate a knee joint image set, wherein the knee joint image set includes meniscus position information corresponding to the knee joint image and the knee joint material parameter values ​​of the target user; preprocessing the training data set to generate a training data set with identification information, wherein the identification information is used to characterize the physiological factors affecting meniscus injury; processing the training data set with identification information based on preset processing rules to generate a training set and a verification set; training the initial knee joint image fusion model based on the training set and the verification set to generate a target knee joint image fusion model; processing the knee joint image set based on the target knee joint image fusion model to generate knee joint wear information of the target user.

9. An electronic device, characterized in that: include: a first processor; and a memory for storing executable instructions of the first processor; Wherein, the first processor is configured to execute the coordinate association-based knee joint image information fusion method described in any one of claims 1 to 7 by executing the executable instructions.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by the second processor, the coordinate association-based knee joint image information fusion method described in any one of claims 1 to 7 is implemented.

Images