Hip protection treatment auxiliary method and device and readable storage medium

Abstract

The embodiment of the invention provides a hip protection treatment auxiliary method and device and a readable storage medium. The method comprises the following steps: converting one or more MRI image data for a hip joint region into a group of 3D voxels; using a convolutional neural network to classify and position the group of 3D voxels and identify the necrosis degree of the femoral head; and performing image overlapping display on the MRI image data and the necrosis degree of the femoral head.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of communication technologies, and particularly to a hip-preserving treatment assistance method, device and readable storage medium. Background Art

[0002] According to statistics, there are currently about 10 million patients with osteonecrosis of the femoral head in China, and 200,000 - 300,000 new cases are added every year. In order to retain the patient's natural hip joint, clinical decisions on hip-preserving treatment are often based on imaging results.

[0003] Currently, the decision-making for hip-preserving treatment relies on the results of magnetic resonance imaging (MRI) examinations. However, these examination methods have a series of problems and are difficult to provide complete and accurate information for clinicians. First, due to the influence of resolution and noise, early osteonecrosis of the femoral head or minor changes are usually easily overlooked in image interpretation. Second, the subjectivity of manual decision-making is strong, which is prone to cause diagnostic errors. Finally, in clinical decision-making, doctors usually face multiple MRI images, which often leads to low diagnostic efficiency. Summary of the Invention

[0004] The embodiments of the present application aim to provide a hip-preserving treatment assistance method, device and readable storage medium to solve the problem of how to improve the efficiency of hip-preserving treatment.

[0005] In a first aspect, a hip-preserving treatment assistance method is provided, including:

[0006] Converting one or more MRI image data for the hip joint region into a set of 3D voxels;

[0007] Using a convolutional neural network to classify, locate and identify the degree of necrosis of the femoral head for a set of 3D voxels;

[0008] Performing image overlay display on the MRI image data and the degree of necrosis of the femoral head.

[0009] Optionally, converting one or more MRI image data for the hip joint region into a set of 3D voxels includes:

[0010] Performing segmentation processing on the MRI image to obtain MRI image data for the hip joint region;

[0011] Performing denoising processing on the MRI image data for the hip joint region by a denoising method using wavelet transform;

[0012] Converting the denoised MRI image data for the hip joint region into a set of 3D voxels.

[0013] Optionally, perform segmentation processing on the MRI image to obtain MRI image data of the hip joint region, including:

[0014] Perform preliminary segmentation on the MRI image through a pre-trained neural network model to obtain a hip joint region image sub-array, and the pre-trained neural network model is used to extract the hip joint-related region from the MRI image;

[0015] Perform deep wavelet auto-encoding on the segmented hip joint region image sub-array to obtain the final MRI image data of the hip joint region.

[0016] Optionally, the denoising method of wavelet transform includes:

[0017] Provide M original image data sets from the same object, where M is greater than or equal to 1;

[0018] Correct the potential misalignment of the image data and perform consistent interpolation to avoid changes in noise statistics in the image series;

[0019] If the signal phase shows a consistent time evolution related to the preparation of the rotation system in the image series, optionally apply approximate phase correction in the image space;

[0020] In the image space, calculate the average signal intensity of each voxel in the image series;

[0021] Generate a mask that contains the voxels with the highest average signal intensity as a representative sample for statistical evaluation;

[0022] Pass the image data set and the phase standard deviation of the mask voxels in the image series to a processing pipeline running in the Fourier domain.

[0023] Optionally, use a convolutional neural network to classify, locate and identify the degree of necrosis of the femoral head for a group of 3D voxels, including:

[0024] Use a convolutional neural network with the AlexNet architecture, an MRI training data set, and object detection technology to classify, locate and identify the degree of necrosis of the femoral head for a group of 3D voxels.

[0025] Optionally, the MRI training data set is generated in the following way:

[0026] Receive MRI image data containing the femoral head region;

[0027] Divide the MRI image data into a group of 3D voxels;

[0028] Process this MRI image data and generate one or more 3D layers;

[0029] For each 3D layer, highlight the necrotic region and provide a necrosis category;

[0030] Generate a dataset containing the spatial coordinates of each highlighted region and the necrosis category label associated therewith;

[0031] Process the dataset and generate a file containing spatial coordinates and labels with the same resolution as the 3D voxel set;

[0032] Generate a label file for the 3D voxel set, where each such voxel contains 0 or 1 for each necrosis category;

[0033] Provide data for the computational model using a set of 3D voxels and the label file for each necrosis category, which label file contains an indication of 0 or 1 as to whether the voxel contains such necrosis for each corresponding voxel of the MRI image data.

[0034] Optionally, the process of the object detection technique includes:

[0035] Generate candidate regions using the methods of selective search and region proposal network, where the region contains an object;

[0036] Divide the image of the candidate region into predefined grids and predict multiple bounding boxes and their confidences for each grid;

[0037] Predefine a set of bounding boxes at different spatial positions, scales, and aspect ratios for predicting the adjustment amount of the true bounding box that best matches them;

[0038] If multiple bounding boxes overlap and contain the same object, retain the bounding box with the highest classification score and delete the other overlapping bounding boxes;

[0039] Output a set of bounding boxes and their corresponding class labels and classification scores, representing the objects detected in the image.

[0040] Optionally, the image superposition display of the MRI image data and the necrosis degree of the femoral head includes:

[0041] Generate a necrosis degree map by multiplying the MRI image by the binary mask of the necrotic tissue corresponding to the necrosis degree of the femoral head;

[0042] Adopt the method of image superposition to superpose the necrosis degree map on the MRI image.

[0043] In a second aspect, there is provided an auxiliary device for hip-preserving treatment, characterized by comprising:

[0044] An image preprocessing module for converting one or more MRI image data for the hip joint region into a set of 3D voxels;

[0045] A voxel classification module, configured to classify a set of 3D voxels by using a convolutional neural network, and locate and identify the degree of necrosis of the femoral head;

[0046] A display module, configured to perform image overlay display on the MRI image data and the degree of necrosis of the femoral head.

[0047] In a third aspect, a readable storage medium is provided, on which a program or instructions are stored, and when the program or instructions are executed by a processor, the steps of the method described in the first aspect are implemented.

[0048] In this application, through image preprocessing technology, the image can be protected from noise interference and the resolution can be provided, laying a foundation for subsequent image analysis. Secondly, by combining the convolutional neural network with the target detection technology, the system can accurately determine whether a patient is suitable for hip-preserving treatment. At the same time, the MRI intelligent decision-making system will provide a simple and intuitive interface, which can quickly display key information for doctors and improve the efficiency of clinical decision-making. Description of the Drawings

[0049] By reading the following detailed description of the preferred embodiments, various other advantages and benefits will become clear to those of ordinary skill in the art. The drawings are only for the purpose of showing the preferred embodiments and are not considered to be a limitation of this application. Moreover, throughout the drawings, the same reference numerals are used to represent the same components. In the drawings:

[0050] Figure 1 is a flowchart of the hip-preserving treatment assistance method provided by an embodiment of this application;

[0051] Figure 2 is a structural diagram of the hip-preserving treatment assistance device provided by an embodiment of this application;

[0052] Figure 3 is a flowchart of the image segmentation method in the image preprocessing module provided by an embodiment of this application;

[0053] Figure 4 is an overview diagram of the denoising autoencoder;

[0054] Figure 5 is a schematic diagram of the architecture of the convolutional neural network;

[0055] Figure 6 is a schematic diagram of the process of the convolutional layer in voxel analysis. Detailed Embodiments

[0056] The following will clearly and completely describe the technical solutions in the embodiments of the present application with reference to the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are some, but not all, of the embodiments of the present application. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present application without creative efforts shall fall within the protection scope of the present application.

[0057] The term "including" and any variations thereof in the specification and claims of the present application are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that includes a series of steps or units does not necessarily have to be limited to those steps or units clearly listed, but may include other steps or units not clearly listed or inherent to these processes, methods, products, or devices. In addition, the use of "and / or" in the specification and claims means at least one of the connected objects. For example, A and / or B means including three cases: A alone, B alone, and both A and B exist.

[0058] In the embodiments of the present application, words such as "exemplary" or "for example" are used to represent examples, illustrations, or explanations. Any embodiment or design solution described as "exemplary" or "for example" in the embodiments of the present application should not be construed as being more preferred or having more advantages than other embodiments or design solutions. Rather, the use of words such as "exemplary" or "for example" is intended to present relevant concepts in a specific manner.

[0059] See Figure 1 , the embodiments of the present application provide an auxiliary method for hip preservation treatment, which is applied to an MRI intelligent decision-making system. The specific steps include: step 101 and step 102.

[0060] Step 101: Convert one or more MRI image data for the hip joint region into a set of 3D voxels;

[0061] Step 102: Use a convolutional neural network to classify, locate, and identify the degree of necrosis of the femoral head for a set of 3D voxels;

[0062] Optionally, in another implementation, while locating and identifying the degree of necrosis of the femoral head, treatment suggestions can be generated.

[0063] Step 103: Perform image overlay display on the MRI image data and the degree of necrosis of the femoral head.

[0064] In one implementation of the present application, dividing one or more MRI image data for the hip joint region into a set of 3D voxels includes:

[0065] Perform segmentation processing on the MRI image to obtain MRI image data for the hip joint region;

[0066] A denoising method based on wavelet transform is used to denoise the MRI image data of the hip joint region;

[0067] The denoised MRI image data of the hip joint region is converted into a set of 3D voxels.

[0068] In one embodiment of the present application, the MRI image is segmented to obtain the MRI image data of the hip joint region, including:

[0069] The MRI image is preliminarily segmented by a pre-trained neural network model to obtain a hip joint region image sub-array, and the pre-trained neural network model is used to extract the hip joint-related region from the MRI image;

[0070] The segmented hip joint region image sub-array is subjected to deep wavelet auto-encoding to obtain the final MRI image data of the hip joint region.

[0071] In one embodiment of the present application, the denoising method based on wavelet transform includes:

[0072] Provide M original image data sets from the same object, where M is greater than or equal to 1;

[0073] Correct the potential misalignment of the image data and perform consistent interpolation to avoid changes in noise statistics in the image series;

[0074] Optionally, apply approximate phase correction in the image space if the signal phase shows a consistent temporal evolution related to the preparation of the rotation system in the image series;

[0075] In the image space, calculate the average signal intensity of each voxel in the image series;

[0076] Generate a mask that contains the voxels with the highest average signal intensity as a representative sample for statistical evaluation;

[0077] Transfer the image data set and the phase standard deviation of the masked voxels in the image series to a processing pipeline running in the Fourier domain.

[0078] In one embodiment of the present application, a convolutional neural network is used to classify, locate and identify the necrosis degree of the femoral head for a set of 3D voxels, including:

[0079] Use a convolutional neural network with the AlexNet architecture and an MRI training data set to classify, locate and identify the necrosis degree of the femoral head for a set of 3D voxels.

[0080] In one embodiment of the present application, the process of the object detection technology includes:

[0081] Use the method of selective search and region proposal network to generate candidate regions, where the region contains an object;

[0082] Divide the image of the candidate region into predefined grids, and predict multiple bounding boxes and their confidences for each grid;

[0083] Predefine a set of bounding boxes at different spatial positions, scales and aspect ratios for predicting the adjustment amount of the true bounding box that best matches them;

[0084] If multiple bounding boxes overlap and contain the same object, retain the bounding box with the highest classification score and delete other overlapping bounding boxes;

[0085] Output a set of bounding boxes and their corresponding class labels and classification scores, indicating the objects detected in the image.

[0086] In one embodiment of the present application, the MRI training dataset is generated by the following method:

[0087] Receive MRI image data containing the femoral head region;

[0088] Divide the MRI image data into a set of 3D voxels;

[0089] Process this MRI image data and generate one or more 3D layers;

[0090] For each 3D layer, highlight the necrotic region and provide a necrosis category;

[0091] Generate a dataset containing the spatial coordinates of each highlighted region and the necrosis category label associated therewith;

[0092] Process the dataset and generate a file containing spatial coordinates and labels with the same resolution as the set of 3D voxels;

[0093] Generate a label file for the set of 3D voxels, where each such voxel contains 0 or 1 for each necrosis category;

[0094] Use a set of 3D voxels and the label file for each necrosis category to provide data for the computational model, which for each corresponding voxel of the MRI image data contains an indication of 0 or 1 whether the voxel contains such necrosis.

[0095] In one embodiment of the present application, the MRI image data is superimposed and displayed with the necrosis degree of the femoral head, including:

[0096] Generate a necrosis degree map by multiplying the MRI image by the binary mask of the necrotic tissue corresponding to the necrosis degree of the femoral head;

[0097] The method of image superposition is adopted to superpose the necrosis degree map on the MRI image.

[0098] In this application, through wavelet transform technology and image segmentation algorithms, image preprocessing provides clearer and more discriminative image data for subsequent feature extraction and classification. Through convolutional neural network and object detection technologies, the present invention improves the analysis efficiency of medical image images and reduces the error of artificial decision-making. Through image superposition technology and 3D display technology, the present invention enables doctors to interpret complex medical data more intuitively and comprehensively.

[0099] This application also provides an auxiliary method for hip-preserving treatment, and the specific steps include:

[0100] Step 201: Use image preprocessing technology to provide a clearer data basis for subsequent analysis and processing

[0101] The image preprocessing module is used to receive and process one or more MRI image data sets of the subject, which are for the hip joint area, and divide the MRI image data set into a group of 3D voxels.

[0102] To solve the problem of noise interference caused by instruments or other factors, in this embodiment, wavelet transform technology is used to perform noise reduction processing on the image. To remove irrelevant backgrounds or structures in the image and focus on the femoral head area, in this embodiment, image segmentation algorithms are used, and at the same time, adaptive histogram equalization technology is used to enhance the details in the image and make the necrosis area more obvious.

[0103] The specific process is as follows:

[0104] Step 1: Provide multiple complex MRI image data sets.

[0105] Step 2: Use wavelet transform to perform waveform decomposition to generate a waveform coefficient data set representing the MRI image.

[0106] Step 3: Denoise in the waveform frequency domain;

[0107] The specific steps are as follows:

[0108] a: Calculate the normalized coefficient data set of the waveform coefficient through statistical amplitude processing.

[0109] b: Perform statistical phase processing to calculate the phase difference mapping in the coefficient data set.

[0110] c: Apply a specific transfer function to the coefficient data set to calculate the rescaled coefficient data set. Perform waveform reconstruction to obtain the denoised MRI image.

[0111] Step 4: Use an image segmentation algorithm to extract regions from the MRI image, focusing on the femoral head.

[0112] Step 1015: Enhance the image using adaptive histogram equalization technology to make the details of the necrotic region more prominent.

[0113] Step 202: Use convolutional neural networks and object detection technology to improve the accuracy of clinical decision-making.

[0114] To solve the problem of large errors in clinical manual decision-making, this embodiment designs a voxel classification module using a convolutional neural network and object detection technology based on the AlexNet architecture, aiming to automatically and accurately identify and quantify the degree of necrosis of the femoral head. Each layer of the CNN learns to detect various images. Image processing is applied to process images of different resolutions, and the output of each image is processed and used as the input for the next layer. The architecture of the CNN can be divided into feature extraction and classification. In the feature extraction part, there are three operation stages, namely convolution operation, rectified linear unit (ReLU), and pooling. The classification part also has two operation stages, namely fully connected and activation function. In addition, the present invention will also combine object detection technology to determine the exact position of the object, and the specific steps are as follows:

[0115] Step 1: Image input;

[0116] The input image needs to be preprocessed by methods such as normalization and scaling to adapt to a specific model architecture.

[0117] Step 2: Use the CNN for feature extraction;

[0118] The specific method is as follows:

[0119] a. Convolutional layer: Use multiple convolutional layers to extract spatial features from the image. This is done by applying filters that can identify low-level features such as edges and textures in the image.

[0120] b. Pooling layer: Reduce the computational amount by reducing the spatial dimension of the feature map.

[0121] c. Activation function: Such as ReLU, used to increase the non-linearity of the model.

[0122] Step 3: Region proposal;

[0123] Generate candidate regions: Use methods such as selective search and region proposal network (RPN) to generate candidate regions that contain an object.

[0124] Step 4: YOLO (You Only Look Once) algorithm, that is, perform object detection and location regression simultaneously in one forward pass;

[0125] The specific methods are as follows:

[0126] a. Direct regression: YOLO divides the image into an SxS grid and predicts multiple bounding boxes and their confidences for each grid.

[0127] b. Class prediction: Simultaneously predicts the probabilities of C classes for each bounding box.

[0128] c. Overall prediction: It is the product of the confidence of the bounding box and the class probability.

[0129] Step 5: Single Shot MultiBox Detector (SSD);

[0130] a. Multi-scale feature maps: SSD makes predictions on feature maps at multiple scales, allowing it to detect objects of various sizes.

[0131] b. Anchor boxes: Uses multiple predefined anchor boxes for prediction at each feature map location.

[0132] c. Bounding box adjustment and class prediction: Similar to YOLO, predicts the adjustment of the bounding box and the class probability for each anchor box.

[0133] Step 6: Anchor Boxes;

[0134] The specific methods are as follows:

[0135] a. Predefine a set of bounding boxes at different spatial positions, scales, and aspect ratios.

[0136] b. Used to predict the adjustment amount of the ground truth bounding box that best matches it.

[0137] Step 7: Non-Maximum Suppression (NMS);

[0138] The specific methods are as follows:

[0139] a. Duplicate removal: If multiple bounding boxes overlap and contain the same object, NMS will retain the bounding box with the highest classification score and delete other overlapping bounding boxes.

[0140] b. Confidence threshold: Only retain those bounding boxes whose classification scores exceed a certain threshold.

[0141] Step 8: Output.

[0142] Finally, the model outputs a set of bounding boxes and their corresponding class labels and classification scores, representing the objects detected in the image.

[0143] Compared with traditional manual analysis methods, this model can ensure the most accurate and reliable diagnosis of the degree of necrosis provided to doctors and patients, thereby reducing the risk of human judgment errors.

[0144] Step 203: Use 3D display technology to provide more intuitive and visual information for clinical decision-making.

[0145] To solve the problem of decision-making efficiency of MRI imaging, this embodiment uses 3D display technology to quickly display key information for doctors through a simple and intuitive interface.

[0146] The specific process is as follows:

[0147] a. Image overlay: Merge and overlay images from different data sources through a specific algorithm. The purpose of this step is to overlay images from multiple data sources on the same 3D model to form a complete view.

[0148] b. 3D modeling: Convert the overlaid images into a 3D model. At this time, virtual colors, textures, and lighting effects can be added as needed to enhance the realism and layering of the images.

[0149] c. Real-time rendering: Use a high-performance graphics processor to perform real-time rendering on the 3D model to ensure that doctors can smoothly rotate, zoom in, zoom out, or move the model to obtain the best viewing angle.

[0150] 4. Interactive interface design: To ensure that both doctors and patients can use the system conveniently, design an intuitive interactive interface. The interface should include basic functions such as rotation, zoom in, zoom out, and slice viewing.

[0151] 5. Continuous monitoring and update: The system should have the ability to continuously monitor, be able to receive new medical data in real time and update the 3D model.

[0152] See Figure 2 This application embodiment provides a hip-preserving treatment assistance device including an image preprocessing module, a voxel classification module, and a display module. The image preprocessing module 201 is used to denoise and segment the MRI image using wavelet transform and image segmentation algorithms to provide a clearer image for subsequent voxel classification. The voxel classification module is used to accurately locate and identify the degree of femoral head necrosis using convolutional neural network and object detection technology to determine whether the patient needs hip-preserving treatment. The display module is used to fuse different image information based on image overlay technology and 3D display technology, visualize important data points in a single view, and at the same time provide more intuitive result data for clinical staff to improve the efficiency of decision-making.

[0153] (1) Image preprocessing module

[0154] The described image preprocessing module uses wavelet transform and image segmentation algorithms to denoise and enhance hip joint MRI images, providing clearer and more discriminative image data for subsequent feature extraction and classification, and a clearer data basis for subsequent analysis and processing.

[0155] First, to better identify the pelvic region, this embodiment uses a segmentation method based on classification techniques, such as using a supervised learning method. The innovation of this embodiment lies in generating segmentation label predictions by training a neural network with loose annotation of the hip joint region. Through the above method, the calculation model is trained only using the voxels within the segmented hip joint region in the preliminary segmentation step. This not only reduces memory usage during training but also improves the accuracy of the system.

[0156] As Figure 3 shown, the specific process is as follows:

[0157] 1. Image preprocessing conforming to the Digital Imaging and Communications in Medicine (DICOM) standard:

[0158] Process DICOM - formatted medical images and extract the required image data.

[0159] 2. Image sub - array segmentation:

[0160] Since the original MRI image is large in size, it can be segmented into small sub - arrays for subsequent processing.

[0161] 3. Pre - segmentation of the hip joint region:

[0162] Use a pre - trained neural network model to perform preliminary segmentation on the image. This model is trained through a supervised learning method to extract regions related to the pelvis and hip joint, reducing the amount of data to be processed in subsequent operations.

[0163] 4. Application of the Deep Wavelet Auto - Encoder (DWA):

[0164] Perform deep wavelet auto - encoding on the segmented hip joint region image sub - arrays to further refine the segmentation results. Among them, Xi represents the input, hi represents the hidden layer, bi represents the bias, fi represents the activation function, and Wi represents the weight matrix used to connect the input Xi and the hidden layer hi. The specific formula is as follows:

[0165] h i =f i (W i X i +b i ), i = 1, 2, 3, 4

[0166] 5. Segmentation refinement:

[0167] a. With approximate images, these are the encoded approximate images obtained by the deep wavelet autoencoder for further neural network segmentation.

[0168] b. Utilize the autoencoder, especially the sparse autoencoder, to further refine and optimize the segmentation effect.

[0169] c. In the field of mathematics, the basic sparse autoencoder consists of a single hidden layer. The activation values of the hidden layer are represented as X, and this hidden layer is connected to the input vector v through the weight matrix W. Wv is the product of the weight matrix W and the input vector v, and the bias is represented as s. This is usually referred to as the encoding step. The output is the vector generated from the hidden layer, and this vector is reconstructed as v', which uses a new weight matrix Wt, the activation function is represented as f, and bi represents the bias term.

[0170] The specific formula is as follows:

[0171] X = f(W v + s)

[0172] v ′ = f(W t X + b ′ )

[0173] 6. Details processing of the deep wavelet autoencoder (DWA):

[0174] a. Perform discrete wavelet transform (DWT) on the encoded image of the hip region to further refine the segmentation result.

[0175] b. Use the Daubechies 2 mother wavelet for processing to generate approximation and detail coefficients.

[0176] c. Further optimize the segmentation effect according to these coefficients.

[0177] 7. Result output: According to all the processing steps, obtain the final segmentation result of the hip region and perform subsequent applications and analyses as needed.

[0178] Next, this embodiment uses the denoising method of wavelet transform to process the image for noise, providing a basis for subsequent image analysis. As Figure 4 shown, the specific steps are as follows:

[0179] 1. First, provide M original image datasets (M original images) from the same object. For example, use an MRI device to measure and provide complex data in the form of amplitude and phase.

[0180] 2. Correct the potential misalignment of the image data and perform consistent interpolation to avoid changes in noise statistics in the image series.

[0181] 3. If the signal phase shows a consistent temporal evolution related to the preparation of the rotation system in the image series, approximately phase correction can be optionally applied in the image space. This phase correction is to avoid, during the averaging effect in the method of the present invention, the information part other than the random noise being destructively interfered. The phase correction has no effect on the noise statistics. The parameters of the correction scheme to be applied are stored in the memory and used for the inverse phase operation after the reconstruction step. Beneficially, all voxels have a consistent phase after the phase correction. This simplifies step 5, reducing it to only calculating the standard deviation of the masked voxels. In addition, smooth phase characteristics are obtained in the object, thus avoiding the phase oscillation in the object domain being misinterpreted as a structure in the Boer domain.

[0182] 4. In the image space, calculate the average signal intensity of each voxel in the image series. During this process, in order to provide better performance and fast training, noise is artificially added to each layer. As Figure 3 shown, the denoising autoencoder is an extension of the standard autoencoder and is introduced as the basis of the deep network. The specific steps are as follows:

[0183] a. Data injection of noise: In order to simulate the data corruption or damage that may occur in the real scenario, noise is artificially added to the data in each layer.

[0184] b. Reconstruction and training: Design the denoising autoencoder to recover the original data from the data added with noise, rather than simply copying the data. This enables the model to learn and identify the robust features in the data.

[0185] c. Design of the model structure: Compared with the traditional autoencoder, the denoising autoencoder not only encodes the input data, but also tries to eliminate the noise in the input data.

[0186] d. Multiple training strategies: The model can be trained by artificially corrupting the data set and inputting it into the neural network, or simulating data corruption, such as deleting part of the information, to train the model to predict the lost data.

[0187] e. Deep learning optimization: In order to further optimize the performance, deep iterative learning can be carried out by stacking multiple denoising autoencoders.

[0188] 5. Generate a mask that contains the voxels with the highest average signal intensity as a representative sample for statistical evaluation. The percentage of the voxels included in this mask (for example, selected in the range of 5% to 15%) will vary according to the resolution in the series and the total number of images to achieve a good balance between sufficiently fast calculation and reliable / representative results. It should be noted that steps 4 and 5 are conventional operations and can be carried out according to the general knowledge in the field of MRI image processing.

[0189] 6. The phase standard deviation of the masked voxels in the image series is denoted as σψ. During the Boll decomposition process, the phase is preserved. In this embodiment, using the expected variation to distinguish whether the Boll coefficient is a meaningful signal or just noise is of practical significance. In addition to calculating the standard deviation, other quantitative statistical measurements can be taken on the phase of the masked voxels in this embodiment. The specific calculation formula is as follows:

[0190]

[0191] where σ ψ is the standard deviation of the phase, ψ i is the phase of the i-th voxel, ψ is the average phase, and N is the number of voxels within the mask.

[0192] 7. The image dataset (including multiple complex MR images, preprocessed complex image data) and σ ψ are passed to a processing pipeline running in the Boll domain. Unless otherwise specified, all subsequent steps are performed after the discrete wavelet transform. According to standard practice, the data in the wavelet transform is hereafter referred to as "coefficients".

[0193] After the preprocessing is completed, the next step of the image preprocessing module is to convert the MRI image data into voxels to prepare for subsequent classification.

[0194] (2) Voxel Classification Module

[0195] As Figure 5 shown, the convolutional neural network (CNN) based on the AlexNet architecture is the core technology of the present invention. First, the classification of each 3D voxel is based on a computational model trained with a large amount of MRI image data. These images have been labeled with specific categories of necrotic regions to assist doctors in making judgments. When processing these images, preprocessing such as normalization and scaling is required to better identify signs of osteonecrosis of the femoral head.

[0196] In the convolutional neural network (CNN) of the AlexNet architecture, feature extraction is accomplished by sliding filters over the entire input image. As Figure 6 shown, a 4x4x4 3D filter determines how neighboring voxels affect the output of the central voxel. And each layer contains multiple such filters for learning different features to better understand the complexity of the input image. After feature extraction, the ReLU activation function increases the non-linearity of the model, while the pooling layer further reduces the size of the feature map.

[0197] For object detection, in this embodiment, selective search and a Region Proposal Network (RPN) are utilized to generate candidate regions. In addition, a series of anchor boxes are predefined in this embodiment, which consider different spatial positions, scales, and aspect ratios to better match the true bounding boxes. Meanwhile, Non-Maximum Suppression (NMS) ensures that each region is detected only once and removes overlapping low-confidence bounding boxes.

[0198] In the classification part, the fully connected layer converts the feature map of the multi-dimensional array into a one-dimensional array and performs classification through the softmax activation function, with the output being a probability distribution. When specifically applied to the MRI images of osteonecrosis of the femoral head, this model will convolve with specific filters in each hidden layer and use ReLU activation and max pooling on specific hidden layers. After this series of operations, the model finally outputs a set of probability scores, thereby helping doctors determine whether the femoral head is necrotic and whether the patient is suitable for hip-preserving treatment.

[0199] To further improve the accuracy of the model and distinguish the present invention from models of the same type, this embodiment generates an MRI training dataset according to the following steps:

[0200] 1. Receive MRI image data containing the femoral head region;

[0201] 2. Divide the MRI image data into a set of 3D voxels;

[0202] 3. Process the MRI image data and generate one or more 3D layers;

[0203] 4. For each 3D layer, use a display to highlight the necrotic region and provide a necrosis category;

[0204] 5. Generate a dataset containing the spatial coordinates of each highlighted region and the associated necrosis category label;

[0205] 6. Process the dataset and generate a file containing spatial coordinates and labels with the same resolution as the 3D voxel set;

[0206] 7. Generate a label file for the 3D voxel set, where each such voxel contains 0 or 1 for each necrosis category;

[0207] 8. Use a set of 3D voxels and the label file for each necrosis category to provide data for the computational model, and the label file contains an indication of 0 or 1 for whether each corresponding voxel of the MRI image data contains such necrosis.

[0208] The innovative advantage of this embodiment is that these steps ensure that the CNN model obtains sufficient training data so that it can accurately identify the degree of osteonecrosis of the femoral head and provide important support for doctors' clinical decisions. The entire process enables the CNN to automatically learn and extract information related to necrosis features. Compared with other CNN models of the same type, this embodiment has the best effect, with an accuracy of 95%, a loss of 0.1643, and the values of precision, recall, and f1-score being approximately 0.91 - 1.

[0209] Next, regarding the recommendation on whether to perform hip-preserving treatment, this embodiment realizes auxiliary decision-making through multi-task learning, that is, simultaneously identifying the degree of necrosis and treatment recommendations in one model. This method requires sufficient data annotated with treatment recommendations. At the same time, model fusion technology is used to combine the prediction results of multiple models to make the final decision. For example, one model focuses on identifying the degree of necrosis, another model gives treatment recommendations, and then a comprehensive decision is made based on the outputs of these two models.

[0210] In summary, when inventors use deep learning models to assist medical decision-making, it is necessary to ensure that the model output is combined with medical knowledge and practice. In all cases, the model's recommendations are only for auxiliary reference.

[0211] (3) Display module

[0212] This embodiment will be based on a computer program and a high-resolution display, and use 3D display technology to visualize the final result through image overlay in three dimensions, ensuring that doctors can more intuitively interpret complex medical data, effectively and quickly communicate the symptoms with patients, and maintain the ability of continuous monitoring.

[0213] 1. Display: Responsible for converting image data into output images meaningful to users. Considering that doctors need to compare multiple images to determine the degree of necrosis, this module will be able to display one or more output images.

[0214] 2. Superposition of necrosis degree map and MRI: By multiplying the MRI image with the binary mask of the necrotic tissue, a necrosis degree map can be generated. At the same time, the necrotic tissue area is displayed as a specific value (usually a non-zero value) in the necrosis degree map, while the healthy tissue area is zero. The specific formula is as follows:

[0215] Necrosis_Map = MRI_Image * Binary_Mask

[0216] Where, Necrosis_Map is the necrosis degree map.

[0217] Then, the method of image overlay is adopted to overlay the necrosis degree information on the MRI image. The specific formula is as follows:

[0218] Combined_Image = MRI_Image+(Alpha*Necrosis_Map)

[0219] Among them, Combined_Image is the superimposed image, and Alpha is a parameter that controls the display intensity of the necrosis degree map on the MRI.

[0220] This process will allow doctors to visually see the location, extent, and degree of necrotic tissue on standardized MRI images. Adjusting the Alpha parameter can control the visualization degree of necrosis degree information on the MRI image to ensure that doctors can clearly see the necrotic tissue while retaining the structural information of the MRI image.

[0221] 3. Display the necrosis degree map in layers: In some application scenarios, doctors need to view the necrosis situation of each part in layers. The original 3D necrosis degree map will be decomposed into multiple layers, and each layer corresponds to a part of the original MRI image dataset. For example, if doctors want to view the necrosis situation in the middle of the femoral head, they can select the layer corresponding to this part.

[0222] 4. Application of the 3D viewer: In addition to layer display, a 3D viewer can also be used to observe the 3D model of the entire femoral head. In this way, doctors can observe the degree and extent of necrosis from any angle and more intuitively evaluate the condition.

[0223] This embodiment is mainly a software application running on a computer device, aiming to intelligently identify the necrosis degree of the femoral head in MRI images and then determine whether the patient is suitable for hip-preserving treatment. This application program can exist in various programming forms, such as source code, object code, etc., and is executed by a computer device. It contains multiple subroutines or components, and the main program and subroutines can interact through calls.

[0224] This application embodiment also provides a readable storage medium, on which a program or instruction is stored. When the program or instruction is executed by a processor, it implements each process of the method embodiment shown above Figure 1 and can achieve the same technical effects. To avoid repetition, it will not be elaborated here.

[0225] Among them, the processor is the processor in the terminal described in the above embodiment. The readable storage medium includes computer-readable storage media, such as computer read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disc, etc.

[0226] The steps of the methods or algorithms described in connection with the disclosure of this application may be implemented in hardware or by a processor executing software instructions. The software instructions may be composed of corresponding software modules, and the software modules may be stored in RAM, flash memory, ROM, EPROM, EEPROM, registers, hard disks, removable hard disks, CD-ROMs, or any other form of storage medium well known in the art. An exemplary storage medium is coupled to the processor so that the processor can read information from the storage medium and write information to the storage medium. Of course, the storage medium may also be a component of the processor. The processor and the storage medium may be carried in an ASIC. Additionally, the ASIC may be carried in a core network interface device. Of course, the processor and the storage medium may also exist as discrete components in the core network interface device.

[0227] Those skilled in the art should be able to realize that in one or more of the above examples, the functions described in this application can be implemented using hardware, software, firmware, or any combination thereof. When implemented using software, these functions can be stored in a computer-readable medium or transmitted as one or more instructions or codes on a computer-readable medium. Computer-readable media include computer storage media and communication media, where communication media includes any medium that facilitates the transfer of a computer program from one place to another. The storage medium can be any available medium accessible by a general or special-purpose computer.

[0228] The specific embodiments described above further elaborate on the purpose, technical solutions, and beneficial effects of this application. It should be understood that the above description is only the specific embodiments of this application and is not used to limit the protection scope of this application. Any modifications, equivalent replacements, improvements, etc. made on the basis of the technical solutions of this application should be included within the protection scope of this application.

[0229] Those skilled in the art should understand that the embodiments of this application can be provided as a method, a system, or a computer program product. Therefore, the embodiments of this application can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the embodiments of this application can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0230] Embodiments of the present application are described with reference to the flowcharts and / or block diagrams of methods, apparatuses (systems), and computer program products according to embodiments of the present application. It should be understood that each process and / or block in the flowchart and / or block diagram, and the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to generate a machine, such that the instructions executed by the processor of the computer or other programmable data processing devices generate a means for implementing the functions specified in one process Figure 1 one process or multiple processes and / or blocks Figure 1 or a means for implementing the functions specified in multiple blocks.

[0231] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, such that the instructions stored in the computer-readable memory generate a manufactured article including an instruction means that implements the functions specified in one process Figure 1 one process or multiple processes and / or blocks Figure 1 or a means for implementing the functions specified in multiple blocks.

[0232] These computer program instructions can also be loaded onto a computer or other programmable data processing device, such that a series of operation steps are executed on the computer or other programmable device to generate a computer-implemented process, so that the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in one process Figure 1 one process or multiple processes and / or blocks Figure 1 or a means for implementing the functions specified in multiple blocks.

[0233] Obviously, those skilled in the art can make various modifications and variations to the embodiments of the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the embodiments of the present application fall within the scope of the claims of the present application and their equivalent technologies, the present application is also intended to include these modifications and variations.

Claims

1. A hip-preserving treatment assistance method, characterized in that, it includes: Converting one or more MRI image data of the hip joint area into a set of 3D voxels; Using a convolutional neural network to classify, locate and identify the necrosis degree of the femoral head for a set of 3D voxels; Performing image overlay display of the MRI image data and the necrosis degree of the femoral head.

2. The method according to claim 1, characterized in that, Dividing one or more MRI image data of the hip joint area into a set of 3D voxels, including: Performing segmentation processing on the MRI image to obtain MRI image data of the hip joint area; Performing denoising processing on the MRI image data of the hip joint area by a wavelet transform denoising method; Converting the denoised MRI image data of the hip joint area into a set of 3D voxels.

3. The method according to claim 2, characterized in that, Performing segmentation processing on the MRI image to obtain MRI image data of the hip joint area, including: Performing preliminary segmentation on the MRI image through a pre-trained neural network model to obtain a hip joint area image sub-array, and the pre-trained neural network model is used to extract the hip joint-related area from the MRI image; Performing deep wavelet auto-encoding on the segmented hip joint area image sub-array to obtain the final MRI image data of the hip joint area.

4. The method according to claim 2, characterized in that, The wavelet transform denoising method includes: Providing M original image data sets from the same object, where M is greater than or equal to 1; Correcting the potential misalignment of the image data and performing consistent interpolation to avoid changes in noise statistics in the image series; Optionally applying approximate phase correction in the image space if the signal phase shows a consistent time evolution related to the preparation of the rotation system in the image series; Calculating the average signal intensity of each voxel in the image series in the image space; Generating a mask that contains the voxels with the highest average signal intensity as a representative sample for statistical evaluation; Transmitting the image data set and the phase standard deviation of the masked voxels in the image series to a processing pipeline running in the Fourier domain.

5. The method according to claim 1, characterized in that, Using a convolutional neural network to classify, locate and identify the necrosis degree of the femoral head for a set of 3D voxels, including: Using a convolutional neural network with the AlexNet architecture and an MRI training data set, as well as object detection technology, to classify, locate and identify the necrosis degree of the femoral head for a set of 3D voxels.

6. The method according to claim 5, characterized in that, The MRI training data set is generated by the following method: Receiving MRI image data containing the femoral head area; Dividing the MRI image data into a set of 3D voxels; Processing this MRI image data and generating one or more 3D layers; For each 3D layer, highlighting the necrosis area and providing a necrosis category; Generating a data set containing the spatial coordinates of each highlighted area and the necrosis category label associated therewith; Process the dataset and generate a file containing spatial coordinates and labels with the same resolution as the 3D voxel set; Generate a label file for the 3D voxel set, where each such voxel contains 0 or 1 for each necrosis category; Use a set of 3D voxels and the label file for each necrosis category to provide data for the computational model, which for each corresponding voxel of the MRI image data contains an indication of 0 or 1 as to whether the voxel contains such necrosis.

7. The method according to claim 5, wherein, the process of the object detection technique includes: Generate candidate regions using the method of selective search and region proposal network, where the region contains an object; Divide the image of the candidate region into predefined grids and predict multiple bounding boxes and their confidence levels for each grid; Predefine a set of bounding boxes at different spatial positions, scales, and aspect ratios for predicting the adjustment amount of the ground truth bounding box that best matches them; If multiple bounding boxes overlap and contain the same object, retain the bounding box with the highest classification score and delete the other overlapping bounding boxes; Output a set of bounding boxes and their corresponding class labels and classification scores representing the objects detected in the image.

8. The method according to claim 1, wherein, The image overlay display of the MRI image data and the necrosis degree of the femoral head includes: Generate a necrosis degree map by multiplying the MRI image by the binary mask of the necrotic tissue corresponding to the necrosis degree of the femoral head; Adopt the method of image overlay to overlay the necrosis degree map on the MRI image.

9. An auxiliary device for hip-preserving treatment, wherein, it includes: An image preprocessing module for converting one or more MRI image data for the hip joint region into a set of 3D voxels; A voxel classification module for classifying a set of 3D voxels using a convolutional neural network to locate and identify the necrosis degree of the femoral head; A display module for performing image overlay display of the MRI image data and the necrosis degree of the femoral head.

10. A readable storage medium, wherein, The readable storage medium stores programs or instructions, and when the programs or instructions are executed by a processor, the steps of the method according to any one of claims 1 to 8 are implemented.

Images