Femoral head necrosis recognition system and method based on deep learning

Abstract

The invention provides a femoral head necrosis recognition system and method based on deep learning, and the method comprises the steps: obtaining a multi-mode magnetic resonance image of a hip joint of an examined person, and the multi-mode magnetic resonance image comprises a magnetic resonance image formed by a plurality of magnetic resonance scanning sequences; loading a pre-trained image segmentation model to identify a femoral head and an acetabulum in the hip joint of the multi-modal magnetic resonance image, and segmenting the femoral head and the acetabulum to obtain a femoral head segmentation image of the magnetic resonance image of each scanning sequence; performing enhancement processing on each femoral head segmentation image; extracting femoral head feature data of each femoral head segmentation image type and forming splicing feature data; and loading a pre-trained necrosis recognition model to process the spliced feature data to obtain femoral head necrosis recognition results under each magnetic resonance scanning sequence and output the femoral head necrosis recognition results. The diagnosis accuracy can be improved, and the misdiagnosis rate is reduced.

Description

Technical Field

[0001] The present invention relates to the technical field of medical image processing, and in particular to a femoral head necrosis recognition system and method based on deep learning. Background Art

[0002] Osteonecrosis of the femoral head (ONFH) is a bone tissue necrosis disease caused by multiple factors. It usually causes bone cell death due to interruption of blood supply, which leads to joint pain, limited function and eventual joint collapse, and may eventually lead to complete loss of joint function. Late-stage patients can often only recover partial function through total hip replacement surgery, which seriously affects their quality of life. Therefore, early diagnosis and accurate identification of necrotic areas are crucial to patients' treatment options and prognosis.

[0003] Traditional X-rays, Computerized Tomography (CT), and Magnetic Resonance Imaging (MRI) are commonly used imaging diagnostic methods. Among them, X-rays are the oldest, fastest, and most commonly used imaging method, but it is difficult to diagnose early femoral head necrosis. CT scans are the main tool for diagnosing early femoral head collapse, and surgical methods combined with CT examinations are relatively accurate. Because MRI is more accurate and sensitive in examining femoral head images, and has no radiation compared to X-rays and CT, it is more accepted by physicians and patients. However, traditional imaging diagnostic methods usually rely on the subjective judgment of doctors and have certain limitations, such as low diagnostic accuracy, large errors, and different results due to differences in experience among different doctors.

[0004] In recent years, with the rapid development of deep learning technology, image recognition methods based on artificial intelligence have been widely used in medical image analysis. Deep learning, especially convolutional neural networks (CNNs), can automatically extract features from image data and complete classification tasks. Compared with traditional image processing methods, deep learning has shown higher accuracy and robustness in the fields of image classification and pattern recognition. Although deep learning technology has shown great potential in medical image analysis, there is currently a lack of identification methods for femoral head necrosis, especially in terms of automation and accurate classification. There is still a large research gap. Summary of the invention

[0005] Based on the above problems, the present invention provides a femoral head necrosis identification system based on deep learning, which aims to solve the technical problems of the prior art such as the difficulty of early manual identification of femoral head necrosis and the high misdiagnosis rate.

[0006] The present invention provides a femoral head necrosis recognition system based on deep learning, comprising:

[0007] A data acquisition module, used for acquiring a multimodal magnetic resonance image of a hip joint of a subject, wherein the multimodal magnetic resonance image includes magnetic resonance images formed by a plurality of magnetic resonance scanning sequences;

[0008] A preprocessing module, connected to the data acquisition module, for preprocessing the multimodal magnetic resonance image;

[0009] An image segmentation module, connected to the preprocessing module, for loading a pre-trained image segmentation model to identify and segment the femoral head and acetabulum in the hip joint of the pre-processed multimodal magnetic resonance image, and obtain a femoral head segmentation image of the magnetic resonance image of each scanning sequence;

[0010] An enhancement processing module, connected to the image segmentation module, for performing enhancement processing on each of the femoral head segmentation images;

[0011] A feature splicing module, connected to the enhancement processing module, is used to extract the femoral head feature data from each of the femoral head segmentation images and splice them to form spliced ​​feature data;

[0012] The necrosis recognition module is connected to the feature splicing module and is used to load a pre-trained necrosis recognition model to process the splicing feature data, obtain the femoral head necrosis recognition results under each magnetic resonance scanning sequence and output them.

[0013] Furthermore, the feature splicing module includes:

[0014] An image segmentation unit, used for segmenting each femoral head segmentation image into blocks to form a left femoral head sub-image and a right femoral head sub-image;

[0015] A feature extraction unit, connected to the image segmentation unit, for extracting feature data of the left femoral head sub-image to obtain first femoral head feature data, and extracting feature data of the right femoral head sub-image to obtain second femoral head feature data;

[0016] A feature splicing unit, connected to the feature extraction unit, for splicing the first femoral head feature data under each scanning sequence to obtain first spliced ​​feature data, and splicing the second femoral head feature data under each scanning sequence to obtain second spliced ​​feature data;

[0017] The necrosis recognition module is used to use a necrosis recognition model to process the first splicing feature data and the second splicing feature data respectively to obtain necrosis recognition results of the left femoral head and the right femoral head.

[0018] Furthermore, the image segmentation model is a UNet network model.

[0019] Furthermore, the necrosis recognition model is a SE-ResNet50 model.

[0020] Furthermore, the multimodal magnetic resonance image includes a magnetic resonance image formed by at least two magnetic resonance scanning sequences of T1-weighted imaging, T2-weighted imaging, proton density-weighted imaging, and proton density-weighted short-time inversion recovery sequence.

[0021] Furthermore, it also includes:

[0022] An image annotation module is connected to the necrosis identification module and is used to annotate the femoral head necrosis identification result on the multimodal magnetic resonance image.

[0023] Furthermore, it also includes:

[0024] The data analysis module is connected to the necrosis identification module and is used to analyze the femoral head necrosis identification results obtained under each magnetic resonance scanning sequence to obtain a femoral head necrosis analysis report.

[0025] Furthermore, the femoral head necrosis analysis report includes examination recommendations and treatment recommendations.

[0026] A method for identifying femoral head necrosis based on deep learning, using the aforementioned femoral head necrosis identification system based on deep learning, comprising:

[0027] Step A1, obtaining a multimodal magnetic resonance image of a hip joint of a subject, wherein the multimodal magnetic resonance image includes magnetic resonance images formed by multiple magnetic resonance scanning sequences;

[0028] Step A2, preprocessing the multimodal magnetic resonance image;

[0029] Step A3, loading a pre-trained image segmentation model to identify and segment the femoral head and acetabulum in the hip joint of the pre-processed multimodal magnetic resonance image, and obtaining a femoral head segmentation image of the magnetic resonance image of each scanning sequence;

[0030] Step A4, performing enhancement processing on each of the femoral head segmentation images;

[0031] Step A5, extracting the femoral head feature data from each of the femoral head segmentation images and splicing them to form spliced ​​feature data;

[0032] Step A6, loading a pre-trained necrosis recognition model to process the spliced ​​feature data, obtain the femoral head necrosis recognition results under each magnetic resonance scanning sequence and output them.

[0033] The beneficial technical effect of the present invention is that by utilizing deep learning algorithms for magnetic resonance technology, especially integrating multiple different types of magnetic resonance scanning sequences, femoral head segmentation and necrosis identification can be performed, which can not only improve the accuracy of diagnosis and reduce the misdiagnosis rate, but also alleviate the work difficulty of clinical physicians. It has important clinical application value and research significance. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1-2 A schematic diagram of a module of a femoral head necrosis identification system based on deep learning according to the present invention;

[0035] Figure 3 This is a step flow chart of a method for identifying femoral head necrosis based on deep learning in the present invention. DETAILED DESCRIPTION

[0036] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0037] It should be noted that, in the absence of conflict, the embodiments of the present invention and the features in the embodiments may be combined with each other.

[0038] The present invention will be further described below in conjunction with the accompanying drawings and specific embodiments, but they are not intended to limit the present invention.

[0039] See also Figure 1 The present invention provides a femoral head necrosis recognition system based on deep learning, comprising:

[0040] A data acquisition module (1) is used to acquire a multimodal magnetic resonance image of a hip joint of a subject, wherein the multimodal magnetic resonance image includes magnetic resonance images formed by multiple magnetic resonance scanning sequences;

[0041] A preprocessing module (2), connected to the data acquisition module (1), and used for preprocessing the multimodal magnetic resonance image;

[0042] An image segmentation module (3) is connected to the preprocessing module (2) and is used to load a pre-trained image segmentation model to identify and segment the femoral head and acetabulum in the hip joint of the pre-processed multimodal magnetic resonance image, thereby obtaining a femoral head segmentation image of the magnetic resonance image of each scanning sequence;

[0043] An enhancement processing module (4), connected to the image segmentation module (3), for performing enhancement processing on each of the femoral head segmentation images;

[0044] A feature splicing module (5), connected to the enhancement processing module (4), is used to extract the femoral head feature data from each of the femoral head segmentation images and splice them to form spliced ​​feature data;

[0045] The necrosis recognition module (6) is connected to the feature splicing module (5) and is used to load a pre-trained necrosis recognition model to process the spliced ​​feature data, obtain the femoral head necrosis recognition results under each magnetic resonance scanning sequence, and output them.

[0046] The preprocessing module (2) includes extracting the region of interest (ROI) and so on.

[0047] The image segmentation model identifies the hip joint, further identifies the acetabulum and femoral head of the hip joint, and segments the femoral head to obtain a femoral head segmentation image.

[0048] The enhancement processing module (4) performs further enhancement processing on the segmented femoral head segmentation image, such as contrast enhancement, image noise reduction processing, edge enhancement processing on the noise-reduced image, etc. to improve the recognition of subsequent femoral head necrosis. Specifically, the contrast-limited adaptive histogram equalization (CLAHE) method is used to enhance the details of the image, that is, to perform contrast enhancement, and the Gaussian filtering algorithm is used to reduce the noise of the image. The Roberts gradient operator is used for edge enhancement processing.

[0049] Use the Gaussian filter algorithm to reduce the noise of the image. The calculation formula is as follows:

[0050]

[0051] Where (x, y) is any pixel in the image covered by the filter, (ux, uy) represents the coordinates of the center point, and the standard deviation σ determines the shape of the Gaussian function.

[0052] Specifically, two 2×2 kernel Roberts gradient operators are used to calculate the horizontal and vertical gradients for edge enhancement. The calculation formula is as follows:

[0053]

[0054] Where f(i,j) represents a pixel at any point in the image, and R(i,j) represents the vector sum of pixels at any point in the image.

[0055] Magnetic resonance images obtained by different MRI scanning sequences have their own advantages. The present invention utilizes deep learning algorithms for magnetic resonance technology, especially the integration of multiple different types of MRI scanning sequences, to perform femoral head segmentation and necrosis identification, especially for the identification of early femoral head necrosis. It can not only improve the accuracy of diagnosis and reduce the misdiagnosis rate, but also alleviate the work difficulty of clinical physicians. It has important clinical application value and research significance.

[0056] In the process of training the image segmentation model and the necrosis recognition model, data is extracted from the medical digital imaging and communication standard DICOM (Digital Imaging and Communications in Medicine) and converted into PNG format as sample data. The preprocessing module (2) processes the sample data and performs the first type of annotation (annotating the hip joint, femoral head and acetabulum) to form a first data set. The first data set is divided into a training set, a verification set and a test set to train the image segmentation model. The first data set and its annotation are used to further obtain femoral head segmentation images. After the images are enhanced, they are annotated (annotating necrotic tissue and non-necrotic tissue), and then feature splicing is performed to obtain a second data set. The second data set is divided into a training set, a verification set and a test set to train the necrosis recognition model.

[0057] DICOM is a standard format used by medical imaging devices such as MRI to store, exchange and transmit images and related information.

[0058] See also Figure 2 Furthermore, the feature splicing module (5) comprises:

[0059] An image segmentation unit (51) is used to perform segmentation processing on each femoral head segmentation image to form a left femoral head sub-image and a right femoral head sub-image;

[0060] A feature extraction unit (52), connected to the image segmentation unit (51), used for extracting feature data of the left femoral head sub-image to obtain first femoral head feature data, and extracting feature data of the right femoral head sub-image to obtain second femoral head feature data;

[0061] A feature splicing unit (53), connected to the feature extraction unit (52), used for splicing the first femoral head feature data under each scanning sequence to obtain first spliced ​​feature data, and splicing the second femoral head feature data under each scanning sequence to obtain second spliced ​​feature data;

[0062] The necrosis recognition module (6) is used to use a necrosis recognition model to process the first splicing feature data and the second splicing feature data respectively, so as to obtain necrosis recognition results of the left femoral head and the right femoral head.

[0063] The human body generally has a left femoral head and a right femoral head. By dividing the image into blocks, the left femoral head and the right femoral head are identified separately, thereby improving the recognition accuracy.

[0064] Furthermore, the image segmentation model is a UNet network model.

[0065] UNet is a convolutional neural network designed specifically for medical image segmentation. The model adopts a U-shaped structure and is divided into two parts: the encoder (downsampling path) and the decoder (upsampling path). The encoder is responsible for extracting feature information from the input image, while the decoder is used to restore the spatial resolution and generate the final segmentation result. A key feature of UNet is that it uses skip connections, which allow low-level spatial information to be passed directly from the encoder to the corresponding decoder layer, thereby helping to preserve details and improve segmentation accuracy.

[0066] Furthermore, the necrosis recognition model is a SE-ResNet50 model.

[0067] SE-ResNet50 is an improved version of ResNet with the Squeeze-and-Excitation (SE) module added. It introduces residual blocks to make it easier for the network to learn identity mapping, thus overcoming the problem of performance degradation of traditional CNN as the number of layers increases.

[0068] Furthermore, the multimodal magnetic resonance image includes a magnetic resonance image formed by at least two magnetic resonance scanning sequences of T1-weighted imaging, T2-weighted imaging, proton density-weighted imaging, and proton density-weighted short-time inversion recovery sequence.

[0069] Different MRI sequences have different imaging focuses. T1-weighted imaging T1WI has clear tissue anatomical structure, T2-weighted imaging T2WI is sensitive to tissue pathological changes, proton density weighted imaging PDW sequence signals mainly come from water molecules in tissues and / or hydrogen protons in fat, and proton density weighted short-time inversion recovery sequence PDW-SPAIR is a unique fat suppression technology proposed by PHILIPS, with clear image anatomical structure and rich layers. It can detect early femoral head necrosis very well.

[0070] Furthermore, it also includes:

[0071] An image annotation module (7) is connected to the necrosis identification module (6) and is used to annotate the femoral head necrosis identification result on the multimodal magnetic resonance image.

[0072] By annotating the necrosis identification results, the necrotic area can be intuitively presented to the doctor, assisting the doctor in making a diagnosis and further treatment plan.

[0073] Furthermore, it also includes:

[0074] The data analysis module (8) is connected to the necrosis identification module (6) and is used to analyze the femoral head necrosis identification results obtained under each magnetic resonance scanning sequence to obtain a femoral head necrosis analysis report.

[0075] The present invention uses artificial intelligence to perform analysis based on the femoral head necrosis identification results to assist doctors in reaching a diagnosis conclusion and further treatment plan.

[0076] Furthermore, the femoral head necrosis analysis report includes examination recommendations and treatment recommendations.

[0077] See also Figure 3 The present invention also provides a method for identifying femoral head necrosis based on deep learning, using the aforementioned femoral head necrosis identification system based on deep learning, comprising:

[0078] Step A1, obtaining a multimodal magnetic resonance image of a hip joint of a subject, wherein the multimodal magnetic resonance image includes magnetic resonance images formed by multiple magnetic resonance scanning sequences;

[0079] Step A2, preprocessing the multimodal magnetic resonance image;

[0080] Step A3, loading a pre-trained image segmentation model to identify and segment the femoral head and acetabulum in the hip joint of the pre-processed multimodal magnetic resonance image, and obtaining a femoral head segmentation image of the magnetic resonance image of each scanning sequence;

[0081] Step A4, performing enhancement processing on each of the femoral head segmentation images;

[0082] Step A5, extracting the femoral head feature data from each of the femoral head segmentation images and splicing them to form spliced ​​feature data;

[0083] Step A6, loading a pre-trained necrosis recognition model to process the spliced ​​feature data, obtain the femoral head necrosis recognition results under each magnetic resonance scanning sequence and output them.

[0084] Magnetic resonance images obtained by different MRI scanning sequences have their own advantages. The present invention utilizes deep learning algorithms for magnetic resonance technology, especially the integration of multiple different types of MRI scanning sequences, to perform femoral head segmentation and necrosis identification, especially for the identification of early femoral head necrosis. It can not only improve the accuracy of diagnosis and reduce the misdiagnosis rate, but also alleviate the work difficulty of clinical physicians. It has important clinical application value and research significance.

[0085] The above description is only a preferred embodiment of the present invention, and does not limit the implementation mode and protection scope of the present invention. For those skilled in the art, it should be aware that all solutions obtained by equivalent substitutions and obvious changes made using the description and illustrations of the present invention should be included in the protection scope of the present invention.

Claims

1. A femoral head necrosis recognition system based on deep learning, characterized in that: include: A data acquisition module, used for acquiring a multimodal magnetic resonance image of a hip joint of a subject, wherein the multimodal magnetic resonance image includes magnetic resonance images formed by a plurality of magnetic resonance scanning sequences; A preprocessing module, connected to the data acquisition module, for preprocessing the multimodal magnetic resonance image; An image segmentation module, connected to the preprocessing module, for loading a pre-trained image segmentation model to identify and segment the femoral head and acetabulum in the hip joint of the pre-processed multimodal magnetic resonance image, and obtain a femoral head segmentation image of the magnetic resonance image of each scanning sequence; An enhancement processing module, connected to the image segmentation module, for performing enhancement processing on each of the femoral head segmentation images; A feature splicing module, connected to the enhancement processing module, is used to extract the femoral head feature data from each of the femoral head segmentation images and splice them to form spliced ​​feature data; The necrosis recognition module is connected to the feature splicing module and is used to load a pre-trained necrosis recognition model to process the splicing feature data, obtain the femoral head necrosis recognition results under each magnetic resonance scanning sequence and output them.

2. A femoral head necrosis identification system based on deep learning as described in claim 1, characterized in that: The feature splicing module includes: An image segmentation unit, used for segmenting each femoral head segmentation image into blocks to form a left femoral head sub-image and a right femoral head sub-image; A feature extraction unit, connected to the image segmentation unit, for extracting feature data of the left femoral head sub-image to obtain first femoral head feature data, and extracting feature data of the right femoral head sub-image to obtain second femoral head feature data; A feature splicing unit, connected to the feature extraction unit, for splicing the first femoral head feature data under each scanning sequence to obtain first spliced ​​feature data, and splicing the second femoral head feature data under each scanning sequence to obtain second spliced ​​feature data; The necrosis recognition module is used to use a necrosis recognition model to process the first splicing feature data and the second splicing feature data respectively to obtain necrosis recognition results of the left femoral head and the right femoral head.

3. A femoral head necrosis identification system based on deep learning as claimed in claim 1, characterized in that: The image segmentation model is a UNet network model.

4. A femoral head necrosis identification system based on deep learning as claimed in claim 1, characterized in that: The necrosis recognition model is a SE-ResNet50 model.

5. A femoral head necrosis identification system based on deep learning as claimed in claim 1, characterized in that: The multimodal magnetic resonance image includes a magnetic resonance image formed by at least two magnetic resonance scanning sequences of T1-weighted imaging, T2-weighted imaging, proton density-weighted imaging, and proton density-weighted short-time inversion recovery sequence.

6. A femoral head necrosis identification system based on deep learning as claimed in claim 1, characterized in that: Also includes: An image annotation module is connected to the necrosis identification module and is used to annotate the femoral head necrosis identification result on the multimodal magnetic resonance image.

7. A femoral head necrosis identification system based on deep learning as claimed in claim 1, characterized in that: Also includes: The data analysis module is connected to the necrosis identification module and is used to analyze the femoral head necrosis identification results obtained under each magnetic resonance scanning sequence to obtain a femoral head necrosis analysis report.

8. A femoral head necrosis identification system based on deep learning as claimed in claim 7, characterized in that: The femoral head necrosis analysis report includes examination recommendations and treatment recommendations.

9. A method for identifying femoral head necrosis based on deep learning, characterized in that: A femoral head necrosis recognition system based on deep learning as described in any one of claims 1 to 8 is used, comprising: Step A1, obtaining a multimodal magnetic resonance image of a hip joint of a subject, wherein the multimodal magnetic resonance image includes magnetic resonance images formed by multiple magnetic resonance scanning sequences; Step A2, preprocessing the multimodal magnetic resonance image; Step A3, loading a pre-trained image segmentation model to identify and segment the femoral head and acetabulum in the hip joint of the pre-processed multimodal magnetic resonance image, and obtaining a femoral head segmentation image of the magnetic resonance image of each scanning sequence; Step A4, performing enhancement processing on each of the femoral head segmentation images; Step A5, extracting the femoral head feature data from each of the femoral head segmentation images and splicing them to form spliced ​​feature data; Step A6, loading a pre-trained necrosis recognition model to process the spliced ​​feature data, obtain the femoral head necrosis recognition results under each magnetic resonance scanning sequence and output them.

Images