Automatic knee arthritis grading system
By designing an automatic grading system for knee arthritis, the lesion area in knee joint images is automatically segmented and graded, and the problem of low reading rate and accuracy of knee joint images in the prior art is solved, achieving more efficient and accurate diagnosis, reducing diagnostic deviation and labor costs.
Patent Information
- Application Number
- CN202411863040.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-12-17
- Publication Date
- 2025-05-02
AI Technical Summary
In the prior art, the reading rate and accuracy of knee joint images are low, which can easily lead to missed diagnosis or misdiagnosis, and there are large diagnostic deviations between doctors.
An automatic knee arthritis classification system was designed, including image acquisition module, preprocessing module, lesion recognition module and lesion classification module. The system automatically segments the images of both knees, recognizes and grades the lesion area, generates the lesion level and its probability values, and assists the doctor in diagnosis.
It improves the reading rate and accuracy of knee joint images, reduces the possibility of missed diagnosis and misdiagnosis, reduces the diagnostic deviation between doctors, saves diagnosis labor costs, and provides more objective diagnostic results.
Smart Images
Figure CN119919350A_ABST
Abstract
Description
Technical Field
[0001] The invention relates to the technical field of medical image processing, and in particular to an automatic grading system for knee arthritis. Background Art
[0002] Osteoarthritis is a typical chronic joint disease that causes pain and reduced motor function, resulting in inconvenience in life. It is one of the main causes of disability in the elderly, affecting about 10% of men and about 18% of women over 60 years old. Today, the aging population has made osteoarthritis one of the main factors of disability in the elderly. The lesions mainly involve the knee, hip, cervical spine, ankle and other joints. Intense or uncoordinated activities, as well as many factors including age, gender, previous joint loss, obesity, etc. are all causes of osteoarthritis. The knee joint is one of the main areas involved in osteoarthritis pathology, which can cause knee cartilage damage. Clinically, its main pathological characteristics include knee joint space stenosis, osteophytes and sclerosis; the prevalence rate increases sharply with age, and it is most common in obese or physically active people over 45 years old and the elderly. Knee arthritis will seriously affect people's lives in the later stage. If it is not treated early, it will cause disability due to knee joint deformation.
[0003] With the advancement of modern medical technology, a variety of examination methods for knee arthritis are currently used in clinical practice. These examination methods mainly include physical examination, imaging examination, and some invasive examination methods that cause certain harm to the human body, such as pathological examination. Imaging examination, a non-invasive, safe and fast examination method, has become the most common disease examination method. The development of modern imaging technology has provided a more transparent and reliable basis for the diagnosis of diseases, but it has also created new additional problems. Since each examination of the patient will generate a large number of medical images, the repetitive reading and reading fatigue of radiologists can easily lead to missed diagnoses or misdiagnoses. The attending physician needs to make a diagnosis based on the pathological characteristics provided by the image, and the differences in diagnostic experience of different doctors will inevitably lead to diagnostic bias.
[0004] Therefore, how to improve the reading speed and accuracy of knee joint images has become a basic problem that needs to be solved urgently. Summary of the invention
[0005] In view of the problems existing in the prior art, the present invention provides an automatic grading system for knee arthritis, comprising:
[0006] An image acquisition module, used for acquiring images of both knees of patients with knee arthritis;
[0007] A preprocessing module, connected to the image acquisition module, for segmenting a left knee image and a right knee image from the double knee images, and segmenting a knee joint region of interest from the left knee image and the right knee image respectively;
[0008] A lesion recognition module, connected to the preprocessing module, for inputting the knee joint region of interest into a lesion recognition model to identify the lesion region therein;
[0009] The lesion grading module is connected to the lesion identification module and is used to input the lesion area into the knee arthritis grading model to grade the lesion area to obtain the lesion grade of the knee arthritis patient and its corresponding probability value for reference by doctors.
[0010] Preferably, the double knee images are obtained by X-ray acquisition of the knee arthritis patient in an upright position, wherein the upright position is a posterior-anterior weight-bearing position with the knee joint flexed at 45°.
[0011] Preferably, the preprocessing module comprises:
[0012] A left and right knee segmentation unit, configured to set a vertical segmentation line according to the gray value distribution of each pixel point in the double knee image, and segment the double knee image from the vertical segmentation line to obtain the left knee image and the right knee image;
[0013] A region segmentation unit is connected to the left and right knee segmentation units, and is used to generate rectangular frames in the left knee image and the right knee image respectively, and segment out the selected area of the rectangular frame as the knee joint region of interest, wherein the rectangular frame at least includes the lower end of the femur and the upper end of the tibia.
[0014] Preferably, the left and right knee splitting unit comprises:
[0015] A clustering subunit, used for converting the double knee image into a grayscale image, counting the first grayscale values of all the pixels in the double knee image, and performing cluster analysis on all the first grayscale values to obtain three cluster categories;
[0016] a configuration subunit, connected to the clustering subunit, for respectively extracting the first grayscale value of the cluster center of each cluster category, and respectively configuring the cluster categories corresponding to the first grayscale value as bone, muscle and fat in descending order, and then determining the grayscale range of each cluster category based on the extracted first grayscale value;
[0017] a setting subunit, connected to the configuration subunit, for obtaining a second grayscale value of each pixel on the vertical midline of the grayscale image, and if each of the second grayscale values meets a preset condition, taking the vertical midline as the vertical dividing line, otherwise taking the vertical center line as the starting point, adjusting the vertical center line left and right according to a preset step length until the vertical dividing line that meets the preset condition is obtained;
[0018] The preset condition is that each of the second grayscale values is not within the grayscale range associated with the bone and more than 80% are not within the grayscale range associated with the muscle and the fat.
[0019] Preferably, the left and right knee splitting unit further comprises:
[0020] A first positioning subunit is used to define the left knee image and the right knee image as initial segmentation images, then locate the pixel points belonging to the bones closest to the left and right edges in the initial segmentation images, and generate vertical lines along the two pixel points respectively;
[0021] A second positioning subunit is used to locate the pixel points belonging to the skeleton closest to the upper and lower edges in the initial segmented image, and generate horizontal lines along the two pixel points respectively;
[0022] The re-segmentation subunit is respectively connected to the first positioning subunit and the second positioning subunit, and is used to cut out the areas enclosed by the vertical lines and the horizontal lines from the left knee image and the right knee image respectively, to obtain new left knee images and right knee images and input them into the area segmentation unit.
[0023] Preferably, the lesion recognition model adopts a YOLOv5s target detection model, and a CA attention mechanism is embedded in the YOLOv5s target detection model.
[0024] Preferably, the knee arthritis grading model adopts the EfficientNetV2-S network model, and the attention mechanism in the MBConv module in the EfficientNetV2-S network model is the CBAM attention mechanism, and the activation function of the EfficientNetV2-S network model is the SMU activation function.
[0025] Preferably, the lesion grade is KL grade.
[0026] Preferably, it also includes a human-computer interaction interface, which is respectively connected to the image acquisition module, the preprocessing module, the lesion recognition module and the lesion grading module, and the lesion grading module also includes a thermal map generation unit, which is used to generate a thermal feature map of the lesion area through the CBAM attention mechanism;
[0027] The human-computer interaction interface is provided with a selection button and a grading button. The user selects the double knee images that need to be automatically graded by clicking the selection button. When the human-computer interaction interface detects that the grading button is clicked, the acquired double knee images are sent to the preprocessing module, and the lesion area, the thermal map, the lesion grade and its corresponding probability value are displayed accordingly.
[0028] Preferably, it also includes a local database for storing the double knee images and the lesion areas, the thermal maps, the lesion levels and their corresponding probability values contained therein.
[0029] The above technical scheme has the following advantages or beneficial effects: the present invention can automatically segment the double knee images into the left knee image and the right knee image, and respectively identify the lesion areas in the left knee image and the right knee image, and then grade the lesion areas to obtain the lesion grade of the knee arthritis patient and its corresponding probability value, assisting doctors in grading knee arthritis, which helps to improve the diagnosis efficiency, and then provide treatment plans for patients as early as possible to prevent the disease from worsening and causing irreversible consequences; at the same time, it can solve certain emergencies or the problem of insufficient number of doctors in remote areas, and save diagnosis labor costs; in addition, compared with the subjective manual diagnosis of doctors, the severity of knee arthritis is assessed through a neural network model, and the judgment made by the computer is more objective than the subjective judgment of doctors. BRIEF DESCRIPTION OF THE DRAWINGS
[0030] Figure 1 It is a structural schematic diagram of an automatic grading system for knee arthritis in a preferred embodiment of the present invention;
[0031] Figure 2 It is a schematic diagram of a human-computer interaction interface in a preferred embodiment of the present invention. DETAILED DESCRIPTION
[0032] The present invention is described in detail below in conjunction with the accompanying drawings and specific embodiments. The present invention is not limited to this embodiment, and other embodiments may also fall within the scope of the present invention as long as they conform to the gist of the present invention.
[0033] In a preferred embodiment of the present invention, based on the above problems existing in the prior art, an automatic grading system for knee arthritis is provided. Figure 1 As shown, including:
[0034] An image acquisition module 1 is used to acquire images of both knees of a patient with knee arthritis;
[0035] The preprocessing module 2 is connected to the image acquisition module 1 and is used to segment the left knee image and the right knee image from the double knee images, and to segment the knee joint region of interest from the left knee image and the right knee image respectively;
[0036] The lesion recognition module 3 is connected to the preprocessing module 2 and is used to input the knee joint region of interest into the lesion recognition model to identify the lesion region therein;
[0037] The lesion grading module 4 is connected to the lesion identification module 3 and is used to input the lesion area into the knee arthritis grading model to grade the lesion area to obtain the lesion grade of the knee arthritis patient and its corresponding probability value for reference by doctors.
[0038] Specifically, the main lesion site of knee osteoarthritis is in the knee, which includes not only tendons, bones, meridians, and contact surfaces, but also skin and blood vessels because the weight-bearing can simulate the force that the patient bears during walking. X-ray examination is widely used in clinical practice because of its simple examination method and easy-to-understand images. It has a high detection rate and has great advantages, which is conducive to its widespread use in patients with knee osteoarthritis. X-ray examination of knee osteoarthritis generally adopts conventional anteroposterior upright position and supine position, but because the knee joint is not weight-bearing in the supine position, the ligaments and muscles are in a relaxed state. Although there may be wear of the articular cartilage, it will still appear normal during the examination. Accurate and effective X-ray examination is very important for the early detection of knee osteoarthritis. Based on this. In this embodiment, the images of both knees are obtained by X-ray collection of patients with knee arthritis in an upright position. However, in the conventional anteroposterior upright position, the patient's weight load is transmitted downward through the cartilage in front of the femur and tibial plateau, and the force applied is reduced during X-ray examination. The most serious part of the articular cartilage wear is the contact surface when the knee joint is flexed about 45°. At the same time, the force borne by the weight-bearing patient during walking can increase the severity of knee arthritis. Therefore, in this embodiment, the upright position adopts an anterior-posterior weight-bearing position with the knee joint flexed at 45°.
[0039] In a preferred embodiment of the present invention, the preprocessing module 2 includes:
[0040] The left and right knee segmentation unit 21 is used to set a vertical segmentation line according to the gray value distribution of each pixel point in the double knee image, and segment the double knee image from the vertical segmentation line to obtain a left knee image and a right knee image;
[0041] The region segmentation unit 22 is connected to the left and right knee segmentation units 21, and is used to generate rectangular frames in the left knee image and the right knee image respectively, and segment the selected area of the rectangular frame as the knee joint area of interest, and the rectangular frame at least includes the lower end of the femur and the upper end of the tibia.
[0042] Specifically, since the analysis of knee arthritis is for a single joint, the segmentation of the left and right legs is the basis of the entire automatic grading process. In the double knee image of the X-ray film, the left and right leg images are usually not evenly distributed on both sides of the image, each occupying half. Sometimes the image is on the left, and sometimes on the right, so it is necessary to accurately set the vertical segmentation line to avoid it falling on the medial joint of the left knee joint or the right knee joint, affecting the subsequent lesion identification and lesion grading. Based on this, in a preferred embodiment of the present invention, the left and right knee segmentation unit 21 includes:
[0043] The clustering subunit 211 is used to convert the double knee image into a grayscale image, count the first grayscale values of all pixels in the double knee image, and perform cluster analysis on all the first grayscale values to obtain three cluster categories;
[0044] The configuration subunit 212 is connected to the clustering subunit 211, and is used to extract the first grayscale value of the cluster center of each cluster category, and configure the cluster categories corresponding to the corresponding first grayscale value as bone, muscle and fat in order from large to small, and then determine the grayscale range of each cluster category based on the extracted first grayscale value;
[0045] The setting subunit 213 is connected to the configuration subunit 212, and is used to obtain the second grayscale value of each pixel on the vertical dividing line of the grayscale image. If each second grayscale value meets the preset condition, the vertical dividing line is used as the vertical dividing line. Otherwise, the vertical center line is used as the starting point, and the vertical center line is adjusted left and right according to the preset step length until a vertical dividing line that meets the preset condition is obtained;
[0046] The preset condition is that each second grayscale value is not within the grayscale range associated with bones and more than 80% are not within the grayscale range associated with muscles and fat.
[0047] Specifically, since bone tissue has a high density due to containing a large amount of calcium, it usually appears as a white or brighter area on an X-ray photograph, and its grayscale value is usually higher. Adipose tissue is a relatively low-density tissue, and usually appears as a gray-black or darker area on an X-ray photograph. Its grayscale value is relatively low, which forms a sharp contrast with the bone tissue. The density of muscle tissue is between that of bone and fat, and therefore it usually appears as an off-white or medium-brightness area on an X-ray photograph. Its grayscale value is also between that of bone and fat. Based on the above-mentioned differences in density and X-ray absorption capacity of bones, fat and muscles, they will have different appearances in grayscale images. Therefore, in this embodiment, the bones, fat and muscles in the images of both knees are distinguished by grayscale values, and then the vertical dividing line is set to avoid the bone area to ensure that the segmented left knee image and right knee image contain complete left knee joint information and right knee joint information, respectively.
[0048] To be more specific, since bones have the highest grayscale value and fat has the lowest grayscale value, by performing cluster analysis on each first grayscale value, the cluster category can be determined by the size of the first grayscale value of the corresponding cluster center, whether it belongs to bones, fat, or muscle.
[0049] After determining the specific cluster category, the grayscale range of the cluster category can be determined based on the first grayscale value of the cluster center and the minimum first grayscale value within the preset range of the cluster center. When performing double knee image segmentation, vertical segmentation is required. Although the left knee and the right knee are not necessarily in the center of the image, they have a similar proportion in the image. Even if there is a deviation, it may be that only the fat part exceeds the vertical center line and does not affect the bones. Even if the fat belonging to the left knee is segmented to the right knee, it will not affect the final recognition of the knee joint. Therefore, in this embodiment, when setting the vertical segmentation line, first determine whether the vertical center line meets the preset conditions. If it does, segmentation is performed directly from the vertical center line. Otherwise, the position of the vertical center line can be adjusted left and right according to the preset step size to obtain the vertical segmentation line.
[0050] Among them, adjusting the position of the vertical center line left and right according to the preset step length to obtain the vertical dividing line includes:
[0051] First, move the vertical center line to the left by a preset step size. If it is found that the proportion of pixels on the vertical center line that fall within the grayscale range corresponding to fat does not change significantly after the movement, but the proportion of pixels that fall within the grayscale range corresponding to muscle and the proportion of pixels that fall within the grayscale range corresponding to bones increase, it means that the left knee may be biased to the right in the double knee image, that is, it should not be moved to the left any further. Then move the vertical center line to the right, and so on, until the preset conditions are met.
[0052] In a preferred embodiment of the present invention, the left and right knee splitting unit 22 further includes:
[0053] The first positioning subunit 221 is used to define the left knee image and the right knee image as initial segmentation images, then locate the pixel points belonging to the bones closest to the left and right edges in the initial segmentation images, and generate vertical lines along the two pixel points respectively;
[0054] The second positioning subunit 222 is used to locate the pixel points belonging to the skeleton closest to the upper and lower edges in the initial segmented image, and generate horizontal lines along the two pixel points respectively;
[0055] The re-segmentation subunit 223 is respectively connected to the first positioning subunit 221 and the second positioning subunit 222, and is used to cut out the areas enclosed by vertical lines and horizontal lines from the left knee image and the right knee image respectively, to obtain new left knee images and right knee images and input them into the area segmentation unit.
[0056] Specifically, in this embodiment, by segmenting the segmented left knee image and the right knee image again, irrelevant muscle and fat parts contained therein can be further removed, thereby improving the subsequent recognition rate and accuracy.
[0057] In a preferred embodiment of the present invention, the lesion recognition model adopts the YOLOv5s target detection model, and the CA attention mechanism is embedded in the YOLOv5s target detection model, which can solve the problem of low confidence of the final detection box and large deviation of the predicted position information when the attention mechanism is not embedded.
[0058] In a preferred embodiment of the present invention, the knee arthritis grading model adopts the EfficientNetV2-S network model, and the attention mechanism in the MBConv module in the EfficientNetV2-S network model is the CBAM attention mechanism, and the activation function of the EfficientNetV2-S network model is the SMU activation function.
[0059] Specifically, in this embodiment, by selecting the EfficientNetV2-S network model, the model performance and processing speed are taken into account, and at the same time, detailed features can be fully extracted and network parameters can be utilized; and on this basis, the SE attention mechanism in the MBConv module is replaced by the CBAM attention mechanism and the activation function of the overall network is replaced by the SMU activation function. This attention mechanism combines the spatial and channel attention mechanisms, so that the lesion area receives more attention during model detection, and the SMU activation function better improves the problems of neuron death and overfitting.
[0060] In a preferred embodiment of the present invention, the lesion grade is the KL grade.
[0061] Specifically, the KL level includes five levels, namely:
[0062] Grade 0: Normal knee joint.
[0063] Grade 1: Suspicious knee joint space narrowing is present.
[0064] Grade 2: Small osteophytes and possible joint space narrowing are clearly seen on standing knee radiographs.
[0065] Grade 3: Grade III knee osteoarthritis is characterized by a large number of moderate osteophytes, clear joint space narrowing, and subchondral bone sclerosis.
[0066] Grade 4: There are a large number of large osteophytes, severe joint space narrowing, and obvious subchondral bone sclerosis.
[0067] In a preferred embodiment of the present invention, a human-computer interaction interface 5 is also included. Figure 2As shown, the image acquisition module 1, the preprocessing module 2, the lesion recognition module 3 and the lesion grading module 4 are respectively connected, and the lesion grading module 4 also includes a thermal map generation unit 41, which is used to generate a thermal feature map of the lesion area through the CBAM attention mechanism;
[0068] A selection button 51 and a grading button 52 are provided on the human-computer interaction interface 5. The user selects the double knee images that need to be automatically graded by clicking the selection button 51. When the human-computer interaction interface 5 detects that the grading button 52 is clicked, the acquired double knee images are sent to the preprocessing module 2, and the lesion area, thermal map, lesion grade and its corresponding probability value are displayed accordingly.
[0069] In a preferred embodiment of the present invention, a local database 6 is also included for storing the images of both knees and the lesion areas, thermal maps, lesion levels and their corresponding probability values contained therein.
[0070] Specifically, in this embodiment, by setting up a local database 6, historical diagnostic information, including images of both knees of the patient, identified lesion areas, lesion levels and thermal characteristic maps, are automatically stored as a reliable data source for subsequent refinement of the model to improve diagnostic accuracy and facilitate subsequent case data analysis.
[0071] The above description is only a preferred embodiment of the present invention, and does not limit the implementation mode and protection scope of the present invention. For those skilled in the art, it should be aware that all solutions obtained by equivalent substitutions and obvious changes made using the contents of this specification and illustrations should be included in the protection scope of the present invention.
Claims
1. An automatic grading system for knee arthritis, characterized in that: include: An image acquisition module, used for acquiring images of both knees of patients with knee arthritis; A preprocessing module, connected to the image acquisition module, for segmenting a left knee image and a right knee image from the double knee images, and segmenting a knee joint region of interest from the left knee image and the right knee image respectively; A lesion recognition module, connected to the preprocessing module, for inputting the knee joint region of interest into a lesion recognition model to identify the lesion region therein; The lesion grading module is connected to the lesion identification module and is used to input the lesion area into the knee arthritis grading model to grade the lesion area to obtain the lesion grade of the knee arthritis patient and its corresponding probability value for reference by doctors.
2. The automatic grading system for knee arthritis according to claim 1, characterized in that: The double knee images are obtained by X-ray acquisition of the knee arthritis patient in an upright position, where the upright position is a 45° flexion posterior-anterior weight-bearing position of the knee joint.
3. The automatic grading system for knee arthritis according to claim 1, characterized in that: The preprocessing module comprises: A left and right knee segmentation unit, configured to set a vertical segmentation line according to the gray value distribution of each pixel point in the double knee image, and segment the double knee image from the vertical segmentation line to obtain the left knee image and the right knee image; A region segmentation unit is connected to the left and right knee segmentation units, and is used to generate rectangular frames in the left knee image and the right knee image respectively, and segment out the selected area of the rectangular frame as the knee joint region of interest, wherein the rectangular frame at least includes the lower end of the femur and the upper end of the tibia.
4. The automatic grading system for knee arthritis according to claim 3, characterized in that: The left and right knee segmentation unit comprises: A clustering subunit, used for converting the double knee image into a grayscale image, counting the first grayscale values of all the pixels in the double knee image, and performing cluster analysis on all the first grayscale values to obtain three cluster categories; a configuration subunit, connected to the clustering subunit, for respectively extracting the first grayscale value of the cluster center of each cluster category, and respectively configuring the cluster categories corresponding to the first grayscale value as bone, muscle and fat in descending order, and then determining the grayscale range of each cluster category based on the extracted first grayscale value; a setting subunit, connected to the configuration subunit, for obtaining a second grayscale value of each pixel on the vertical midline of the grayscale image, and if each of the second grayscale values meets a preset condition, taking the vertical midline as the vertical dividing line, otherwise taking the vertical center line as the starting point, adjusting the vertical center line left and right according to a preset step length until the vertical dividing line that meets the preset condition is obtained; The preset condition is that each of the second grayscale values is not within the grayscale range associated with the bone and more than 80% are not within the grayscale range associated with the muscle and the fat.
5. The automatic grading system for knee arthritis according to claim 4, characterized in that: The left and right knee segmentation unit further includes: A first positioning subunit is used to define the left knee image and the right knee image as initial segmentation images, then locate the pixel points belonging to the bones closest to the left and right edges in the initial segmentation images, and generate vertical lines along the two pixel points respectively; A second positioning subunit is used to locate the pixel points belonging to the skeleton closest to the upper and lower edges in the initial segmented image, and generate horizontal lines along the two pixel points respectively; The re-segmentation subunit is respectively connected to the first positioning subunit and the second positioning subunit, and is used to cut out the areas enclosed by the vertical lines and the horizontal lines from the left knee image and the right knee image respectively, to obtain new left knee images and right knee images and input them into the area segmentation unit.
6. The automatic grading system for knee arthritis according to claim 1, characterized in that: The lesion recognition model adopts the YOLOv5s target detection model, and the YOLOv5s target detection model is embedded with a CA attention mechanism.
7. The automatic grading system for knee arthritis according to claim 1, characterized in that: The knee arthritis grading model adopts the EfficientNetV2-S network model, and the attention mechanism in the MBConv module in the EfficientNetV2-S network model is the CBAM attention mechanism, and the activation function of the EfficientNetV2-S network model is the SMU activation function.
8. The automatic grading system for knee arthritis according to claim 1, characterized in that: The lesion grade is the KL grade.
9. The automatic grading system for knee arthritis according to claim 7, characterized in that: It also includes a human-computer interaction interface, which is respectively connected to the image acquisition module, the preprocessing module, the lesion recognition module and the lesion grading module, and the lesion grading module also includes a thermal map generation unit for generating a thermal feature map of the lesion area through the CBAM attention mechanism; The human-computer interaction interface is provided with a selection button and a grading button. The user selects the double knee images that need to be automatically graded by clicking the selection button. When the human-computer interaction interface detects that the grading button is clicked, the acquired double knee images are sent to the preprocessing module, and the lesion area, the thermal map, the lesion grade and its corresponding probability value are displayed accordingly.
10. The automatic grading system for knee arthritis according to claim 9, characterized in that: It also includes a local database for storing the double knee images and the lesion areas, the thermal maps, the lesion levels and their corresponding probability values contained therein.