Knee joint medical image quality evaluation control system and method

Abstract

The invention provides a knee joint medical image quality evaluation control system and method.The knee joint medical image quality evaluation control system comprises a knee joint imaging auxiliary device and an image quality evaluation control device, and the image quality evaluation control device comprises an image quality evaluation unit, an image quality judgment unit and an auxiliary device control unit; the image quality evaluation unit evaluates the image quality of the knee joint X-ray image by using the trained deep learning network model, so that an evaluation result can be automatically and quickly obtained, poor or unqualified image quality can be found in time, a patient can shoot again in time, the evaluation result can be used for guiding the next shooting, and the shooting efficiency is improved. Therefore, a next image with better quality can be obtained. When the image quality is judged to be unqualified, the auxiliary device control unit can correspondingly control the knee joint imaging auxiliary device according to the evaluation result, so that the problem of low correction accuracy caused by manually correcting the re-shot posture of the patient can be avoided or reduced, and the quality of the next image can be improved.

Description

Technical Field

[0001] The invention belongs to the technical field of medical image analysis, and in particular relates to a knee joint medical image (X-ray image) quality assessment control system and method. Background Art

[0002] Knee Joint X-ray Imaging is one of the commonly used imaging technologies in the field of orthopedics and radiology. It is mainly used to evaluate knee joint diseases such as osteoarthritis, fractures, and joint diseases. Generally speaking, knee joint X-ray imaging is the simplest and most valuable imaging examination, so it is often the first choice in related clinical examinations and diagnoses. The image quality of knee joint X-ray images will directly affect the accuracy of doctors' reading and diagnosis, so it is crucial to ensure its image quality.

[0003] However, due to the complexity of the X-ray imaging process and limited irradiation conditions, knee X-ray images often have a variety of image quality problems. The factors that affect image quality are mainly the following two aspects: on the one hand, the patient's position during shooting. Incorrect position or relatively large deviation will cause knee X-ray images to have quality problems such as distortion, distortion, and artifacts; on the other hand, the shooting parameters of the X-ray imaging equipment. Inappropriate parameters will cause knee X-ray images to have quality problems such as insufficient contrast, insufficient clarity, insufficient exposure, and more noise.

[0004] To avoid the above-mentioned image quality problems, traditionally, experienced radiologists or technicians conduct visual inspections of knee X-ray images to evaluate the patient's body position, image contrast, clarity, exposure, noise level, etc. Such traditional evaluation methods rely on the experience and professional knowledge of the examiner, which is very demanding of the examiner and highly subjective. It is difficult to achieve complete uniformity in the evaluation standards among different examiners. In addition, this method takes a long time to evaluate, and it is difficult to give evaluation results to each patient in a timely manner during clinical examinations. When the image quality of a knee X-ray image is found to be unsatisfactory, the patient will have to be notified to retake the image, resulting in delays in diagnosis and treatment. On the other hand, when retaking the image, the radiologist or technician needs to correct the patient's posture based on experience according to the problems in the previous knee X-ray image, and it is also difficult to accurately adjust the patient's posture. Summary of the invention

[0005] The present invention is made to solve the above problems, and aims to provide a system and method that can automatically evaluate the image quality when taking knee joint X-ray images of patients and effectively control the image quality according to the evaluation results. The present invention adopts the following technical solutions:

[0006] The present invention provides a knee joint medical image quality assessment and control system, which has the following technical features: an image quality assessment control device, which is used to assess and control the image quality of a knee joint X-ray image, wherein the image quality assessment control device comprises: an image quality assessment unit, which is used to assess the image quality of the knee joint X-ray image and obtain an assessment result; and an image quality judgment unit, which judges whether the image quality is qualified according to the assessment result and a predetermined image quality judgment rule, and the image quality assessment unit comprises: a target segmentation unit, which is used to perform target detection and instance segmentation of multiple predetermined targets from the knee joint X-ray image through a target detection and segmentation model to obtain a segmentation result, wherein the predetermined target includes multiple key areas, and the segmentation result includes the image part of the key area; and an image scoring unit, which is at least used to score the segmented image part of the key area through a key area scoring model, and the evaluation result includes the score, and the target detection and segmentation model and the key area scoring model are both trained deep learning network models.

[0007] The knee joint medical image quality assessment control system provided by the present invention may also have such technical features, wherein the knee joint X-ray image includes an anteroposterior image and a corresponding lateral image of the knee joint, the multiple key areas are five, including an anteroposterior patella area, an anteroposterior fibula head area, a lateral patella area, a lateral fibula head area, and a lateral medial and lateral condyle area, the image quality assessment unit also includes: an overlapping area segmentation unit, for performing instance segmentation of the lateral medial and lateral condyle areas through a trained overlapping area segmentation model, and segmenting the overlapping area and non-overlapping area of ​​the medial epicondyle and the lateral epicondyle, the image scoring unit includes: an anteroposterior and lateral position recognition and determination unit, for identifying and determining the anteroposterior image and the lateral image from the knee joint X-ray image; and a first key area scoring unit, for scoring the key area through the key area evaluation The sub-model scores the image portions of the anteroposterior patella region, the anteroposterior fibula head region, the lateral patella region, and the lateral fibula head region; the second key area scoring portion is used to score the image portion of the lateral medial and lateral condyle region according to the ratio between the overlapping area and the non-overlapping area and the predetermined key area scoring rules; the anteroposterior image scoring calculation portion calculates the total score of the anteroposterior image at least based on the score of the anteroposterior patella region and the score of the anteroposterior fibula head region; and the lateral image scoring calculation portion calculates the total score of the lateral image at least based on the score of the lateral patella region, the score of the lateral fibula head region, and the scores of the medial and lateral condyle regions, and the image quality judgment unit makes a judgment based on the total score and the image quality judgment rule.

[0008] The knee joint medical image quality assessment control system provided by the present invention may also have such technical features, wherein the predetermined target also includes three key points, namely the midpoint of the lateral femoral upper end medullary cavity, the midpoint of the lateral femoral lower end medullary cavity, and the midpoint of the lateral tibia lower end medullary cavity, the segmentation result also includes the image part of the key points, and the image quality assessment unit also includes: a bending angle calculation unit, which is used to calculate the angle between the long axis of the lateral femur and the long axis of the tibia according to the three key points, the image scoring unit also includes a bending angle scoring part, which scores the bending angle according to the angle and a predetermined angle scoring rule; the lateral image scoring calculation part calculates the total score of the lateral image based on at least the score of the lateral patella area, the score of the lateral fibula head area, the score of the medial and lateral condyle areas, and the score of the bending angle.

[0009] The knee joint medical image quality assessment control system provided by the present invention may also have such technical features, wherein the predetermined target also includes a foreign body, the segmentation result also includes an image portion of the foreign body, the image quality assessment unit also includes: a logo detection unit, which is used to identify a predetermined text logo in the knee joint X-ray image through an optical character recognition model, and obtain its position information when the text logo is identified, and the image scoring unit also includes: a foreign body scoring unit, which is used to score the foreign body in the knee joint anteroposterior image and the knee joint lateral image respectively according to whether the image portion of the foreign body is segmented out; a logo scoring unit, which is used to score the text logo in the knee joint anteroposterior image and the knee joint lateral image respectively according to whether the text logo is identified and the position information when the text logo is identified. score; and a basic quality scoring part, which is used to score the contrast and field of view of the knee joint AP image and the knee joint lateral image respectively through a trained image basic quality scoring model, the AP image scoring calculation part calculates the total score of the AP image according to the score of the AP patella area, the score of the AP fibula head area, and the foreign body score, the score of the marker, the contrast score and the field of view corresponding to the AP image, and the lateral image scoring calculation part calculates the total score of the lateral image according to the score of the lateral patella area, the score of the lateral fibula head area, the score of the medial and lateral condyle areas, the score of the bending angle, and the foreign body score, the score of the marker, the contrast score and the field of view corresponding to the lateral image.

[0010] The knee joint medical image quality assessment control system provided by the present invention may also have such a technical feature, wherein the training data set of the key area scoring model includes a plurality of image portions of the key areas and their annotated scores, and the annotated scores are set according to a predetermined scoring standard. The scoring standard for the anteroposterior fibular head area is: in the corresponding image portion, if the patella is centered, it is rated as a qualified score; if the patella is not centered and does not exceed the edge of the femur, it is rated as a basically qualified score; if the patella is not centered and exceeds the edge of the femur, it is rated as an unqualified score. The scoring standard for the anteroposterior fibular head area is: in the corresponding image portion, if the occluded area of ​​the fibular head is less than 1 / 3, it is rated as a qualified score; if the occluded area of ​​the fibular head is greater than or equal to 1 / 3 and less than 2 / 3, it is rated as a basically qualified score; if the occluded area of ​​the fibular head is greater than or equal to 2 / 3, it is rated as an unqualified score. The scoring criteria for the lateral patella area are: in the corresponding image portion, if the patella and the posterior edge of the femur do not intersect and there is a gap between the two, it is rated as a qualified score; if the patella and the posterior edge of the femur intersect but the gap is visible, it is rated as a basically qualified score; if the patella and the posterior edge of the femur intersect and the gap is not visible, it is rated as an unqualified score. The scoring criteria for the lateral tibiofibular head area are: in the corresponding image portion, if the area of ​​the fibular head blocked is less than 1 / 3, it is rated as a qualified score; if the area of ​​the fibular head blocked is greater than or equal to 1 / 3 and less than 2 / 3, it is rated as a basically qualified score; if the area of ​​the fibular head blocked is greater than or equal to 2 / 3, it is rated as an unqualified score. The key area scoring rule is: calculate the proportion of the non-overlapping area, proportion = area of ​​the non-overlapping area / (area of ​​the overlapping area + area of ​​the non-overlapping area); when the proportion is less than 10%, it is scored as a qualified score; when the proportion is greater than or equal to 10% and less than 20%, it is scored as a basically qualified score; when the proportion is greater than or equal to 20%, it is scored as an unqualified score.

[0011] The knee joint medical image quality assessment control system provided by the present invention may also have such a technical feature, wherein the angle scoring rule is: when the angle is less than 90°, it is scored as the unqualified score; when the angle is greater than or equal to 90° and less than 120°, it is scored as the basic qualified score; when the angle is greater than or equal to 120° and less than 135°, it is scored as the qualified score; when the angle is greater than or equal to 135° and less than 150°, it is scored as the basic qualified score; when the angle is greater than or equal to 150°, it is scored as the unqualified score. The scoring rule of the contrast is: in the knee joint X-ray image, if the bone and tissue cannot be distinguished at all, it is scored as the unqualified score; if the bone anatomical structure can be distinguished but the trabecular structure cannot be distinguished, it is scored as the basic qualified score; if the trabecular structure can be distinguished, it is scored as the qualified score. The scoring rule for the visual field is as follows: if the knee joint structure is not fully included in the knee joint X-ray image, it is scored as the unqualified score; if the knee joint structure is fully included, but the lateral width of the frontal image and the lateral image in the knee joint X-ray image is greater than or equal to its longitudinal width, it is scored as the basically qualified score; if the knee joint structure is fully included, but the key part of the visual field is not in the center of the knee joint X-ray image, it is scored as the basically qualified score; if the knee joint structure is fully included, and the lateral width is less than the longitudinal width, and the key part of the visual field is in the center of the knee joint X-ray image, it is scored as the qualified score.

[0012] The knee joint medical image quality assessment control system provided by the present invention may also have such technical features, wherein the qualified score is 2 points, the basically qualified score is 1 point, the unqualified score is 0 point, the foreign body scoring part is scored as 2 points when the image part of the foreign body is not segmented, 1 point when the image part of one foreign body is segmented, and 2 points when the image part of two or less foreign bodies is segmented, the mark scoring part is scored as 1 point when the text mark is recognized and the text mark is outside the range of the knee joint, 1 point when the text mark is recognized but the text mark is within the range of the knee joint, and 0 point when the text mark is not recognized, and the image quality judgment rule is: The image quality judgment rule is: for the frontal image and the lateral image Determinations are made respectively, and when the corresponding contrast or the corresponding field of view is rated as the unqualified score, it is determined that the image quality is unqualified and needs to be re-photographed; when the contrast and the field of view are both rated as the basic qualified score or the qualified score, further determination is made based on the corresponding total score, and when the total score is less than 60% of the full score, it is determined that the image quality is unqualified and needs to be re-photographed; when the total score is between 60% and 80% of the full score, it is determined that the image quality is qualified and does not need to be re-photographed; when the total score is between 80% and 90% of the full score, it is determined that the image quality is good and does not need to be re-photographed; when the total score is greater than or equal to 90% of the full score, it is determined that the image quality is excellent and does not need to be re-photographed.

[0013] The knee joint medical image quality assessment and control system provided by the present invention may also have such technical features, wherein the image quality assessment control device also includes: a prompt information generation unit, which, when the image quality judgment unit determines that it is unqualified, generates corresponding prompt information according to the scores of each of the five key areas, the score of the angle, the score of the foreign matter, and the score of the text mark; and an input display unit for displaying the prompt information.

[0014] The knee joint medical image quality assessment control system provided by the present invention may also have such technical features, which also includes: a knee joint imaging auxiliary device, used to assist in positioning the patient's knee joint when taking the knee joint X-ray image, wherein the knee joint imaging auxiliary device includes: two lateral positioning plates, whose bending angles are adjustable and detachably arranged in the knee joint imaging auxiliary device, used to assist in positioning the patient's leg placed in the lateral position; a driver, used to drive the lateral positioning plates to change their bending angles; and an adjusting airbag, arranged between one ends of the two lateral positioning plates, used to raise the tibia and fibula of the patient's leg placed in the lateral position to a predetermined angle, the The image quality assessment control device also includes: an auxiliary device control unit, which controls the knee joint imaging auxiliary device according to the assessment result when the image quality judgment unit judges that it is unqualified, so that it can assist in positioning for the next shooting. The auxiliary device control unit includes: an angle difference calculation unit, which is used to calculate the angle difference between the included angle and the predetermined standard angle; an adjustment amount calculation unit, which is used to calculate the adjustment amount used to control the driver according to the angle difference, the parameters of the driver, and the size data of the lateral positioning plate; and a driver control unit, which controls the driver according to the adjustment amount, thereby adjusting the bending angle of the lateral positioning plate.

[0015] The present invention provides a knee joint medical image quality assessment and control method, which has the following technical features and comprises the following steps: an assessment step of evaluating the image quality of the knee joint X-ray image to obtain an assessment result; and a judgment step of judging whether the image quality is qualified according to the assessment result and a predetermined image quality judgment rule, wherein in the assessment step, target detection and instance segmentation of multiple predetermined targets are performed from the knee joint X-ray image through a target detection segmentation model to obtain a segmentation result, wherein the predetermined target includes multiple key areas, the segmentation result includes the image part of the key area, and the segmented image part of the key area is scored at least through a key area scoring model, the assessment result includes the score, and the target detection segmentation model and the key area scoring model are both trained deep learning network models.

[0016] Functions and Effects of the Invention

[0017] The knee joint medical image quality assessment control system and method according to the present invention include an image quality assessment control device, which has an image quality assessment unit and an image quality determination unit, wherein the image quality assessment unit identifies and segments image parts of multiple key areas from the knee joint X-ray image through a trained target detection and segmentation model, and inputs these image parts into a trained key area scoring model to obtain their scores as evaluation results. Since a trained deep learning network model is used to perform target recognition and instance segmentation, as well as automatic scoring, the image quality assessment results can be obtained quickly and accurately, and the obtained assessment results are objective and quantitative, and the assessment results can be conveniently stored and further utilized. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 is a logic block diagram of a knee joint medical image quality assessment control system in an embodiment of the present invention;

[0019] Figure 2 is a stereoscopic diagram of a knee joint imaging assisting device according to an embodiment of the present invention;

[0020] Figure 3 is a logic block diagram of an image quality assessment control device in an embodiment of the present invention;

[0021] Figure 4 is a schematic diagram of a target detection segmentation model in an embodiment of the present invention;

[0022] Figure 5 is a flow chart of training a target detection segmentation model in an embodiment of the present invention;

[0023] Figure 6 is a schematic diagram of annotated anteroposterior image of a knee joint in an embodiment of the present invention;

[0024] Figure 7 Schematic diagram of the annotation of the lateral image of the knee joint in the embodiment of the present invention Figure 1 ;

[0025] Figure 8 Schematic diagram of the annotation of the lateral image of the knee joint in the embodiment of the present invention Figure 2 ;

[0026] Fig. 9 is a labeled schematic diagram of the entire knee joint area in an embodiment of the present invention;

[0027] Fig.10 is a flow chart of training an overlapping region segmentation model in an embodiment of the present invention;

[0028] Fig.11 is a schematic diagram of marking overlapping areas and non-overlapping areas in an image portion of the medial and lateral condyle regions in an embodiment of the present invention;

[0029] Fig.12 is a flow chart of training a key area scoring model in an embodiment of the present invention;

[0030] Fig.13 is an example diagram of patellar azimuth scoring in an embodiment of the present invention;

[0031] Fig.14 is an example diagram of the anteroposterior fibular head regional scoring in an embodiment of the present invention;

[0032] Fig.15 is an example diagram of lateral patellar area scoring in an embodiment of the present invention;

[0033] Fig.16 is an example diagram of lateral tibiofibular head area scoring in an embodiment of the present invention;

[0034] Fig.17 is an example diagram of image contrast scoring in an embodiment of the present invention;

[0035] Fig.18 is an example diagram of image visual field scoring in an embodiment of the present invention;

[0036] Fig.19 is a flow chart of a knee joint medical image quality assessment control method according to an embodiment of the present invention;

[0037] Fig. 20 4 is a flow chart of step S2 in an embodiment of the present invention.

[0038] Reference numerals:

[0039] A knee joint medical image quality assessment control system 100; an image quality assessment control device 10; an image acquisition unit 11; an image quality assessment unit 12; a target segmentation unit 121; a marker detection unit 122; an angle calculation unit 124; an overlapping area segmentation unit 125; an anteroposterior position determination unit 126; an image scoring unit 123; a first key area scoring unit 1231; a second key area scoring unit 1232; a foreign body scoring unit 1233; a marker scoring unit 1234; an angle scoring unit 1235; a basic quality scoring unit 1236; an anteroposterior image scoring calculation unit 1237; a lateral image scoring calculation unit 1238; a total score calculation unit 1239; an image quality determination unit 13; a prompt information generation unit 14; an input display unit 15; an auxiliary device control unit 16; an angle difference value calculation unit 161; an adjustment amount calculation unit 162; a driver control unit 163; an assessment device side communication unit 17; an assessment device side control unit 18; a knee joint imaging auxiliary device 20; a mounting table 21;

[0040] Anteroposterior auxiliary positioning mechanism 22; Anteroposterior contour base plate 221; Anteroposterior positioning plate 222; Sheet-like airbag 223; Lateral auxiliary positioning mechanism 23; Lateral contour base plate 231; Sliding groove 231a; Lateral positioning plate 232; Sheet-like airbag 233; Angle adjustment driver 234; Adjustment airbag 235; X-ray imaging device 200; Storage device 300; Knee joint X-ray image 400; Knee joint Anteroposterior X-ray image 410; Anteroposterior patella area 411; Anteroposterior fibula head area 412; Knee joint lateral X-ray image 420; Lateral patella area 421; Lateral fibula head area 422; Lateral medial and lateral condyle area 423; Lateral femoral upper end medullary cavity midpoint A; Lateral femoral lower end medullary cavity midpoint B; Lateral tibia lower end medullary cavity midpoint C; Overlapping area 423a; Non-overlapping area 423b. DETAILED DESCRIPTION

[0041] In order to make the technical means, creative features, objectives and effects achieved by the present invention easy to understand, the knee joint medical image quality assessment control system and method of the present invention are specifically described below in conjunction with embodiments and drawings.

[0042] <Example>

[0043] Figure 1 It is a logic block diagram of the knee joint medical image quality assessment control system in this embodiment.

[0044] like Figure 1 As shown, a knee joint medical image quality assessment control system 100 (hereinafter referred to as system 100 ) includes an image quality assessment control device 10 and a knee joint imaging auxiliary device 20 , which are communicatively connected to each other.

[0045] The knee joint imaging auxiliary device 20 is arranged near the X-ray imaging device 200 (such as an X-ray machine) to assist in positioning the knee joint when the X-ray imaging device 200 images the patient's knee joint.

[0046] Figure 2 It is a stereoscopic diagram of the knee joint imaging assisting device in this embodiment.

[0047] like Figure 2 As shown, the knee joint imaging auxiliary device 20 includes a loading platform 21 , an auxiliary anteroposterior positioning mechanism 22 and an auxiliary lateral positioning mechanism 23 arranged on the loading platform 21 .

[0048] The auxiliary positioning mechanism 22 for azimuth correction includes an azimuth correction profiling base plate 221, two groups of four azimuth correction positioning plates 222, and a plurality of sheet air bags 223. The azimuth correction profiling base plate 221 has a relatively soft upper surface, and the shape of the middle part of the upper surface corresponds to the shape of the lower surface of the human knee joint, which can better support the azimuth correction knee joint, and the other parts of the upper surface are planes. The two azimuth correction positioning plates 222 are arranged above the azimuth correction profiling base plate 221 in roughly parallel. The sheet air bags 223 are installed on the facing surfaces of a group of two azimuth correction positioning plates 222, and are connected to the inflation and deflation device through an air tube.

[0049] The lateral auxiliary positioning mechanism 23 comprises a lateral profiling base plate 231, two lateral positioning plates 232, a plurality of sheet airbags 233, an angle adjustment transmission assembly, an angle adjustment driver 234, and an adjustment airbag 235. The lateral profiling base plate 231 has a relatively soft upper surface, and the shape of the middle part of the upper surface corresponds to the shape of the outer surface of the human knee joint, which can better support the lateral knee joint, and the other parts of the upper surface are planes. Two parallel sliding grooves 231a are provided on the lateral profiling base plate 231. Two lateral positioning plates 232 are arranged above the lateral profiling base plate 231, and each lateral positioning plate 232 is L-shaped, has a bending portion, and its bending angle is adjustable, for example, it is composed of two plates and a hinged component. The two ends of each lateral positioning plate 232 respectively have a connector, and the connector is movably embedded in the corresponding sliding groove 231a. The angle adjustment transmission assembly and the driver 234 are used to drive the two lateral positioning plates 232 to change the bending angle. The driver 234 can be a motor, for example. Under the drive of the motor, the transmission assembly drives the two ends of each lateral positioning plate 232 to move in a small range toward the direction of approaching or moving away from each other, thereby adjusting the bending angle of the two lateral positioning plates 232. The sheet airbag 233 is installed on the facing surface of the two lateral positioning plates 232, and is connected to the inflation and deflation device through an air pipe. The adjustment airbag 235 is installed on the lateral profiling base plate 231 between one end of the two lateral positioning plates 232, and is also connected to the inflation and deflation device through a corresponding air pipe. After the lateral leg is adjusted to a certain bending angle, the tibia and fibula of the leg are raised by 0 to 15 degrees, which is more conducive to photographing the lateral leg.

[0050] In addition, the four anteroposterior positioning plates 222 can be detachably mounted on the anteroposterior contoured base plate 221, and the two lateral positioning plates 232 can be detachably arranged in the sliding groove 231a. Therefore, optionally, after the auxiliary positioning of the legs of the patient to be filmed is completed using the above-mentioned device, the anteroposterior positioning plates 222 and the lateral positioning plates 232 can also be removed to avoid the generation of foreign body artifacts during filming.

[0051] When in use, the patient lies flat or sideways on the loading platform 21. When taking an X-ray of the knee joint in the anteroposterior position, the knee joint is placed between two sets of anteroposterior positioning plates 222, so that the knee patella faces upward, and the sheet-like airbag 223 is inflated so that the airbag flexibly clamps the leg. When taking an X-ray of the knee joint in the lateral position, the knee joint is placed between two lateral positioning plates 232, so that the knee joint bends accordingly, and the sheet-like airbag 233 is inflated so that the knee joint is flexibly clamped. Optionally, the two ends of the two lateral positioning plates 232 can be driven by a motor to move relative to each other to adjust the bending angle of the lateral positioning plates 232.

[0052] In an alternative solution, one of the frontal auxiliary positioning mechanism 22 and the lateral auxiliary positioning mechanism 23 may also be movably disposed on the support table 21 in the horizontal direction, so that the distance between the two mechanisms can be adjusted to better suit patients of different body shapes.

[0053] The image quality evaluation control device 10 is also connected to the X-ray imaging device 200 and the storage device 300 (such as a storage server) for communication. The image quality evaluation control device 10 obtains the knee joint X-ray image 400 taken by the X-ray imaging device 200, performs image quality detection on it, and controls the knee joint imaging auxiliary device 20 according to the evaluation result to assist the X-ray imaging device 200 in obtaining the knee joint X-ray image 400 with better image quality, and stores the knee joint X-ray image 400 and its corresponding evaluation result in the storage device 300.

[0054] Figure 3 It is a logic block diagram of the image quality evaluation control device in this embodiment.

[0055] like Figure 3 As shown, the image quality assessment control device 10 includes an image acquisition unit 11, an image quality assessment unit 12, an image quality determination unit 13, a prompt information generation unit 14, an input display unit 15, an auxiliary device control unit 16, an assessment device side communication unit 17, and an assessment device side control unit 18.

[0056] The image acquisition unit 11 is used to acquire the knee joint X-ray image 400 taken by the X-ray imaging device 200. The image acquisition unit 11 may include an image acquisition card, which is connected to the X-ray imaging device 200 and can acquire the image displayed by the X-ray imaging device 200 in real time; or, the image acquisition unit 11 may include a preset interface, which can be connected to the X-ray imaging device 200 through a cable and receive image data. The acquired knee joint X-ray image 400 is a grayscale image, in which the black is the background; the gray and white are the knee joint tissue and bones, and may be foreign matter irrelevant to the diagnosis.

[0057] The image quality evaluation unit 12 is used to evaluate the image quality of the acquired knee joint X-ray image 400 to obtain an evaluation result. The image quality evaluation unit 12 includes a target segmentation unit 121, a marker detection unit 122, an angle calculation unit 124, an overlap region segmentation unit 125, a frontal and lateral position identification and determination unit 126, and an image scoring unit 123.

[0058] The anteroposterior and lateral position recognition and determination unit 126 is used to recognize and determine the anteroposterior image of the knee joint and the lateral image of the knee joint in the knee joint X-ray image 400 .

[0059] The target segmentation unit 121 is used to detect and segment multiple predetermined targets on the knee joint X-ray image 400, identify zero, one or more predetermined targets from the image 400, and segment the image portion of the target.

[0060] In this embodiment, the predetermined targets include foreign matter, five key areas, three key points, and the entire knee joint area in the knee joint frontal image and the lateral image.

[0061] Foreign matter includes bright objects in the image that are not relevant to diagnosis, such as metal shadows caused by keys, jewelry, mobile phones, etc.

[0062] The five key areas include: anteroposterior patella area, anteroposterior fibular head area, lateral patella area, lateral tibiofibular head area, and lateral medial and lateral condyle area. These five key areas are the five key areas for medical imaging of the knee joint recommended by the Radiology Branch of the Chinese Medical Association.

[0063] The three key points include: the midpoint of the medullary cavity at the upper end of the femur in the lateral view, the midpoint of the medullary cavity at the lower end of the femur in the lateral view, and the midpoint of the medullary cavity at the lower end of the tibia in the lateral view.

[0064] The entire knee joint area is also the knee joint area in the image, and the area other than this area in the image is the black background part.

[0065] In this embodiment, the target segmentation unit 121 includes a trained target detection and segmentation model, which is a deep learning network model.

[0066] Figure 4 Schematic diagram of the target detection segmentation model in this embodiment.

[0067] like Figure 4As shown, the detection and segmentation model is preferably a YOLO (You Only Look Once version) model, whose input is a preprocessed knee joint X-ray image 400. The backbone network (Backbone) of the model extracts features from the input image, the neck (Neck) performs feature fusion based on the extracted features, and the head (Head, also known as the detection head) outputs the detection frame and segmentation mask of the detected target according to the fused features. Based on the detection frame and the segmentation mask, the image part of each target can be obtained. In this embodiment, the image part of the five key areas is also the rectangular image part in the corresponding detection frame.

[0068] Figure 5 It is a flowchart of training the target detection segmentation model in this embodiment.

[0069] like Figure 5 As shown, the YOLO model can be trained by following the steps below:

[0070] Step SA1, collect a large number of knee joint X-ray images, and exclude images with predetermined types of conditions, such as images with fractures, congenital variations or acquired deformities.

[0071] In this embodiment, knee X-ray images of all patients aged between 7 and 70 years old from 2020 to 2022 were collected, and images of predetermined types of conditions were included, such as images with fractures, congenital variations, or acquired deformities, etc. were excluded.

[0072] Step SA2, in each knee joint X-ray image, five key areas, three key points, the entire knee joint area and foreign bodies are labeled accordingly to form a training data set.

[0073] Figure 6 Schematic diagram of the annotated anteroposterior image of the knee joint in this embodiment.

[0074] like Figure 6 As shown, when annotating the knee joint anteroposterior image 410, the anteroposterior patellar region 411 completely including the patella and the anteroposterior fibular head region 412 completely including the fibular head are respectively framed, and corresponding labels (such as different texts) are set respectively. The segmentation result obtained by the target segmentation unit 123 after performing instance segmentation is also the rectangular multiple image parts in the above-mentioned regions 411 and 412.

[0075] Figure 7 is a schematic diagram of the annotation of the lateral image of the knee joint in this embodiment Figure 1 , Figure 8 is a schematic diagram of the annotation of the lateral image of the knee joint in this embodiment Figure 2 .

[0076] like Figure 7 and Figure 8 As shown, when the lateral image 420 of the knee joint is annotated, the lateral patellar region 421 completely including the patella, the lateral fibular head region 422 completely including the fibular head, and the lateral medial and lateral condyle region 423 including the medial condyle and lateral condyle of the tibia are respectively framed, and corresponding marks (such as different texts) are respectively set, and the midpoint A of the lateral upper end of the femur medullary cavity (when the image is placed forwardly with the patient's head above, the midpoint in the width direction of the upper end of the femur in the image), the midpoint B of the lateral lower end of the femur medullary cavity (when placed forwardly, the midpoint in the width direction of the lower end of the femur in the figure), and the midpoint C of the lateral lower end of the tibia medullary cavity (when placed forwardly, the midpoint in the width direction of the lower end of the tibia in the figure). If the above-mentioned foreign matter exists in the figure, the area containing each foreign matter is further framed, and the corresponding mark is set.

[0077] Fig. 9 is a labeled schematic diagram of the entire knee joint area in this embodiment, Fig. 9 A is the frontal image of the knee joint before annotation, B is the frontal image of the knee joint after annotation, C is the lateral image of the knee joint before annotation, and D is the lateral image of the knee joint after annotation.

[0078] like Fig. 9 As shown, in the frontal image and the lateral image of the knee joint, the entire knee joint area in the image is selected by enclosing a polygon, and a corresponding mark (such as text) is set for the selected image area (optionally also the background area).

[0079] Step SA3: input a batch of data in the training data set into the target detection segmentation model to train it.

[0080] Step SA4, determines whether the predetermined iteration stop condition is reached, and if it is not, returns to step SA3, that is, inputs the next batch of data and continues iterative training.

[0081] The iteration stopping condition may be reaching a predetermined number of training rounds, or testing the model using a test data set, and the model reaching a predetermined prediction accuracy, etc.

[0082] Step SA5, when the judgment in step SA4 is yes, stop the iteration and obtain the trained target detection and segmentation model, which can detect and identify the above five key areas, three key points, the overall knee joint area and foreign matter in the newly input knee joint X-ray image, and can output the corresponding segmentation results. The output segmentation results are similar to the corresponding annotation results in the above annotation schematic diagram.

[0083] The logo detection unit 122 is used to detect a predetermined text logo in the knee joint X-ray image 400, and obtain its position information when the text logo is detected. In this embodiment, the logo detection unit 122 includes an optical character recognition (OCR) model, which is used to detect the white text logos "L" and "R" in the image, which are used to identify that the frontal image and the lateral image are images of the patient's left knee joint and right knee joint, respectively. When the text logo is detected, the position information (coordinates) of the logo can be further obtained. In an alternative, according to the text logo used in the actual image, the logo detection unit 122 can also be used to detect other text logos in the image, such as for detecting the Chinese logos "left" and "right".

[0084] The angle calculation unit 124 calculates the angle between the long axis of the femur and the long axis of the tibia in the lateral position according to the three key points segmented by the target segmentation unit 121. Specifically, the calculation is performed according to the coordinates of the detection frames of the three key points A to C, and a line is connected between point A and point B, and a line is connected between point B and point C, and then the angle α between the two lines is calculated.

[0085] The overlapping region segmentation unit 125 includes an overlapping region segmentation model, which is also a deep learning network model, whose input is the image portion of the lateral medial and lateral condyle regions segmented by the target segmentation unit 121, and the output is the segmentation result of the overlapping region and the non-overlapping region in the image portion, and the segmentation result also includes a detection frame and a segmentation mask. Among them, the overlapping region refers to the overlapping region of the medial epicondyle and the lateral epicondyle, and the non-overlapping region refers to the non-overlapping region of the two, which is usually the edge portion of the medial epicondyle or the lateral epicondyle. Preferably, the overlapping region segmentation model is also a YOLO model.

[0086] Fig.10 It is a flowchart of training the overlapping region segmentation model in this embodiment.

[0087] like Fig.10 As shown, the overlapping region segmentation model can be trained by following the steps below:

[0088] Step SD1, collect a large number of knee joint X-ray images, and exclude images with fractures, congenital variations or acquired deformities.

[0089] Step SD2, using the above detection and segmentation model to segment the image parts of five key areas from the knee joint X-ray image, and retain the image parts of the medial and lateral condyle areas in the lateral view.

[0090] Step SD3: in each lateral image portion of the medial and lateral condyle regions, the overlapping region and the non-overlapping region of the medial epicondyle and the lateral epicondyle are respectively marked, and corresponding annotations are respectively set for them, thereby forming a training data set.

[0091] Fig.111 is a schematic diagram of the annotation of overlapping areas and non-overlapping areas in the image part of the medial and lateral condyle regions in this embodiment. The upper row in the figure is the unannotated original image part, and the lower row is the corresponding annotated image.

[0092] like Fig.11 As shown, when labeling, an experienced radiologist or technician paints the overlapping area 423a and the non-overlapping area 423b of the medial epicondyle and the lateral epicondyle in different colors (or different shades) in the image part, and sets corresponding labels (such as different texts) for these two areas. In this embodiment, a large number (such as 1000) of image parts of the medial and lateral condyle regions are labeled to form a training data set.

[0093] Step SD4, inputting a batch of data in the training data set into the key area scoring model to train it.

[0094] Step SD5, determines whether the predetermined iteration stop condition is reached. If it is not reached, returns to step SD4, that is, inputs the next batch of data and continues iterative training.

[0095] Step SD6, when the judgment in step SD5 is yes, stop the iteration and obtain the trained overlapping area segmentation model.

[0096] The image scoring unit 123 is used to score the knee joint X-ray image 400 according to the above-mentioned target segmentation result, the mark detection result, and the predetermined scoring rule. The image scoring unit 123 includes a first key area scoring part 1231, a second key area scoring part 1232, a foreign body scoring part 1233, a mark scoring part 1234, a bending angle scoring part 1235, a basic quality scoring part 1236, an anteroposterior image scoring calculation part 1237, and a lateral image scoring calculation part 1238. That is, the image scoring unit 123 calculates a corresponding total score for the anteroposterior image and the lateral image of the knee joint.

[0097] The first key area scoring part 1231 includes a key area scoring model, which is a deep learning network model, whose input is the image part of four key areas (except the other four lateral condyle areas) segmented by the target segmentation part 121, and the output is the score of the corresponding image part.

[0098] Fig.12 It is a flowchart of training the key area scoring model in this embodiment.

[0099] like Fig.12 As shown, the key area scoring model can be trained by following the steps below:

[0100] Step SC1, collect a large number of knee joint X-ray images, and exclude images with fractures, congenital variations or acquired deformities.

[0101] Step SC2, using the above-mentioned target detection segmentation model to segment image parts of five key areas from the knee joint X-ray image, and retain image parts of the other four key areas except the lateral medial and lateral condyle areas.

[0102] Step SC3: score each image part according to the corresponding scoring standard, and use the score as the label corresponding to the image part, thereby forming a training data set.

[0103] Among them, according to the standards of the Radiology Branch of the Chinese Medical Association, experienced radiologists or technicians will score the image quality of the image parts of the four key areas respectively. If the image quality of the image part is unqualified, it will be scored as 0 points, if it is basically qualified, it will be scored as 1 point, and if it is qualified, it will be scored as 2 points.

[0104] Fig.13 is an example diagram of the patellar azimuth score in this embodiment.

[0105] like Fig.13 As shown in the figure, the scoring criteria for the anteroposterior patella area are: if the patella is centered in the corresponding image part and the image is clear, it is scored as 2 points (the three image parts shown in A in the figure); if the patella is not centered but does not exceed the edge of the femur and the image is clear, it is scored as 1 point (the three images shown in B in the figure); if the patella is not centered and exceeds the edge of the femur, or the image is unclear, it is scored as 0 points (the three images shown in C in the figure).

[0106] Fig.14 4 is an example diagram of the anteroposterior fibular head area scoring in this embodiment.

[0107] like Fig.14 As shown in the figure, the scoring criteria for the anteroposterior fibular head area are: if in the corresponding image part, the blocked area of ​​the fibular head is less than one-third and the image is clear, it is scored as 2 points (the three pictures shown in A in the figure); if the blocked area of ​​the fibular head is greater than or equal to one-third and less than two-thirds, and the image is clear, it is scored as 1 point (the three pictures shown in B in the figure); if the blocked area of ​​the fibular head is greater than or equal to two-thirds, or the image is unclear, it is scored as 0 points (the three pictures shown in C in the figure).

[0108] Fig.15 is an example diagram of the lateral patellar area scoring in this embodiment.

[0109] like Fig.15As shown in the figure, the scoring criteria for the lateral patella area are: if the patella and the posterior edge of the femur do not intersect in the corresponding image part, there is an obvious gap between the two, and the image is clear, it is scored as 2 points (the three pictures shown in A in the figure); if the patella and the posterior edge of the femur intersect a little, the gap between the two is still visible, and the image is clear, it is scored as 1 point (the three pictures shown in B in the figure); if the patella and the posterior edge of the femur intersect, the gap between the two is not visible, or the image is unclear, it is scored as 0 points (the three pictures shown in C in the figure).

[0110] Fig.16 4 is an example diagram of the lateral tibiofibular head area scoring in this embodiment.

[0111] like Fig.16 As shown in the figure, the scoring criteria for the lateral tibiofibular head area are: if the blocked area of ​​the fibular head in the corresponding image part is less than one-third and the image is clear, it is scored as 2 points (the three pictures shown in A in the figure); if the blocked area of ​​the fibular head is greater than or equal to one-third and less than two-thirds, and the image is clear, it is scored as 1 point (the three pictures shown in B in the figure); if the blocked area of ​​the fibular head is greater than or equal to two-thirds, or the image is unclear, it is scored as 0 points (the three pictures shown in C in the figure).

[0112] If the image part of a key area is not segmented, the corresponding score is also 0.

[0113] Step SC4, inputting a batch of data in the training data set into the key area scoring model to train it.

[0114] Step SC5, judging whether the predetermined iteration stop condition is reached, and if it is judged as no, returning to step SC4, that is, inputting the next batch of data and continuing the iterative training.

[0115] Step SC6, when the judgment in step SC5 is yes, the iteration is stopped to obtain a trained key area scoring model, which can output the score of the input image part.

[0116] The second key area scoring part 1233 calculates the proportion of the non-overlapping area according to the overlapping area and the non-overlapping area segmented by the overlapping area segmentation part 125, and scores according to the proportion and the predetermined key area scoring rule. Proportion = area of ​​the non-overlapping area / (area of ​​the overlapping area + area of ​​the non-overlapping area). When the proportion is less than 10%, it is scored as 2 points; when the proportion is greater than or equal to 10% and less than 20%, it is scored as 1 point; when the proportion is greater than or equal to 20%, it is scored as 0 points.

[0117] The foreign body scoring part 1233 scores according to whether the image part of the foreign body is segmented out. The knee joint frontal image and lateral image are scored separately. When the image part of more than two foreign bodies is segmented out, the score is 0 points, when the image part of one foreign body is segmented out, the score is 1 point, and if the image part is not segmented out, the score is 2 points.

[0118] The identification scoring part 1234 scores according to whether the identification detection unit 122 recognizes the predetermined text identification and its position information when the text identification is recognized, and scores the knee joint anteroposterior image and lateral image respectively. Among them, when the "L" or "R" text identification is recognized in the anteroposterior image or lateral image of the knee joint, it is determined whether the text identification is within the knee joint range based on the corresponding position information (coordinates), the position information of the segmented anteroposterior knee joint area (detection frame coordinates), and the position information of the lateral knee joint area. Among them, the knee joint area can also be segmented from the X-ray image using a deep learning network model. When the corresponding text identification is identified and the identification is outside the knee joint area, it is scored as 2 points; when the text identification is identified but the identification is within the knee joint area, it is scored as 1 point; when the text identification is not identified, it is scored as 0 points.

[0119] The bending angle scoring part 1235 scores according to the angle between the lateral femur long axis and the tibia long axis calculated by the angle calculation part 124 and the predetermined angle scoring rules, and scores 0 points when the angle α is less than 90°; 1 point when the angle α is greater than or equal to 90° and less than 120°; 2 points when the angle α is greater than or equal to 120° and less than 135°; 1 point when the angle α is greater than or equal to 135° and less than 150°; and 0 points when the angle α is greater than or equal to 150°.

[0120] The basic quality scoring part 1236 includes an image basic quality scoring model, which is a trained deep learning network model, whose input is the preprocessed knee joint anteroposterior image and lateral image, and whose output is the basic imaging quality score for the anteroposterior image and the lateral image, respectively. In this embodiment, the model outputs two aspects of the score, one is the score of the image contrast, and the other is the score of the image field of view.

[0121] Similarly, when training the basic image quality scoring model, images with different contrasts are labeled and scored accordingly and used as training data, and images with different fields of view are labeled and scored accordingly and used as training data to train the model.

[0122] In an alternative solution, two deep learning network models may be used to respectively train a scoring model for scoring image contrast and a scoring model for scoring image field of view.

[0123] Fig.17 is an example diagram of image contrast scoring in this embodiment.

[0124] like Fig.17 As shown in the figure, the scoring criteria for image contrast are: for the frontal image or the lateral image, in the corresponding image, if the bones and tissues cannot be distinguished at all, it means that the contrast is seriously insufficient, and it is scored as 0 points (the column shown in the figure A); if the bone anatomical structure can be distinguished, but the fine trabecular structure cannot be distinguished, it means that the contrast is insufficient, and it is scored as 1 point (the column shown in the figure B); if the anatomical structure and the fine trabecular structure can be distinguished, it is scored as 2 points (the column shown in the figure C).

[0125] Fig.18 is an example diagram of image visual field scoring in this embodiment.

[0126] like Fig.18 As shown in the figure, the scoring criteria for the image field of view are: for the anteroposterior image or lateral image, if the knee joint structure is not completely included in the corresponding image, it is scored as 0 points (the column shown in A in the figure); if the knee joint structure is completely included, but the lateral width of the image (anteroposterior image and lateral image) is greater than or equal to the longitudinal width, it is scored as 1 point (the column shown in B in the figure); if the knee joint structure is completely included and the lateral width of the image is less than the longitudinal width, it is scored as 2 points (the column shown in C in the figure).

[0127] In an alternative solution, the image field of view can be further scored in combination with whether the key part of the field of view is in the center of the entire image. Specifically, for the anteroposterior image, the offset of the detection frame relative to the center of the anteroposterior image is calculated based on the coordinates of the detection frame of the segmented anteroposterior patella image portion and the pixel size of the entire anteroposterior image. For the lateral image, the above-mentioned key part of the field of view is detected in the center of the entire image based on the segmented lateral patella image portion. The scoring criteria can be corresponding to: in the corresponding image, if the knee joint structure is not completely included, it is scored as 0 points; if the knee joint structure is completely included, but the horizontal width of the image is greater than or equal to the vertical width, it is scored as 1 point; if the knee joint structure is completely included, but the offset of the key part of the field of view is greater than or equal to the predetermined threshold, it is scored as 1 point; if the knee joint structure is completely included, and the horizontal width of the image is less than the vertical width, and the offset of the key part of the field of view is less than the predetermined threshold, it is scored as 2 points.

[0128] The AP image score calculation part 1237 adds the scores of multiple key areas related to the AP image of the knee joint, the score of foreign matter, the score of text labels, and the score of basic image quality to obtain the score of the AP image of the knee joint.

[0129] Among them, for the AP images, the above scores that need to be taken into consideration include: the visual field score of the AP images, the contrast score of the AP images, the identification score of the AP images, the foreign body score of the AP images, and the scores of two corresponding key areas in the AP images (the AP patella area and the AP fibular head area).

[0130] The lateral image score calculation part 1238 adds the scores of multiple key areas related to the lateral image of the knee joint, the score of foreign matter, the score of text marks, the score of angles, and the score of basic image quality to obtain the score of the lateral image of the knee joint.

[0131] Among them, for the lateral image, the above scores that need to be taken into consideration include: the visual field score of the lateral image, the contrast score of the lateral image, the identification score of the lateral image, the foreign body score of the lateral image, the scores of the three corresponding key areas in the lateral image (lateral patella area, lateral fibular head area and medial and lateral condyle area), and the score of the knee flexion angle in the lateral image.

[0132] The image quality determination unit 13 is used to determine whether the image quality of the knee joint X-ray image 400 is qualified according to the evaluation result of the image quality evaluation unit 12 and a predetermined image quality determination rule. In this embodiment, the image quality determination rule is as follows:

[0133] The knee joint anteroposterior and lateral images were evaluated separately.

[0134] First, determine whether the score of the contrast of the corresponding image is 0 (unqualified score). If it is 0, it is directly determined that the image quality is unqualified and needs to be re-photographed.

[0135] Secondly, determine whether the score of the field of view of the corresponding image is 0 points. If it is 0 points, it is directly determined that the image quality is unqualified and needs to be re-photographed.

[0136] Then, when the scores of contrast and field of view are both 1 or 2 points, that is, at least basically qualified, further judgment is made based on the total score. When the total score is less than 60% of the full score of the image, the image quality is judged to be unqualified and needs to be re-shot; when the total score is between 60% and 80% of the full score, it is judged to be qualified and no re-shooting is required; when the total score is between 80% and 90% of the full score, it is judged to be good and no re-shooting is required; when the total score is greater than or equal to 90% of the full score, it is judged to be excellent and no re-shooting is required.

[0137] The prompt information generating unit 14 is used to generate corresponding prompt information according to the evaluation results, for example, generating prompt information of several unqualified ones in five key areas according to the scores, generating prompt information of the presence of foreign matter according to the scores, generating prompt information of missing logo according to the logo score, generating prompt information of too large or too small angle according to the angle and the angle score, generating prompt information of insufficient contrast or unsatisfactory field of view according to the basic image quality score, and so on.

[0138] In this embodiment, the evaluation device 20 pre-stores a variety of reference prompt images and reference videos. The prompt information generation unit 14 generates corresponding text prompt information according to the evaluation results, and finds the corresponding reference prompt image or reference video, and displays them together, so as to better prompt the doctor or technician to take improvement measures. For example, when the evaluation result includes the situation that the patella is not centered on the left and right of the knee joint, the corresponding prompt text information is displayed on the display screen, prompting the doctor or technician to adjust the patient's toe position, and the reference prompt image is displayed at the same time.

[0139] The input display unit 15 is used to display the evaluation results and corresponding prompt information to prompt the radiologist or technician who takes the knee joint X-ray image of the image quality problems existing in the image.

[0140] The auxiliary device control unit 16 is used to control the knee joint imaging auxiliary device 20 according to the evaluation result, so that it can position the patient's knee joint at a more accurate position when the X-ray imaging device 200 takes the next shot, thereby improving the image quality of the next knee joint X-ray image 400. The auxiliary device control unit 16 includes an angle difference calculation unit 161, an adjustment amount calculation unit 162, and a driver control unit 163.

[0141] The angle difference calculation unit 161 is used to calculate the angle difference between the included angle and a predetermined standard angle, for example, by subtracting the calculated included angle from the standard angle to obtain the difference. A positive angle difference indicates that the included angle in the image is too large, and a negative angle difference indicates that the included angle is too small.

[0142] The adjustment amount calculation unit 162 calculates the adjustment amount for controlling the driver based on the calculated angle difference, the predetermined driver parameters and the relevant dimensional data of the knee joint imaging auxiliary device 20. For example, when the driver is a motor, the rotation direction and number of revolutions of the motor in order to adjust the angle from the current angle to the standard angle are calculated based on the angle difference, the parameters of the motor, the transmission assembly and the dimensional data of the lateral positioning plate 232.

[0143] The driver control unit 163 controls the driver (motor) of the knee joint imaging assisting device 20 according to the adjustment amount calculated by the adjustment amount calculating unit 162 , thereby adjusting the bending angles of the two lateral positioning plates 232 .

[0144] In an alternative solution, the functions or part of the functions of the auxiliary device control unit 16 may also be arranged on the auxiliary device 20 side. For example, the evaluation device 10 sends the calculated angle value that needs to be adjusted to the auxiliary device 20, and the auxiliary device 20 further calculates the corresponding number of motor rotations and controls the motor accordingly.

[0145] The evaluation device side communication unit 17 is used to communicate with other devices, including transmitting the temporarily stored knee joint X-ray images and corresponding evaluation results to the storage device 300. In an alternative solution, only the knee joint X-ray images with qualified image quality can be transmitted to the storage device 300 according to the evaluation results.

[0146] The evaluation device-side control unit 18 is used to control the operation of the above-mentioned units.

[0147] Optionally, the evaluation device 20 may further include a historical data statistics unit, which is used to obtain the knee joint X-ray image and its corresponding evaluation result from the storage device 300, that is, to obtain historical data, and then perform statistical analysis based on the historical data.

[0148] Optionally, the image quality assessment control device 10 may further include an information storage unit for temporarily storing the acquired knee joint X-ray images and the assessment results thereof.

[0149] Fig.19 4 is a flow chart of the knee joint medical image quality assessment control method in this embodiment.

[0150] like Fig.19 As shown, the method for quality assessment and control of knee joint X-ray images using the above system 100 includes the following steps:

[0151] In step S1, the image acquisition unit 11 acquires a knee joint X-ray image taken by the X-ray imaging device 200, which includes a set of knee joint anteroposterior images and knee joint lateral images.

[0152] Step S2, the image quality assessment unit 12 evaluates the knee joint X-ray image to obtain an assessment result, including the score of each of the five key areas, the score of the foreign body, the score of the mark, the angle between the long axis of the lateral femur and the long axis of the tibia, the score of the angle, the score of the basic image quality, and the total score.

[0153] Fig. 20 is a flow chart of step S2 in this embodiment.

[0154] like Fig. 20 As shown, step S2 specifically includes the following sub-steps:

[0155] In step S2-0, the anteroposterior and lateral position determining unit 126 recognizes and determines the anteroposterior image and the lateral image of the knee joint from the knee joint X-ray image.

[0156] Step S2-1, the target segmentation unit 121 uses the above detection and segmentation model to perform target detection and instance segmentation on the knee joint X-ray image for predetermined targets, and the predetermined targets include five key areas, three key points, the entire knee joint area, and foreign matter.

[0157] Step S2-2: the first key region scoring part 1231 uses the above key region scoring model to score the image parts of the four key regions respectively.

[0158] Step S2-3: the overlapping region segmentation unit 125 further segments the overlapping region and non-overlapping region of the medial epicondyle and the lateral epicondyle from the image portion of the medial and lateral condyle regions in the lateral view using the overlapping region segmentation model.

[0159] Step S2-4, the second key area scoring part 1232 calculates the proportion of the non-overlapping area according to the segmented overlapping area and non-overlapping area, and scores the lateral medial and lateral condyle area according to the proportion and the predetermined key area scoring rules.

[0160] Step S2-5: The bending angle calculation unit 124 calculates the angle between the long axis of the femur and the long axis of the tibia in the lateral position according to the three segmented key points.

[0161] Step S2-6: the bending angle scoring part 1233 performs scoring according to the calculated included angle and a predetermined angle scoring rule.

[0162] In step S2-7, the foreign body scoring part 1234 performs foreign body scoring on the frontal image and the lateral image respectively according to the foreign body segmentation result.

[0163] In step S2-8, the logo detection unit 122 detects corresponding text logos in the frontal image and the lateral image respectively using the above OCR model.

[0164] Step S2-9: the mark scoring part 1235 performs mark scoring on the frontal image and the lateral image respectively according to the detection result of the text mark.

[0165] In step S2-10, the basic quality scoring part 1236 uses the above-mentioned basic image quality scoring model to score the contrast and field of view of the frontal image and the lateral image respectively.

[0166] Step S2-11, the AP image score calculation part 1237 calculates the total score of the AP image based on the visual field score, contrast score, marker score, foreign body score, AP patella area score, and AP fibula head area score of the AP image.

[0167] Step S2-12, the lateral image score calculation part 1238 calculates the total score of the lateral image based on the visual field score, contrast score, marker score, foreign body score, lateral patella area score, lateral fibular head area score, medial and lateral condyle joint area score, and bending angle score of the lateral image.

[0168] In step S3 , the evaluation device-side communication unit 17 transmits the knee joint X-ray image and the evaluation result thereof to the storage device 300 .

[0169] Step S4, the image quality determination unit 13 determines whether the image quality of the knee joint X-ray image is qualified and whether it needs to be re-shot according to the evaluation result and the predetermined image quality determination rule.

[0170] Step S5: When the judgment result in step S4 is yes, the prompt information generating unit 14 generates corresponding reshooting prompt information according to the evaluation result, and the input display unit 15 displays the information.

[0171] Step S6, the auxiliary device control unit 16 controls the driver of the knee joint imaging auxiliary device 20 according to the angle between the lateral femoral long axis and the tibia long axis and the predetermined standard angle range, so as to adjust the bending angle of the two lateral positioning plates 232.

[0172] Through the above-mentioned system 100 and method, the image quality of the knee joint X-ray image 400 can be quickly and automatically evaluated, and when the quality is evaluated as unqualified, a variety of prompt information can be given to the radiologist or technician to re-shoot the image. The bending angle of the corresponding lateral positioning plate of the auxiliary device 20 can also be automatically adjusted, thereby ensuring the image quality of the knee joint X-ray image 400.

[0173] After testing, running the program that implements the above method on a GTX4090 graphics card can accurately judge the image quality of a knee joint X-ray image within 0.5 seconds and output the evaluation results, which is much shorter than the time required for manual image quality evaluation.

[0174] Functions and Effects of the Embodiments

[0175] According to the knee joint medical image quality detection control method and system provided by this embodiment, it includes a knee joint imaging auxiliary device and an image quality assessment control device. The image quality assessment control device includes an image quality assessment unit, an image quality determination unit, and an auxiliary device control unit. Since the image quality assessment unit uses a trained deep learning network model to evaluate the image quality of the knee joint X-ray image, it can automatically and quickly obtain the evaluation result. When the image quality is poor or unqualified, it can be discovered in time and the patient can be asked to retake the photo in time, and the evaluation result can be used to guide the next shooting to obtain the next image with better quality. Further, when the image quality determination unit determines that the image quality is unqualified, the auxiliary device control unit can automatically control the knee joint imaging auxiliary device according to the evaluation result, thereby avoiding or reducing the problem of low correction accuracy caused by manually correcting the patient's reshooting posture, which is also conducive to improving the quality of the next image.

[0176] In an embodiment, the knee joint imaging auxiliary device includes two lateral positioning plates with adjustable bending angles and a motor driving the bending thereof. The image quality assessment unit can segment three key points from the image using the trained YOLO model and calculate the corresponding angles, and control the motor according to the difference between the current angle and the standard angle, so that the angle between the patient's femur long axis and the tibia long axis can be adjusted to be close to the standard angle. Since each patient has a different body shape, such as leg thickness, leg muscle mass and fat mass and their distribution, the legs of multiple patients are placed between the lateral positioning plates with the same bending angle, and the angles between the femur long axis and the tibia long axis of these patients will also be different. In the above manner, the patient's leg bending angle can be adjusted more accurately according to the evaluation result of the last image taken.

[0177] Furthermore, the knee joint imaging assisting device also includes an AP positioning plate for assisting in taking AP images, and sheet-like air bags are installed on the facing surfaces of a pair of positioning plates, so that the patient's two legs can be flexibly clamped by the inflated air bags, making the device suitable for patients of different body shapes and causing no harm or discomfort to the patient during shooting positioning.

[0178] Furthermore, the image quality assessment unit can also use the trained YOLO model to segment five key areas from the image, and then use the trained scoring model to automatically score four of the key areas, and use the segmentation model to segment the fifth key area into overlapping areas and non-overlapping areas, and then automatically score based on their proportions. Since these five key areas are the inspection areas recommended by the Radiology Branch of the Chinese Medical Association, the evaluation results obtained by target detection and scoring can well represent the image quality level and have a good reference value in clinical applications. And because the deep learning network model is also used to automatically generate scores, it not only saves a lot of manpower and time, but also generates objective and quantitative evaluation results, which is conducive to further use of the evaluation results.

[0179] Furthermore, when training the scoring model, the image parts of the key areas for scoring are labeled by experienced radiologists or technicians, and a unified and relatively easy-to-judge standard is set for the labeled scores. Therefore, the scoring model trained with such a training data set can automatically give accurate scores for each key area.

[0180] Furthermore, the image quality assessment unit also uses the trained YOLO model to segment foreign objects from the image, and uses the OCR model to identify the text labels used to distinguish the left and right legs, and scores them based on whether they are segmented and identified. Therefore, the final calculated total score comprehensively considers the key bone areas in the image, the bending of the lateral leg bones and the mutual occlusion of the bones, whether there are foreign body influences, and whether there are text labels, and has good practical application value.

[0181] Furthermore, the image quality assessment unit also uses the trained deep learning model to automatically score the contrast and field of view of the image. The final calculated total score also comprehensively considers the contrast and field of view scores, so it is more comprehensive and can also further provide doctors or technicians with a reference for adjusting shooting parameters.

[0182] Furthermore, the image quality assessment control device also includes a prompt information generation unit and a display unit, which can generate and display corresponding prompt information according to multiple scores, total scores, calculated angles, whether foreign matter is recognized and text labels, so as to provide intuitive and comprehensive information to radiologists or technicians who take images, and help doctors or technicians better take the next image.

[0183] In the embodiment, the target detection segmentation model and the overlapping area segmentation model are both YOLO models, which have good target detection performance, are simple and easy to use, and have high development and training efficiency, which is conducive to the rapid deployment of the solution in actual clinical applications.

[0184] The above embodiments are only used to illustrate the specific implementation of the present invention, and the present invention is not limited to the description scope of the above embodiments. Those skilled in the art should understand that the present invention is not limited by the above embodiments. The above embodiments and descriptions are only to illustrate the principles of the present invention. Without departing from the spirit and scope of the present invention, the present invention may have various changes and improvements, and these changes and improvements fall within the scope of the present invention to be protected. The scope of protection of the present invention is defined by the attached claims and their equivalents.

Claims

1. A knee joint medical image quality assessment control system, characterized in that: include: The image quality evaluation and control device is used to evaluate and control the image quality of knee joint X-ray images. Wherein, the image quality assessment control device comprises: an image quality evaluation unit, configured to evaluate the image quality of the knee joint X-ray image to obtain an evaluation result; and an image quality determination unit, for determining whether the image quality is qualified according to the evaluation result and a predetermined image quality determination rule, The image quality assessment unit comprises: an object segmentation unit, configured to perform object detection and instance segmentation of a plurality of predetermined objects from the knee joint X-ray image by using an object detection and segmentation model, and obtain a segmentation result, wherein the predetermined objects include a plurality of key areas, and the segmentation result includes an image portion of the key areas; and an image scoring unit, at least used to score the segmented image portion of the key area using a key area scoring model, wherein the evaluation result includes the score, The target detection segmentation model and the key area scoring model are both trained deep learning network models.

2. The knee joint medical image quality assessment control system according to claim 1, Features: in, The knee joint X-ray image includes an anteroposterior image and a corresponding lateral image of the knee joint. The plurality of key areas are five, including the anteroposterior patella area, the anteroposterior fibular head area, the lateral patella area, the lateral fibular head area, and the lateral medial and lateral condyle areas. The image quality assessment unit also includes: The overlapping region segmentation unit is used to perform instance segmentation on the medial and lateral condyle regions of the lateral position by using the trained overlapping region segmentation model, and to segment the overlapping region and non-overlapping region of the medial epicondyle and the lateral epicondyle. The image scoring unit comprises: an anteroposterior and lateral position identification and determination unit, used for identifying and determining the anteroposterior image and the lateral image from the knee joint X-ray image; and A first key area scoring part is used to score the image parts of the anteroposterior patella area, the anteroposterior fibular head area, the lateral patella area, and the lateral fibular head area by using the key area scoring model; A second key area scoring part is used to score the image part of the lateral medial and lateral condyle region according to the ratio between the overlapping area and the non-overlapping area and a predetermined key area scoring rule; an AP image score calculation section, which calculates a total score of the AP image based on at least the score of the AP patella region and the score of the AP fibula head region; and The lateral image score calculation part calculates the total score of the lateral image based on at least the score of the lateral patella area, the score of the lateral fibular head area, and the score of the medial and lateral condyle areas. The image quality determination unit makes a determination based on the total score and the image quality determination rule.

3. The knee joint medical image quality assessment control system according to claim 2, characterized in that: in, The predetermined target also includes three key points, namely, the midpoint of the lateral upper femoral medullary cavity, the midpoint of the lateral lower femoral medullary cavity, and the midpoint of the lateral lower tibia medullary cavity. The segmentation result also includes the image part of the key points. The image quality assessment unit further includes: The bending angle calculation unit is used to calculate the angle between the long axis of the femur and the long axis of the tibia according to the three key points. The image scoring unit further comprises a bending angle scoring part, which scores the bending angle according to the included angle and a predetermined angle scoring rule; The lateral image score calculation section calculates a total score of the lateral image based on at least the score of the lateral patella region, the score of the lateral fibular head region, the score of the medial and lateral condyle regions, and the score of the bending angle.

4. The knee joint medical image quality assessment control system according to claim 3, Features: in, The predetermined target also includes foreign matter, and the segmentation result also includes an image portion of the foreign matter. The image quality assessment unit also includes: a mark detection unit, used to recognize a predetermined text mark in the knee joint X-ray image through an optical character recognition model, and obtain its position information when the text mark is recognized, The image scoring unit also includes: A foreign body scoring part, used for scoring the foreign bodies in the knee joint frontal image and the knee joint lateral image respectively according to whether the image part of the foreign body is segmented out; a marking scoring section, for scoring the text markings of the knee joint anteroposterior image and the knee joint lateral image respectively according to whether the text marking is recognized and the position information when the text marking is recognized; and The basic quality scoring part is used to score the contrast and visual field of the knee joint anteroposterior image and the knee joint lateral image respectively by using a trained image basic quality scoring model. The anteroposterior image score calculation part calculates the total score of the anteroposterior image according to the score of the anteroposterior patella area, the score of the anteroposterior fibula head area, the foreign body score corresponding to the anteroposterior image, the score of the mark, the score of the contrast and the score of the visual field. The lateral image score calculation part calculates the total score of the lateral image based on the score of the lateral patella area, the score of the lateral fibular head area, the score of the medial and lateral condyle areas, the score of the bending angle, and the foreign body score, the mark score, the contrast score and the field of view score corresponding to the lateral image.

5. The knee joint medical image quality assessment control system according to claim 4, Features: The training data set of the key area scoring model includes a plurality of image portions of the key areas and their annotated scores, and the annotated scores are set according to a predetermined scoring standard. The scoring criteria for the anteroposterior fibular head area are: in the corresponding image portion, if the patella is centered, it is scored as a qualified score; if the patella is not centered and does not exceed the edge of the femur, it is scored as a basically qualified score; if the patella is not centered and exceeds the edge of the femur, it is scored as an unqualified score. The scoring standard for the anteroposterior fibular head area is: in the corresponding image part, if the blocked area of ​​the fibular head is less than 1 / 3, it is scored as a qualified score; if the blocked area of ​​the fibular head is greater than or equal to 1 / 3 and less than 2 / 3, it is scored as a basically qualified score; if the blocked area of ​​the fibular head is greater than or equal to 2 / 3, it is scored as an unqualified score. The scoring criteria for the lateral patella area are as follows: in the corresponding image portion, if the patella and the posterior femur do not intersect and there is a gap between the two, it is scored as a qualified score; if the patella and the posterior femur intersect but the gap is visible, it is scored as a basically qualified score; if the patella and the posterior femur intersect and the gap is not visible, it is scored as an unqualified score. The scoring criteria for the lateral tibiofibular head region are as follows: in the corresponding image portion, if the occluded area of ​​the fibular head is less than 1 / 3, it is scored as a qualified score; if the occluded area of ​​the fibular head is greater than or equal to 1 / 3 and less than 2 / 3, it is scored as a basically qualified score; if the occluded area of ​​the fibular head is greater than or equal to 2 / 3, it is scored as an unqualified score. The key area scoring rule is: calculate the proportion of the non-overlapping area, proportion = area of ​​the non-overlapping area / (area of ​​the overlapping area + area of ​​the non-overlapping area); when the proportion is less than 10%, it is scored as a qualified score; when the proportion is greater than or equal to 10% and less than 20%, it is scored as a basically qualified score; when the proportion is greater than or equal to 20%, it is scored as an unqualified score.

6. The knee joint medical image quality assessment control system according to claim 5, Features: The scoring rule for the angle is as follows: when the angle is less than 90°, it is scored as the unqualified score; when the angle is greater than or equal to 90° and less than 120°, it is scored as the basically qualified score; when the angle is greater than or equal to 120° and less than 135°, it is scored as the qualified score; when the angle is greater than or equal to 135° and less than 150°, it is scored as the basically qualified score; when the angle is greater than or equal to 150°, it is scored as the unqualified score. The scoring rule of the contrast is: in the frontal image or the lateral image, if the bone and tissue cannot be distinguished at all, it is scored as the unqualified score; if the bone anatomical structure can be distinguished but the trabecular structure cannot be distinguished, it is scored as the basically qualified score; if the trabecular structure can be distinguished, it is scored as the qualified score. The scoring rule for the field of view is: if the knee joint structure is not completely included in the frontal image or the lateral image, it is scored as the unqualified score; if the knee joint structure is completely included, but the lateral width of the corresponding image is greater than or equal to its longitudinal width, it is scored as the basically qualified score; if the knee joint structure is completely included, but the key part of the field of view is not in the center of the corresponding image, it is scored as the basically qualified score; if the knee joint structure is completely included, and the lateral width of the corresponding image is less than the longitudinal width, and the key part of the field of view is in the center of the corresponding image, it is scored as the qualified score.

7. The knee joint medical image quality assessment control system according to claim 6, Features: The passing score is 2 points, the basic passing score is 1 point, and the unqualified score is 0 point. The foreign matter scoring part is scored as 2 points when the image part of the foreign matter is not segmented, 1 point when one image part of the foreign matter is segmented, and 0 point when two or more image parts of the foreign matter are segmented. The identification scoring part is scored as 2 points when the text identification is recognized and the text identification is outside the range of the knee joint, 1 point when the text identification is recognized but within the range of the knee joint, and 0 point when the text identification is not recognized. The image quality determination rule is: Determine the frontal image and the lateral image respectively. When the corresponding contrast or the corresponding field of view is rated as the unqualified score, it is determined that the image quality is unqualified and needs to be re-photographed; When the contrast and the field of view are both rated as the basic passing score or the passing score, further judgment is made based on the corresponding total score. When the total score is less than 60% of the full score, it is judged that the image quality is unqualified and needs to be re-shot; when the total score is between 60% and 80% of the full score, it is judged that the image quality is qualified and no re-shooting is required; when the total score is between 80% and 90% of the full score, it is judged that the image quality is good and no re-shooting is required; when the total score is greater than or equal to 90% of the full score, it is judged that the image quality is excellent and no re-shooting is required.

8. The knee joint medical image quality assessment control system according to claim 4, Features: Wherein, the image quality assessment control device further includes: a prompt information generating unit, which generates corresponding prompt information according to the scores of the five key areas, the scores of the angles, the scores of the foreign matter, and the scores of the text marks respectively when the image quality judging unit judges that the image is unqualified; and The input display unit is used to display the prompt information.

9. The knee joint medical image quality assessment control system according to claim 3, characterized in that: Also includes: A knee joint imaging auxiliary device is used to assist in positioning the patient's knee joint when taking the knee joint X-ray image. Wherein, the knee joint imaging auxiliary device comprises: Two lateral positioning plates, whose bending angles are adjustable and which are detachably arranged in the knee joint imaging auxiliary device, are used to assist in positioning the lateral leg of the patient; A driver, used for driving the side positioning plate to change its bending angle; and The regulating airbag is arranged between one end of the two lateral positioning plates and is used to raise the tibia and fibula of the patient's lateral leg to a predetermined angle. The image quality assessment control device also includes: The auxiliary device control unit controls the knee joint imaging auxiliary device according to the evaluation result to assist positioning for the next shooting when the image quality determination unit determines that the image quality is unqualified. The auxiliary device control unit comprises: An angle difference calculation unit, used for calculating the angle difference between the included angle and a predetermined standard angle; an adjustment amount calculation unit, used to calculate an adjustment amount for controlling the driver according to the angle difference, the parameters of the driver, and the size data of the side positioning plate; and The driver control unit controls the driver according to the adjustment amount, thereby adjusting the bending angle of the side positioning plate.

10. A knee joint medical image quality assessment and control method for assessing and controlling the image quality of knee joint X-ray images, characterized in that: The following steps are involved: An evaluation step of evaluating the image quality of the knee joint X-ray image to obtain an evaluation result; as well as a step of judging whether the image quality is qualified according to the evaluation result and a predetermined image quality judgment rule, Wherein, in the evaluation step, target detection and instance segmentation of multiple predetermined targets are performed from the knee joint X-ray image by using a target detection segmentation model to obtain a segmentation result, wherein the predetermined target includes multiple key areas, the segmentation result includes image parts of the key areas, and the image parts of the segmented key areas are scored at least by using a key area scoring model, and the evaluation result includes the score, The target detection segmentation model and the key area scoring model are both trained deep learning network models.

Images