Method for positioning anatomical landmark points of human joints

Abstract

The application discloses a positioning method of human joint anatomical landmark points, and comprises the following steps: collecting human DICOM image data sets of different objects, reconstructing sequence files in the DICOM image data sets to generate three-dimensional CT images; generating two-dimensional DRR images in the orthosteric direction of the three-dimensional CT images, constructing joint regions and anatomical landmark point identification models of the two-dimensional DRR images, and identifying the positions of the anatomical landmark points on the orthosteric DRR images; generating axial tomographic images on the three-dimensional CT images according to the coordinate information of the landmark points on the orthosteric DRR images, constructing joint regions and anatomical landmark point identification models of the two-dimensional axial tomographic images, and identifying the positions of the anatomical landmark points on the axial tomographic images again; positioning the positions of the anatomical landmark points through the outputs of the two models, and obtaining complete three-dimensional coordinates of the joint anatomical landmark points. The application improves the accuracy of joint anatomical landmark point identification and has high identification efficiency.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of medical image processing, in particular to a positioning method of human joint anatomical landmark points. BACKGROUND

[0002] Joint surgery is a common method for treating joint diseases, which requires accurate positioning of anatomical landmark points to ensure the accuracy and safety of the surgery. The traditional method of positioning anatomical landmark points mainly relies on the experience and sense of touch of doctors, which has the problems of inaccurate operation, being greatly affected by the experience level of the operator, and being prone to risks such as surgical failure and complications.

[0003] With the development of computer technology, image processing technology based on artificial intelligence (AI) has been widely applied in the medical field, such as surgical robots, surgical navigation packages, etc. Therefore, developing a positioning method of human joint anatomical landmark points based on AI image processing can greatly improve the accuracy and efficiency of joint surgery and reduce the risk of surgery. Currently, many clinical trials directly use three-dimensional medical images to find landmark points, and the identification of three-dimensional landmark points is very difficult. SUMMARY

[0004] The technical problem to be solved by the present application is to overcome the shortcomings of the prior art and provide a positioning method of human joint anatomical landmark points.

[0005] The technical solution adopted by the present application to solve the technical problem is: a positioning method of human joint anatomical landmark points, used for positioning joint regions and anatomical landmark points in a computed tomography (CT) image of a human body, comprising the following steps:

[0006] S1, obtaining DICOM image data sets of different objects, and reconstructing sequence files in the DICOM image data sets to generate three-dimensional CT images;

[0007] S2, generating a two-dimensional DRR image in the orthogonal direction of the three-dimensional CT image of step S1, labeling corresponding regions and anatomical landmark points on the DRR image as training labels, and obtaining an orthogonal target region detection and anatomical landmark point recognition model based on the training labels using a top-down algorithm;

[0008] S3, based on the orthogonal target detection and anatomical landmark point recognition model obtained in step S2, performing target detection and key point detection on the two-dimensional DRR image in the orthogonal direction generated by the three-dimensional CT image, and obtaining orthogonal coordinate information of the anatomical landmark points;

[0009] S4, based on the positive coordinate information obtained in step S3, generating a two-dimensional axial tomographic image corresponding to the marker point position through three-dimensional CT data, labeling the corresponding region and the marker point position on the two-dimensional axial tomographic image as a training label, and obtaining a target detection and anatomical key point recognition model based on the training label using a top-down algorithm;

[0010] S5, based on the target detection and anatomical key point recognition model obtained in step S4, performing target detection and key point detection on the obtained two-dimensional axial tomographic image to obtain coordinate information of the axial position of the anatomical marker point;

[0011] S6, based on the positive coordinate information of the anatomical marker point obtained in step S3 and the coordinate information of the axial position of the anatomical marker point obtained in step S5, merging into complete three-dimensional coordinates of each anatomical marker point.

[0012] In the above scheme, the sequence file in the DICOM in step S1 is reconstructed to generate a three-dimensional CT image according to the image position, image position, pixel spacing, and slice thickness information included in the header file.

[0013] In the above scheme, the specific operation steps of step S2 are: first, generating a two-dimensional DRR image in the positive direction of the human three-dimensional CT image, the generated DRR image contains the human joint region and other interference regions, manually labeling the human joint region of interest as the training label of the target detection model, then manually labeling the anatomical marker point in the region of interest as the training label of the key point detection model, respectively constructing the target region detection and marker point detection network, and training and saving the model.

[0014] In the above scheme, the top-down algorithm has an RTMDET network and an HRNET network to construct a positioning network of the human joint region, i.e., a joint region positioning network and a marker point positioning network.

[0015] In the above scheme, the specific operation steps of step S4 are: using the obtained marker point coordinate information in the positive direction to generate a two-dimensional axial tomographic image corresponding to the marker point position through three-dimensional CT data, manually labeling the region of interest on the two-dimensional axial tomographic image as the training label of the target detection model; then manually labeling the marker point in the region of interest as the training label of the key point detection model, respectively constructing the target region detection and anatomical key point detection network, and training and saving the model.

[0016] In the above scheme, the top-down algorithm has an RTMDET network and an HRNET network to construct a positioning network of the human joint region, i.e., a joint region positioning network and a marker point positioning network.

[0017] The beneficial effects of the present application are:

[0018] The three-dimensional CT data generation DRR image algorithm in the positioning method provided by the application guarantees the rapidity and effectiveness of the algorithm;

[0019] The top-down algorithm composed of RTMDET and HRNET in the positioning method provided by the application guarantees the accuracy of the algorithm;

[0020] The positioning method provided by the application realizes the detection and positioning of point positions on a three-dimensional target by using two-dimensional image key point detection in different directions, reduces the positioning difficulty of doctors on joint anatomical landmark points, improves the efficiency, and saves the cost. BRIEF DESCRIPTION OF DRAWINGS

[0021] The application will be further described below in combination with the drawings and embodiments.

[0022] Figure 1 is a flowchart of a human joint anatomical landmark positioning method of the application.

[0023] Figure 2 is a principle diagram of a human joint anatomical landmark positioning method of the application. DETAILED DESCRIPTION

[0024] The application will be further described below in combination with the drawings and preferred embodiments. These drawings are all simplified schematic diagrams, and only schematically show the basic structure of the application, and therefore only show the configurations related to the application.

[0025] As shown in Figure 1 A human joint anatomical landmark positioning method is used for positioning anatomical landmark points in a human computer tomography image, and includes the following steps:

[0026] DICOM image data sets of different objects are acquired, and then three-dimensional CT images are generated according to the image orientation (imageOrientation), image position (imagePosition), pixel spacing (pixelSpacing), slice thickness (sliceThickness) and other information in the header file.

[0027] Two-dimensional DRR images are generated on the three-dimensional CT images in the orthotopic direction, and corresponding joint regions and landmark points are labeled on the two-dimensional DRR images as training labels, and a target detection and key point recognition model is constructed through the training labels, and the specific operation steps are as follows:

[0028] First, generate a two-dimensional DRR image in the frontal direction from the three-dimensional CT image of the human body: a digital reconstructed radiograph (drr_main) of the joint region; manually label the joint region of interest and the anatomical landmark points as training labels: frontal labels (region_axial, point_axial); construct a joint positioning network (focus_main_region) in the frontal direction and an anatomical landmark point positioning network (focus_axial_point) using the training labels; and save the model.

[0029] Use the obtained detection and key point model to perform target detection and key point detection on the two-dimensional DRR image generated in the frontal direction from the three-dimensional CT image, and obtain the coordinate information of the landmark point in the frontal direction. The specific operation steps are as follows: before using the top-down algorithm structure to train the model, the data set needs to be preprocessed and data augmented. For the data preprocessing of the generated two-dimensional DRR image, mainly convert the two-dimensional DRR image containing only gray scale information into a pseudo-color image with color information to adapt to the training and inference of the model; after pseudo-color processing of the two-dimensional DRR image and the corresponding label (label), the two-dimensional DRR image and the corresponding label (label) need to be rotated, flipped, cropped, scaled, and other picture enhancement operations, after enhancement, the picture is used mixup operation, further enhance the data, mixup is essentially randomly linearly splicing 4-8 pictures, increase the generalization ability and robustness of the model; use the frontal DRR reconstruction image (drr_main) and the joint region label and the anatomical landmark point label (region_axial, point_axial) in the frontal direction to construct the positioning network of the human body in the frontal direction: joint region positioning (focus_main_region), anatomical landmark point positioning (focus_main_point), train and save the model; use the positioning model to position the frontal label to obtain the two-dimensional coordinates (z-position) of the joint region and the anatomical landmark point.

[0030] The axial coordinate information of the landmark points is used in combination with the three-dimensional CT data to generate a two-dimensional axial tomographic image corresponding to the landmark point position, and a target detection and key point recognition model based on the two-dimensional axial tomographic image is constructed. The specific operation steps are as follows: according to the obtained coordinate of the anatomical landmark point in the frontal direction, an axial direction tomographic two-dimensional DRR image (slice_axial) corresponding to the point coordinate is generated in the three-dimensional CT image. The joint region and the anatomical landmark point are manually labeled on the axial direction tomographic two-dimensional DRR image (slice_axial) as training labels (region_axial, point_axial); the axial tomographic two-dimensional DRR image (slice_axial) and the axial region label and anatomical landmark point label (region_axial, point_axial) are used to construct the axial tomographic positioning network: tomographic region positioning (focus_axial_region), tomographic anatomical landmark point positioning (focus_axial_point), and the model is trained and saved.

[0031] The obtained two-dimensional axial tomographic image is used for target detection and key point detection to obtain the coordinate information of the landmark point in the axial direction. The specific operation steps are as follows: the two-dimensional DRR image generated in the axial direction of the three-dimensional CT image is positioned using the tomographic region positioning (focus_axial_region) and the tomographic anatomical landmark point positioning (focus_axial_point), and the coordinates (x-position, y-position) of the tomographic joint region and the anatomical landmark point are obtained.

[0032] The sequence file of the DICOM image data set is reconstructed into three-dimensional CT volume data, and a two-dimensional digital reconstructed radiographic image (drr_main) is generated in the frontal direction. The joint region positioning (focus_main_region) model and the anatomical landmark point positioning (focus_main_point) model are used to detect the human joint region and the anatomical landmark points in the region on the frontal digital reconstructed radiographic image (drr_main).

[0033] The obtained landmark point position information in the frontal direction and the axial direction is combined into the complete three-dimensional coordinates of each anatomical landmark point. The specific operation steps are as follows: the landmark point information in the frontal direction can determine the Z coordinate of the anatomical landmark point in the three-dimensional coordinates, and the X and Y coordinates of the landmark point are determined through the landmark point information in the tomographic image, so as to determine the complete three-dimensional coordinates of the anatomical landmark point in the three-dimensional CT data.

[0034] Among them, the top-down algorithm structure module is a target detection method based on prior knowledge and context information, which first uses some prior knowledge or model to identify the approximate position or some feature information in the image, and then further detects and identifies the target on this basis; therefore, RTMDET is first used for joint region detection, and then HRNET is used for local feature recognition.

[0035] RTMDET is a high-performance, low-latency single-stage target detector. The overall structure of the RTMDET model is almost the same as YOLOX, but the basic block uses a new CNN module structure, and the data enhancement is also changed. RTMDET introduces a new Cashe scheme in addition to the traditional mixup and random resize methods during data enhancement. By selecting a random number of pictures to be placed in the Cashe cache during reading at each training epoch, the pictures do not need to be read repeatedly, greatly reducing the consumption of memory. Since the model is based on YOLOX evolution, RTMDET is also a one-stage model. The specific pipeline is the same as YOLO. The prepared two-dimensional reconstructed radiographic image (DRR) is input into the network after data enhancement for training. The picture will be predicted and framed according to different sizes. Each predicted object will have multiple probability frames. Non-maximum suppression (NMS), classification loss function (focal loss) and regression loss function (Smooth L1 Loss) algorithms help us select the most suitable bounding box. Finally, the detection head outputs two branches, rtm cls class prediction branch and rtm reg bounding box regression branch, which display the information of each bounding box.

[0036] After the target area detection is completed, the HRNET model is used for the region of the output bounding box. HRNET is also a deep neural network structure with high-resolution feature representation and multi-resolution feature interaction. Its core method is to save high-resolution feature representation while performing multi-resolution feature interaction throughout the model process. This design enables HRNET to maintain high-resolution features while having stronger receptive fields and better semantic information. The neck design of HRNET is different from other networks: the neck of HRNET is composed of multiple stages, each stage containing a bottleneck and multiple basic blocks. In the neck of HRNET, the output of each stage is multiple resolution feature maps, which are specially processed in subsequent operations, and finally output a high-resolution feature map. Through the fusion of different high-resolution feature maps, the subtle points in the input two-dimensional reconstruction radiography (DRR) can also be identified with a certain accuracy, and the coordinate information of the anatomical landmark points is output.

[0037] When identifying key points, a perspective view is first generated because the coordinate information in the perspective view is needed when generating tomographic images. The generated tomographic images are actually tomographic images of different joint parts in the perspective view. And in order to ensure the diversity and clarity of the orthotopic perspective view, the perspective view generated by DICOM within the specified threshold can be used. Moreover, due to the two-stage algorithm (RTMDET and HRNET constitute a top-down algorithm), the generation of axial tomographic images is extremely dependent on the identification and annotation accuracy of the perspective view; and in order to overcome the problem of inconsistent definition of anatomical landmarks in medicine, the relevant landmark points involved in the annotation and identification are covered.

[0038] Due to the lack of medical image landmark recognition network on the market, we also encountered difficulties when choosing a network model. In order to ensure the speed and accuracy of recognition, HRNET is selected for point recognition, and RTMDET is used for box recognition.

[0039] DICOM (Digital Imaging and Communications in Medicine) is an international standard for storing medical images and related information. This standard is widely used in radiology, cardiovascular imaging, and diagnostic equipment such as X-ray, CT, MRI, and ultrasound. The DICOM file format contains patient information and other image-related information. Diagnostic equipment performs cross-sectional scanning around a certain part of the body, resulting in multiple layers of images. These images are stacked in the z-axis to form a three-dimensional image, with each layer represented by a DICOM file. A DICOM file typically consists of a file header and a data set, with the header containing information identifying the data set. Each DICOM file includes a file header.

[0040] DRR (Digitally Reconstructed Radiograph) is a digital reconstruction of a radiograph, which uses simulated light rays to scan the human body in three dimensions and reconstruct a grayscale image representing the internal organs.

[0041] As shown in Figure 2 The principle of the human joint anatomical landmark positioning method is as follows: generate a main two-dimensional DRR image in the frontal direction from the reconstructed three-dimensional human CT image, which is used for joint region and anatomical landmark positioning. Generate a tomographic image of the anatomical landmark in the axial direction from the original body data by positioning the joint region and the position of the anatomical landmark, which is used for axial tomographic joint region and landmark positioning. Generate the complete three-dimensional coordinates (X, Y, Z) of the anatomical landmark by obtaining the Z coordinate from the frontal DRR image and the (X, Y) coordinate from the axial tomographic image, and complete the identification and positioning of the anatomical landmark. This human joint anatomical landmark positioning method determines the three-dimensional coordinates by using two different two-dimensional images, reducing the difficulty of positioning the joint anatomical landmark for doctors, improving efficiency, and saving costs.

[0042] The above examples are only for illustrating the technical concept and characteristics of the present application, and the purpose is to enable those skilled in the art to understand the content of the present application and implement it, and cannot limit the protection scope of the present application. Any equivalent changes or modifications made in accordance with the spirit and essence of the present application should be covered within the protection scope of the present application.

Claims

1. A method for locating anatomical landmark points of a human joint, for locating a joint region and anatomical landmark points in a computed tomography image of a human body, characterized in that, It comprises the following steps: S1, obtaining DICOM image data sets of different objects, and reconstructing sequence files in the DICOM image data sets to generate three-dimensional CT images; S2, generating a two-dimensional DRR image in the orthosteric direction of the three-dimensional CT image of step S1, labeling the corresponding region and the anatomical landmark point on the DRR image as a training label, and obtaining an orthosteric target region detection and anatomical landmark point recognition model based on the training label using a top-down algorithm; S3, based on the orthosteric target detection and anatomical landmark point recognition model obtained in step S2, performing target detection and key point detection on the two-dimensional DRR image generated by the three-dimensional CT image in the orthosteric direction, and obtaining orthosteric coordinate information of the anatomical landmark point; S4, based on the orthosteric coordinate information obtained in step S3, generating a two-dimensional axial tomographic image of the corresponding landmark point through three-dimensional CT data, labeling the corresponding region and landmark point on the two-dimensional axial tomographic image as a training label, and obtaining a target detection and anatomical key point recognition model based on the training label using a top-down algorithm; S5, based on the target detection and anatomical key point recognition model obtained in step S4, performing target detection and key point detection on the obtained two-dimensional axial tomographic image to obtain axial coordinate information of the anatomical landmark point; S6, based on the orthosteric coordinate information of the anatomical landmark point obtained in step S3 and the axial coordinate information of the anatomical landmark point obtained in step S5, merging into complete three-dimensional coordinates of each anatomical landmark point.

2. The method of locating anatomical landmarks of a human joint according to claim 1, wherein: The sequence file in the DICOM in step S1 is reconstructed to generate three-dimensional CT images according to the image orientation, image position, pixel spacing and slice thickness information included in the header file.

3. The method of locating anatomical landmarks of a human joint according to claim 1, wherein: The specific operation steps of step S2 are: first, generating a two-dimensional DRR image in the orthosteric direction of the human three-dimensional CT image, the generated DRR image contains the joint region of the human body and other interference regions, and the interested joint region of the human body is manually labeled as a training label of a target detection model, and then the anatomical landmark points in the interested region are manually labeled as training labels of a key point detection model, and a target region detection and landmark point detection network is constructed respectively, and the model is trained and saved.

4. The method of locating anatomical landmarks of a human joint according to claim 3, wherein: The top-down algorithm has an RTMDET network and an HRNET network to construct a positioning network of the joint region of the human body, i.e., a joint region positioning network and a landmark point positioning network.

5. The method of locating anatomical landmarks of a human joint of claim 1, wherein: The specific operation steps of step S4 are: using the landmark point coordinate information in the orthosteric direction, generating a two-dimensional axial tomographic image of the corresponding landmark point through three-dimensional CT data, manually labeling the interested region on the two-dimensional axial tomographic image as a training label of a target detection model; and then manually labeling the landmark points in the interested region as training labels of a key point detection model, respectively constructing a target region detection and anatomical key point detection network, and training and saving the model.

6. The method of locating anatomical landmarks of a human joint according to claim 5, wherein: The top-down algorithm has an RTMDET network and an HRNET network to construct a positioning network of the joint region of the human body, i.e., a joint region positioning network and a landmark point positioning network.

Images