A hip joint key point detection system based on implicit and explicit structural constraints
Through the hip joint key point detection system based on implicit and explicit structural constraints, the local and global networks combine point multiplication and morphological perception modules, the time-consuming time-consuming of manual labeling and disease detection in adults in the diagnosis of hip joint diseases in the prior art is solved. The unsatisfactory problem achieves more efficient and accurate detection of hip key points.
Patent Information
- Application Number
- CN202311511216.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-11-14
- Publication Date
- 2024-12-06
- Estimated Expiration
- 2043-11-14
AI Technical Summary
In the diagnosis of hip disease, manual labeling of key points of hip joints is time-consuming and labor-intensive. The existing deep learning methods are mainly aimed at children's congenital hip dysplasia, which lacks femoral head necrosis and hip in adults. The practicality of joint impact syndrome and hip osteoarthritis, and the prediction results of key points are not ideal.
Adopt a hip joint key point detection system based on implicit and explicit structural constraints, and obtain local features through local networks, capture overall structural constraints in the global network, adjust non-strongly identified key points, and obtain explicit morphology by morphological perception module. Relevance to improve detection accuracy and reliability.
It significantly improves the performance of hip key points detection, is better than the classic adult detection network, improves the detection accuracy and efficiency of indicators in diagnosis, is practical for common hip diseases, and enhances the reliability of detection results.
Smart Images

Figure CN117765228B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of joint key point detection, and in particular to a hip joint key point detection system based on implicit and explicit structural constraints. Background Art
[0002] The hip joint is the largest and most stable joint in the human body, and is also the most core load-bearing joint in the human body. External forces such as exercise make the hip joint vulnerable to injury, which in turn causes various hip joint diseases. At present, the diagnosis of hip joint diseases in clinical practice is mainly through professional doctors calibrating specific key points in the patient's pelvic X-ray film, and then calculating relevant indicators. The key points commonly used and widely accepted in clinical practice are as follows: Figure 1 shown.
[0003] Figure 1 Several key points of hip X-ray images are shown, including the outermost point of the acetabulum OPAC, the inner edge of the acetabulum subchondral sclerosis zone IPSS, the lowermost point of the teardrop IPTE and the center of the femoral head FHCE. The key points on the left side of the human body are marked with L, and the key points on the right side of the human body are marked with R. However, this manual labeling method is time-consuming and labor-intensive, and is easily affected by the doctor's professional knowledge and clinical experience. Therefore, accurately identifying and locating the key points of hip anatomical joints can help doctors save a lot of time on repeated calibration work.
[0004] However, in order to overcome the above difficulties, a variety of deep learning-based hip joint key point detection methods or models have been proposed in the prior art, which are mainly divided into two categories: two-stage detection methods and single-stage detection methods. However, these two existing methods are basically only used for children's congenital developmental dysplasia of the hip (DDH), but lack practicality for very common hip joint diseases such as osteonecrosis of the femoral head (ONFH), femoroacetabular impingement (FAI), and hip osteoarthritis in adults in clinical practice, and there is a problem that the prediction results of some key points are not ideal. Summary of the invention
[0005] The purpose of the present invention is to overcome the defects of the prior art and provide a hip joint key point detection system based on implicit and explicit structural constraints.
[0006] The present invention provides a hip joint key point detection system based on implicit and explicit structural constraints, and its technical solution is as follows:
[0007] A hip joint key point detection system based on implicit and explicit structural constraints, comprising:
[0008] An input module, used for inputting pelvic images;
[0009] A local network module is used to obtain local features of key points of the hip joint in the pelvic image, realize preliminary positioning of key points of the hip joint, and accurately locate key points with strong local structure identification to obtain a local heat map;
[0010] A global network module, used for obtaining the implicit overall structural constraint relationship of the hip joint in the pelvic image to form a global heat map;
[0011] A dot multiplication unit, used to perform a dot multiplication operation between the local heat map and the global heat map, and adjust the detection accuracy of some non-strong identification key points;
[0012] The morphological perception module is used to obtain explicit morphological associations between key points and adjust key points with large errors;
[0013] in,
[0014] The input module communicates with the local network module and the global network module respectively, the point product unit communicates with the local network module, the global network module and the morphological perception module respectively, and the local network module communicates with the global network module.
[0015] Compared with the prior art, the technical solution proposed in the present invention, firstly, can obtain the local features of the key point positions in the bone image through the local network module, detect all the key points, and realize the accurate positioning of the key points with strong identification local features. Secondly, the global network module can capture the global features of the hip joint as implicit structural constraints. Combined with the point multiplication unit, it can reduce or eliminate the ambiguity caused by feature similarity and adjust the positioning effect of non-strong identification key points. In addition, the explicit morphological association between key points is obtained through the morphological perception module, thereby improving the reliability of the prediction results, so that the technical solution of the present invention is not only practical for the detection of hip joint key points of congenital hip dysplasia in children, but also for very common hip diseases such as femoral head necrosis, hip impingement syndrome, and hip osteoarthritis in adults in clinical practice.
[0016] The technical solution of the present invention can significantly improve the key point detection performance. It is not only superior to the classic and the latest adult hip joint key point detection network, but also can achieve better detection results within a low error allowable range. It has very important practical significance for improving the detection accuracy and diagnostic efficiency of indicators in clinical diagnosis, as well as for the promotion of related downstream tasks.
[0017] Preferably, the local network module includes:
[0018] An encoder, used for acquiring local features of hip joints at different scales in the pelvic image;
[0019] A decoder, used for fusing the local features of each scale obtained by the encoder with the features after upsampling at the previous stage, realizing the preliminary positioning of the key points of the hip joint, and positioning the key points corresponding to the strong local structure identifiers, and obtaining the local heat map;
[0020] in,
[0021] The encoder communicates with the input module, and the decoder communicates with the global network module and the morphology perception module respectively;
[0022] Then, while initially locating the key points, the accurate positioning of the key points with strong identification features is achieved.
[0023] Preferably, the encoder is a convolutional neural network, and the convolution of the convolutional neural network is a residual convolution; since each convolution layer in the convolutional neural network corresponds to a different receptive field size, using residual convolution to replace the standard convolution in the traditional U-Net can capture as much strong identification feature information in the area around the key points as possible, thereby appropriately increasing the receptive field while improving the accuracy of the model.
[0024] Preferably, the global network module comprises:
[0025] A dilated convolution unit, used to obtain contextual structural feature information, obtain a global heat map, and reduce the ambiguity problem of key point identification in the local network module;
[0026] in,
[0027] The dilated convolution unit communicates with the input module and the local network module respectively, and the dot product unit communicates with the dilated convolution unit;
[0028] Furthermore, by expanding the convolutional unit, the receptive field can be gradually increased, a wider range of contextual structural feature information can be captured, and it can be used to constrain the detection of key points. At the same time, combined with the point multiplication unit, the impact of false positive extreme values can be alleviated to obtain more accurate key point prediction results.
[0029] Preferably, the morphological perception module obtains the explicit morphological association between the key points through a stored hip joint morphological perception loss function, wherein the hip joint morphological perception loss function is expressed as follows:
[0030] L m =λL d+βL a
[0031] Where λ and β are loss parameters, L d Represents the distance loss, L a Represents the cosine similarity loss of angle vectors;
[0032] Using this morphologically aware loss function, the explicit structural constraint relationship between the hip joints can be fitted through two high-order statistics, and the positions of key points with large errors can be adjusted. BRIEF DESCRIPTION OF THE DRAWINGS
[0033] Figure 1 A pelvic image corresponding to the background technology of the present invention;
[0034] Figure 2 A schematic diagram of the structure of a hip joint key point detection system based on implicit and explicit structural constraints according to an embodiment of the present invention;
[0035] Figure 3 It is a structural diagram of a local network module according to an embodiment of the present invention;
[0036] Figure 4 A global network module structure diagram of an embodiment of the present invention;
[0037] Figure 5 This is a distance and angle model diagram corresponding to the morphology perception module in an embodiment of the present invention. DETAILED DESCRIPTION
[0038] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the accompanying drawings in the embodiments of the present invention.
[0039] An embodiment of the present invention provides a hip joint key point detection system based on implicit and explicit structural constraints, comprising:
[0040] Input module, used to input pelvic images and resize the images to 512×512;
[0041] The local network module is used to obtain the local features of the key points of the hip joint in the pelvic image, realize the preliminary positioning of the key points of the hip joint, and accurately locate the key points with strong local structure identification to obtain the local heat map;
[0042] The global network module is used to obtain the implicit overall structural constraint relationship of the hip joint in the pelvic image and form a global heat map;
[0043] The dot multiplication unit is used to perform the dot multiplication operation between the local heat map and the global heat map, and adjust the detection accuracy of some non-strong identification key points;
[0044] The morphological perception module is used to obtain explicit morphological associations between key points and adjust key points with large errors;
[0045] in,
[0046] The input module communicates with the local network module and the global network module respectively, the point product unit communicates with the morphological perception module, the local network module and the global network module respectively, and the local network module communicates with the global network module.
[0047] Although the hip joint structure and morphology of different individuals are affected by factors such as age, gender, and disease, and have certain temporal diversity and pathological deformities, there are still certain physiological consensus characteristics between different individuals. However, the facts show that the key points of hip joint anatomy have obvious regional characteristics and relatively stable global structural constraint characteristics, and there are explicit dependencies between key points. The regional characteristics of key points can realize the positioning of key points, among which key points with strong identification features can be accurately located. The stable structure of the hip joint bone architecture provides global implicit structural constraints, and the physiological structure association between key points can be used as an explicit constraint condition for key point detection.
[0048] Therefore, the embodiment of the present invention provides a new end-to-end feature hierarchical extraction hip joint key point detection system, which automatically detects the key points of the hip joint based on the local and global hip joint structure guidance network IIEC-Net.
[0049] In this embodiment, the local network module includes:
[0050] An encoder, used to obtain local features of hip joints at different scales in pelvic images;
[0051] A decoder is used to fuse the local features of each scale obtained by the encoder with the features after upsampling at the previous level, to achieve preliminary positioning of the key points of the hip joint, and to locate the key points corresponding to the strong local structure identifiers to obtain the local heat map;
[0052] in,
[0053] The encoder communicates with the input module, and the decoder communicates with the global network module and the morphology perception module respectively.
[0054] Through the analysis of pelvic X-ray images and pelvic structures, it can be found that the key points of the hip joint have strong local structural characteristics, and some key points have unique and significant local characteristics, which can be used as powerful key point identification marks. Figure 1 As shown in FIG. 1 , the key point OPAC is located at the outermost side of the upper edge of the acetabulum. This powerful feature representation can be fully learned by the network. Therefore, the local network module (such as Figure 3 As shown in the figure, while initially locating the key points, the accurate positioning of the key points with strong identification features is achieved.
[0055] As shown in Figure 3, the local network module used in this embodiment adopts a network structure similar to UNet. The encoder in this embodiment adopts a convolutional neural network, and the convolution of the convolutional neural network is a residual convolution.
[0056] The encoder captures multi-scale local features through step-by-step convolution and downsampling. Since each convolution layer in the convolutional neural network corresponds to a different receptive field size, in order to capture as much strong identifying feature information as possible in the area around the key point, the embodiment of the present invention uses residual convolution to replace the standard convolution in U-Net. Residual convolution adds cross-layer connections on the basis of conventional convolution operations, thereby reducing information loss, that is, improving the accuracy of the model while appropriately increasing the receptive field.
[0057] The pooling part performs downsampling. For the downsampling process, this embodiment uses the maximum pooling operation to obtain feature maps of different scales, while speeding up the training speed of the network and improving the operating efficiency of the network. Through a series of convolution and pooling operations, the encoding of the hip joint feature information in the input image is achieved.
[0058] like Figure 3 As shown, in the decoder of this embodiment, the size of the feature map is doubled and the number of channels is reduced to half of the input for upsampling, and skip connections are used to concatenate and fuse shallow detail discriminant features with deep high-level semantic discriminant features.
[0059] In this embodiment, the global network module includes:
[0060] Dilated convolutional units are used to obtain contextual structural feature information, obtain global heat maps, and reduce the ambiguity of key point identification in local network modules;
[0061] in,
[0062] The dilated convolution unit communicates with the input module and the local network module respectively, and the dot product unit communicates with the dilated convolution unit.
[0063] Although the local network has realized automatic recognition of key points, the edge contour of the hip joint is relatively wide, the key point targets are small and scattered, and the local area features of some key points have strong similarities. Therefore, only focusing on local area features is likely to lead to ambiguity and increase the probability of false detection.
[0064] Considering that the human hip joint has a stable topological structure in the global structure, it can be used as an implicit structural constraint between key points, such as Figure 4As shown, this embodiment provides a global network. It consists of 5 dilated 3×3 convolutions with dilation amounts of [1, 2, 5, 2, 1]. By gradually increasing the receptive field through the dilated convolution layer, the network captures a wider range of contextual structural feature information and uses it to constrain the detection of key points.
[0065] In this embodiment, the morphological perception module obtains the explicit morphological association between the key points through the stored hip joint morphological perception loss function, wherein the hip joint morphological perception loss function is expressed as follows:
[0066] L m =λL d +βL a
[0067] Where λ and β are loss parameters, L d Represents the distance loss, L a Represents the cosine similarity loss of angle vectors.
[0068] The morphological perception loss function (Hip Morphology-Aware loss) can enable the model to learn explicit morphological associations between key points by explicitly utilizing the relationship between key points and the individual information of each key point. Specifically, two high-order statistics (distance loss and cosine similarity loss) are used, which are expressed as the distance between two key points and the angle between three key points. Because obtaining unstable changing relationships will reduce the performance of the model, this embodiment does not use all possible key point set combinations, but uses high-confidence basic key points and low-confidence terminal key points as a distance combination.
[0069] Figure 5 Two subsets are shown: distance subset Pd = {(0,6), (2,6), (4,6), (1,7), (3,7), (5,7)} and angle subset Pa = {(0,2,4), (1,3,5)}. The distance uses the traditional Euclidean distance, and the cosine of the angle is also based on the traditional Euclidean space. For the two parameters of the above loss function, this embodiment takes λ as 3 and β as 1.
[0070] The system of this embodiment can significantly improve the key point detection performance, which is significantly better than the classic and latest adult hip joint key point detection network model, especially in achieving better detection results within the low error allowable range. At the same time, it has relatively few network parameters and reduces the computational complexity.
[0071] Although the embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.
Claims
1. A hip joint key point detection system based on implicit and explicit structural constraints, characterized by: include: An input module, used for inputting pelvic images; A local network module is used to obtain local features of key points of the hip joint in the pelvic image, realize preliminary positioning of key points of the hip joint, and accurately locate key points with strong local structure identification to obtain a local heat map; A global network module, used for obtaining the implicit overall structural constraint relationship of the hip joint in the pelvic image to form a global heat map; A dot multiplication unit, used to perform a dot multiplication operation between the local heat map and the global heat map, and adjust the detection accuracy of some non-strong identification key points; The morphological perception module is used to obtain explicit morphological associations between key points and adjust key points with large errors; in, The input module communicates with the local network module and the global network module respectively, the point product unit communicates with the local network module, the global network module and the morphological perception module respectively, and the local network module communicates with the global network module.
2. The hip joint key point detection system based on implicit and explicit structural constraints according to claim 1, characterized in that: The local network module includes: An encoder, used for acquiring local features of hip joints at different scales in the pelvic image; A decoder, used for fusing the local features of each scale obtained by the encoder with the features after upsampling at the previous stage, realizing the preliminary positioning of the key points of the hip joint, and positioning the key points corresponding to the strong local structure identifiers, and obtaining the local heat map; in, The encoder communicates with the input module, and the decoder communicates with the global network module and the morphology perception module respectively.
3. The hip joint key point detection system based on implicit and explicit structural constraints according to claim 2, characterized in that: The encoder is a convolutional neural network, and the convolution of the convolutional neural network is a residual convolution.
4. The hip joint key point detection system based on implicit and explicit structural constraints according to claim 3, characterized in that: The global network module includes: A dilated convolution unit, used to obtain contextual structural feature information, obtain a global heat map, and reduce the ambiguity problem of key point identification in the local network module; in, The dilated convolution unit communicates with the input module and the local network module respectively, and the dot product unit communicates with the dilated convolution unit.
5. The hip joint key point detection system based on implicit and explicit structural constraints according to claim 4, characterized in that: The morphological perception module obtains the explicit morphological association between the key points through the stored hip joint morphological perception loss function, wherein the hip joint morphological perception loss function is expressed as follows: L m =λL d +βL a Where λ and β are loss parameters, L d Represents the distance loss, L a Represents the cosine similarity loss of angle vectors.
Citation Information
Patent Citations
Skeletal anatomy key point positioning method in hip joint image, electronic equipment and medium
CN113033581A
Child hip joint dysplasia diagnosis system
CN114795258A
