Medical image key point detection method and device, equipment and medium

Abstract

The invention discloses a medical image key point detection method and device, equipment and a medium, and aims at detection of hip joint key points in a medical CT image based on skeletal anatomy prior information and a deep neural network. According to the method, common skeleton structure missing or damage conditions in clinical image data are considered, and anatomical prior information is introduced to realize key point detection of a skeleton defect part. And the current common deep neural network structure for detection is improved, and the balance between the detection precision and the detection efficiency is realized. In addition, the method improves the loss function of the detection network, and uses a skeleton region mask to correct the position of the predicted key point, so that the predicted position result of the key point better conforms to normal cognition.

Description

Technical Field

[0001] The present invention relates to the technical field of key point detection, and in particular to a method, device, equipment and medium for detecting key points of medical images based on anatomical prior information. Background Art

[0002] In modern computer-assisted orthopedic surgery, hip joint key points are crucial for surgical planning of this modern form of surgery. In conventional clinical practice, hip joint key points are manually annotated on CT images by experts, and the annotation process consumes a lot of time and labor costs.

[0003] In addition, manual key point labeling is not only highly dependent on the experience and skills of experts, but also has the problem of large individual differences in labeling results. Therefore, an efficient and accurate automatic hip joint key point detection method is urgently needed in clinical practice.

[0004] However, due to the high resolution of hip CT images, there is a large error in the detection of key points on high-resolution images; and because most clinical hip data contain bone structure defects, the missing bone structure information also greatly reduces the detection effect of automatic methods on hip key points. Therefore, accurate and real-time detection of hip key points under bone defect conditions is still a challenge.

[0005] At present, the automatic detection methods for hip joint key points include atlas-based methods and learning-based methods, but these methods have certain limitations. First, the detection effect of the atlas-based method is greatly affected by the constructed atlas, and the atlas has limited expression ability, and it is difficult to express the distribution of features that are not in the atlas. Secondly, due to its own intelligent agent training mode, the reinforcement learning-based method is difficult to detect multiple target key points at the same time, and its low detection efficiency greatly limits its application in clinical scenarios. The learning-based method has a better detection effect. The commonly used methods are Mask RCNN-based methods and U-shaped network-based methods. However, due to the high resolution of hip CT images, the training parameters of the Mask RCNN method using anchor frames are increased. In addition, most researchers use the network to directly learn the features of CT images for key point detection, but they ignore the correlation between these hip joint key points and bones, which leads to the distribution of the detected key points not conforming to the objective law, resulting in low key point detection accuracy. Summary of the invention

[0006] In view of the above problems, the present invention provides a medical image key point detection method, apparatus, device and medium for overcoming the above problems or at least partially solving the above problems.

[0007] The present invention provides the following scheme:

[0008] A method for detecting key points of a medical image, comprising:

[0009] Acquire a CT image to be processed, where the CT image to be processed is a CT image of a patient at a joint position to be detected;

[0010] Preprocessing the CT image to be processed to obtain a CT image of the joint to be detected with impurities removed; the impurities include artificial prosthesis implants or bone fragments;

[0011] Inputting the joint CT image to be detected into the key point detection network model to obtain the key point positions of the CT image to be processed;

[0012] Wherein, the key point detection network model includes a coarse detection network, an image slicing module and a fine detection network;

[0013] The coarse detection network includes a personalized key point distribution learning module and a local bone segmentation module; the personalized key point distribution learning module is used to fuse the general key point distribution information with the features of the joint CT image to be detected to obtain personalized key point distribution information; the local bone segmentation module is used to extract features from the joint CT image to be detected and output a bone local area prediction mask containing key points in combination with the personalized key point distribution information;

[0014] The image slicing module is used to slice the bone local area prediction mask to obtain a series of local image slices;

[0015] The precision detection network is used to process a series of input local image slices to obtain the key point positions of the CT image to be processed; the precision detection network includes a local learning module and a global learning module; the global learning module is used to capture the overall correlation within the slice and the correlation between slices.

[0016] Preferably: the personalized key point distribution learning module learns and obtains the general key point distribution information from a pre-established key point distribution data set corresponding to the joint to be detected;

[0017] The personalized key point distribution information is obtained by extracting features of the universal key point distribution information of the joint to be detected through a convolutional neural network and fusing the features with the CT image features of the joint to be detected.

[0018] Preferably, the personalized key point distribution learning module learns to obtain general key point distribution information from a pre-established key point distribution data set corresponding to the joint to be detected, including:

[0019] A key point coordinate distribution is randomly selected from the key point distribution data set as a template, and the remaining key point distributions are aligned to the approximate position of the key point distribution using a rigid registration method to construct a key point coordinate data set; a volume data set is constructed for each key point distribution, and the size is the same as the CT joint image after subsequent preprocessing, and each voxel in the volume data is assigned a value to obtain the general key point distribution information, and the formula is as follows:

[0020]

[0021] In the formula, v j represents the voxel value of the voxel position to be calculated, C j voxel represents the coordinates of the voxel, C i ld is the coordinate of the i-th joint key point, dist(·) represents the calculation of the Euclidean distance, and ln represents the logarithmic function with e as the base.

[0022] Preferably, the bone local region prediction mask includes the bone region of the target range around each key point in the personalized key point distribution information.

[0023] Preferably, the slicing process includes taking the center of the mask area of ​​the local bone area prediction mask as the center of the sphere, expanding outward with a radius ε, and finding the optimal ε, and the formula is as follows:

[0024]

[0025] In the formula, ρ is a minimum radius, ζ represents the function of finding the optimal ε, N represents the function of calculating the number of given conditions, and v represents the voxel value in the three-dimensional image;

[0026] An outer bounding box is taken for the spherical area, and the image pixels within the outer bounding box are resampled to a target size to obtain a series of local image slices.

[0027] Preferably, during the coarse detection network training process, the network parameters are optimized by supervising the loss function, and the loss function formula is as follows:

[0028]

[0029] In the formula, The segmentation loss between the local skeleton segmentation mask of the output result and the local skeleton segmentation mask of the gold standard is calculated, which is composed of soft Dice loss; The difference between the predicted values ​​of the predicted region and the gold standard is calculated, which is composed of the MSE loss.

[0030] Preferably, during the training process of the precision detection network, the network parameters are optimized by supervising the loss function, and the loss function formula is as follows:

[0031]

[0032] In the formula, ω i Represents a weight map based on the number of iterations. The weights for the bone area and the non-bone area are different, and the sum of the two is 1; v i With v i ′ They represent the voxel values ​​of the gold standard heat map and the voxel values ​​of the predicted heat map, respectively.

[0033] A medical image key point detection device, used to perform the above-mentioned medical image key point detection method, the device comprising:

[0034] An image acquisition unit, used for acquiring a CT image to be processed, wherein the CT image to be processed is a CT image of a patient at a joint position to be detected;

[0035] An image preprocessing unit, used for preprocessing the CT image to be processed to obtain a CT image of the joint to be detected with impurities removed; the impurities include artificial prosthesis implants or bone fragments;

[0036] A key point detection unit, used for inputting the joint CT image to be detected into the key point detection network model to obtain the key point positions of the CT image to be processed;

[0037] Wherein, the key point detection network model includes a coarse detection network, an image slicing module and a fine detection network;

[0038] The coarse detection network includes a personalized key point distribution learning module and a local bone segmentation module; the personalized key point distribution learning module is used to fuse the general key point distribution information with the features of the joint CT image to be detected to obtain personalized key point distribution information; the local bone segmentation module is used to extract features from the joint CT image to be detected and output a bone local area prediction mask containing key points in combination with the personalized key point distribution information;

[0039] The image slicing module is used to slice the bone local area prediction mask to obtain a series of local image slices;

[0040] The precision detection network is used to process a series of input local image slices to obtain the key point positions of the CT image to be processed; the precision detection network includes a local learning module and a global learning module; the global learning module is used to capture the overall correlation within the slice and the correlation between slices.

[0041] A medical image key point detection device, the device comprising a processor and a memory:

[0042] The memory is used to store program code and transmit the program code to the processor;

[0043] The processor is used to execute the above-mentioned medical image key point detection method according to the instructions in the program code.

[0044] A computer-readable storage medium is used to store program codes, and the program codes are used to execute the above-mentioned medical image key point detection method.

[0045] According to the specific embodiments provided by the present invention, the present invention discloses the following technical effects:

[0046] The embodiments of the present application provide a method, device, equipment and medium for detecting key points of medical images. The method constructs a "coarse detection-fine detection" network and integrates prior information of the bone structure to perform efficient and accurate joint key point detection. The fine detection network adopts a multi-core global learning module, which can well extract image features for the local bone area input by the coarse detection network. At the same time, a loss function based on the bone shape area is provided, so that the predicted distribution of hip joint key points is more consistent with the bone shape, eliminating many unreasonable prediction possibilities, thereby improving the detection accuracy.

[0047] Of course, any product implementing the present invention does not necessarily need to achieve all of the advantages described above at the same time. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention, and for ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0049] Figure 1 is a flow chart of a medical image key point detection method provided by an embodiment of the present invention;

[0050] Figure 2 is a flow chart of a CT hip joint key point detection method based on bone prior information provided by an embodiment of the present invention;

[0051] Figure 3 is a flow chart of a method for constructing personalized key point distribution provided by an embodiment of the present invention;

[0052] Figure 4 is a flow chart of the process of assigning values ​​to each voxel in volume data provided by an embodiment of the present invention;

[0053] Figure 5It is a local skeleton segmentation network diagram of a coarse detection network provided by an embodiment of the present invention;

[0054] Figure 6 It is a local-global information extraction network diagram of a precision detection network provided by an embodiment of the present invention;

[0055] Figure 7 is a schematic diagram of a medical image key point detection device provided by an embodiment of the present invention;

[0056] Figure 8 Schematic diagram of a medical image key point detection device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0057] The technical scheme in the embodiment of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiment of the present invention. Obviously, the described embodiment is only a part of the embodiment of the present invention, not all of the embodiments. Based on the embodiment of the present invention, all other embodiments obtained by ordinary technicians in this field belong to the scope of protection of the present invention.

[0058] See also Figure 1 , is a medical image key point detection method provided by an embodiment of the present invention, such as Figure 1 As shown, the method may include:

[0059] S101: Acquire a CT image to be processed, where the CT image to be processed is a CT image of a patient at a joint position to be detected;

[0060] S102: preprocessing the CT image to be processed to obtain a CT image of the joint to be detected with impurities removed; the impurities include artificial prosthesis implants or bone fragments;

[0061] S103: inputting the joint CT image to be detected into the key point detection network model to obtain the key point positions of the CT image to be processed;

[0062] Wherein, the key point detection network model includes a coarse detection network, an image slicing module and a fine detection network;

[0063] The coarse detection network includes a personalized key point distribution learning module and a local bone segmentation module; the personalized key point distribution learning module is used to fuse the general key point distribution information with the features of the joint CT image to be detected to obtain personalized key point distribution information; the local bone segmentation module is used to extract features from the joint CT image to be detected and output a bone local area prediction mask containing key points in combination with the personalized key point distribution information;

[0064] In specific implementation, the embodiment of the present application can provide that the personalized key point distribution learning module can learn to obtain the general key point distribution information from the pre-established key point distribution data set corresponding to the joint to be detected;

[0065] The personalized key point distribution information is obtained by extracting features of the universal key point distribution information of the joint to be detected through a convolutional neural network and fusing the features with the CT image features of the joint to be detected.

[0066] Furthermore, the personalized key point distribution learning module learns to obtain general key point distribution information from a pre-established key point distribution data set corresponding to the joint to be detected, including:

[0067] A key point coordinate distribution is randomly selected from the key point distribution data set as a template, and the remaining key point distributions are aligned to the approximate position of the key point distribution using a rigid registration method to construct a key point coordinate data set; corresponding volume data is constructed for each key point distribution, and the size is the same as the CT joint image after subsequent preprocessing, and each voxel in the volume data is assigned a value to obtain the general key point distribution information, and the formula is as follows:

[0068]

[0069] In the formula, v j represents the voxel value of the voxel position to be calculated, C j voxel represents the coordinates of the voxel, C i ld is the coordinate of the i-th joint key point, dist(·) represents the calculation of the Euclidean distance, and ln represents the logarithmic function with e as the base.

[0070] The skeleton local area prediction mask includes the skeleton area of ​​the target range around each key point in the personalized key point distribution information.

[0071] The image slicing module is used to slice the bone local area prediction mask to obtain a series of local image slices; in specific implementation, the embodiment of the present application can provide that the slicing process includes taking the mask area center of the bone local area prediction mask as the sphere center, expanding outward with a radius ε, and finding the optimized ε, the formula is as follows:

[0072]

[0073] In the formula, ρ is a minimum radius, ζ represents the function of finding the optimal ε, N represents the function of calculating the number of given conditions, and v represents the voxel value in the three-dimensional image;

[0074] An outer bounding box is taken for the spherical area, and the image pixels within the outer bounding box are resampled to a target size to obtain a series of local image slices.

[0075] The precision detection network is used to process a series of input local image slices to obtain the key point positions of the CT image to be processed; the precision detection network includes a local learning module and a global learning module; the global learning module is used to capture the overall correlation within the slice and the correlation between slices.

[0076] In order to further improve the network training effect, the embodiment of the present application can provide the coarse detection network training process by supervising the loss function to optimize the network parameters, the loss function formula is as follows:

[0077]

[0078] In the formula, The segmentation loss between the local skeleton segmentation mask of the output result and the local skeleton segmentation mask of the gold standard is calculated, which is composed of soft Dice loss; The difference between the predicted values ​​of the predicted region and the gold standard is calculated, which is composed of the MSE loss.

[0079] During the training process of the precision detection network, the network parameters are optimized by supervising the loss function, and the loss function formula is as follows:

[0080]

[0081] In the formula, ω i Represents a weight map based on the number of iterations. The weights for the bone area and the non-bone area are different, and the sum of the two is 1; v i With v i ′ They represent the voxel values ​​of the gold standard heat map and the voxel values ​​of the predicted heat map, respectively.

[0082] The medical image key point detection method provided in the embodiment of the present application solves the challenge of the difficulty of detecting the key points of joints in CT images. It mainly performs efficient and accurate hip joint key point detection by constructing a "coarse detection-fine detection" network and integrating the prior information of the bone structure. Among them, the network at each stage in the network model uses the U-Net network as the main architecture.

[0083] The coarse detection network consists of two core modules: a personalized key point distribution learning module, which is used to learn general key point distribution information from the constructed key point distribution dataset, and becomes personalized key point distribution information after being processed by a convolutional neural network; and a local bone segmentation module, which is used to extract features from the input CT image and predict the bone area containing the key points in combination with the personalized key point distribution information.

[0084] The precision detection network consists of a local learning module and a global learning module. A multi-core global learning module is proposed here, which can extract image features well for the local bone region of the input. In addition, this paper designs a loss function based on the bone shape area, so that the distribution of predicted joint key points is more consistent with the bone shape, eliminating many unreasonable prediction possibilities, thereby improving the detection accuracy.

[0085] The following uses the key point detection of the hip joint as an example to introduce in detail the medical image key point detection method provided in the embodiment of the present application.

[0086] The embodiment of the present application provides a CT hip joint key point detection method based on bone prior information, such as Figure 2 As shown, it mainly includes the following steps:

[0087] First, the collected data is preprocessed, and the key point coordinate labels corresponding to the data are also preprocessed. The two are respectively input into the CT hip joint key point detection network model and the personalized key point distribution learning model. The results of the personalized key point distribution learning model are fused into the network calculation as part of the features in the CT hip joint key point detection network model.

[0088] Then the network model is trained to obtain a trained network model.

[0089] Finally, the test image is input into the trained network model, and the hip joint key point detection results of the test image are output.

[0090] 1. Personalized key point distribution learning model.

[0091] The process of constructing personalized key point distribution is as follows Figure 3 As shown in Figure 1. First, the CT hip joint key point coordinates are preprocessed, a key point coordinate distribution is randomly selected as a template, and the remaining key point distributions are aligned to the approximate position of the key point distribution using a rigid registration method to construct a key point coordinate data set. Figure 4 As shown, for each key point distribution, the corresponding volume data is constructed, and the size is the same as the CT hip joint image after subsequent preprocessing. The voxels in the volume data are assigned values. The formula is as follows:

[0092]

[0093] Among them, v j is the voxel value of the voxel position to be calculated, C j voxel is the coordinate of the voxel, C i ld is the coordinate of the i-th hip joint key point, and dist(·) represents the calculation of the Euclidean distance.

[0094] In this way, a universal key point distribution is constructed, which is then fed into a convolutional neural network for feature extraction and becomes a personalized key point distribution after being fused with the input case image features.

[0095] 2. CT hip joint key point detection network model based on skeleton prior information.

[0096] First, the collected hip CT images were preprocessed, and the metal implants in the images were removed by threshold segmentation. At the same time, large bone fragments were manually outlined to remove the impurities in the hip CT images. The images were then resampled to a fixed size and put into the training network.

[0097] ① Coarse detection network.

[0098] The structure of the coarse detection network is as follows: Figure 5 As shown in the figure. The coarse detection network uses the idea of ​​task conversion. Since it is more difficult to find points of interest in high-resolution images than to find regions of interest, and the training is more efficient after converting the task into finding regions of interest, the coarse detection network uses a local bone segmentation network. The input is a hip CT image, and the output is a bone region within a certain range around each key point. The direct key point detection task is converted into a region segmentation task, which simplifies the task and improves the training effect. Compared with the current mainstream network that uses anchor boxes for region prediction, it greatly reduces the training parameters and training time, while ensuring the accuracy of region search.

[0099] ②Image slicing module.

[0100] After the image passes through the coarse detection network, the local bone region prediction mask is obtained. After passing through the image slicing module, a series of local image slices are obtained. The center of the mask area of ​​the local bone region prediction mask is taken as the center of the sphere, and the radius ε is expanded outward. In this process, the optimal ε is continuously sought. The formula is as follows:

[0101]

[0102] Among them, ρ is a minimum radius.

[0103] Then, an outer bounding box is taken for the spherical area, and the image pixels within the outer bounding box are resampled to a specified size, thereby forming a local image slice.

[0104] ③Precision detection network.

[0105] The precise detection network structure is as follows Figure 6 As shown in the figure. The precision detection network adopts a "local-global" module design, and the network is divided into a local learning module and a global learning module. The network input is a local image slice sequence. Since these local image slice sequences only contain local information and have sparse textures, it is difficult to directly use these slice sequences to extract effective information to predict the location of key points. Therefore, a global learning module is introduced here. The global learning module is used to capture the overall correlation within the slice and the correlation between slices. It adopts a multi-core convolution design to expand the receptive field and improve the information extraction effect.

[0106] 3. Train the CT hip joint key point detection network model.

[0107] The preprocessed CT image and the general hip joint key point distribution are simultaneously input into the coarse detection network of the CT hip joint key point detection network model for training, and the network parameters are optimized through the supervised loss function. The formula is as follows:

[0108]

[0109] in, The segmentation loss between the local skeleton segmentation mask of the output result and the local skeleton segmentation mask of the gold standard is calculated, which is composed of soft Dice loss; The difference between the predicted values ​​of the predicted region and the gold standard is calculated, which is composed of the MSE loss.

[0110] Subsequently, the fine detection network is trained, and the local bone region segmentation mask output by the coarse detection network is input into the image slicing module to obtain an image slice sequence, which is used as input to the training of the fine detection network. The network parameters are optimized through the supervised loss function. The formula is as follows:

[0111]

[0112] Among them, ω i is a weight map based on the number of iterations. The weights for the bone area and the non-bone area are different, and the sum of the two is 1. i With v i ′ are the voxel values ​​of the gold standard heat map and the voxel values ​​of the predicted heat map, respectively.

[0113] The embodiment of the present application constructs a loss function based on the shape of bones, and the weights of the bone area and the non-bone area are applied to the calculated MSE loss amount based on the number of training iterations, thereby constraining the distribution of the key points of the hip joint. Since this weight is based on the number of training iterations, at the beginning of training, the iterations are few, and the weight difference between the bone area and the non-bone area is not obvious. Especially in the first round, the weights for the bone area and the non-bone area are the same. Such a design effectively avoids the occurrence of overfitting and improves the training effect.

[0114] 4. Perform segmentation preprocessing on the test set.

[0115] The test data set is first preprocessed, and then the processed CT images are input into the trained hip joint key point network model for segmentation to obtain the key point positions of the test hip joint images.

[0116] In summary, the medical image key point detection method provided by the present application performs efficient and accurate joint key point detection by constructing a "coarse detection-fine detection" network and integrating the prior information of the bone structure. The fine detection network adopts a multi-core global learning module, which can well extract image features for the local bone area input by the coarse detection network. At the same time, a loss function based on the bone shape area is provided, so that the predicted hip joint key point distribution is more consistent with the bone shape, eliminating many unreasonable prediction possibilities, thereby improving the detection accuracy.

[0117] See also Figure 7 , the embodiment of the present application can also provide a medical image key point detection device, such as Figure 7 As shown, the device may include:

[0118] An image acquisition unit 701 is used to acquire a CT image to be processed, where the CT image to be processed is a CT image of a patient at a joint position to be detected;

[0119] An image preprocessing unit 702 is used to preprocess the CT image to be processed to obtain a CT image of the joint to be detected with impurities removed; the impurities include artificial prosthesis implants or bone fragments;

[0120] A key point detection unit 703 is used to input the joint CT image to be detected into the key point detection network model to obtain the key point positions of the CT image to be processed;

[0121] Wherein, the key point detection network model includes a coarse detection network, an image slicing module and a fine detection network;

[0122] The coarse detection network includes a personalized key point distribution learning module and a local bone segmentation module; the personalized key point distribution learning module is used to fuse the general key point distribution information with the features of the joint CT image to be detected to obtain personalized key point distribution information; the local bone segmentation module is used to extract features from the joint CT image to be detected and output a bone local area prediction mask containing key points in combination with the personalized key point distribution information;

[0123] The image slicing module is used to slice the bone local area prediction mask to obtain a series of local image slices;

[0124] The precision detection network is used to process a series of input local image slices to obtain the key point positions of the CT image to be processed; the precision detection network includes a local learning module and a global learning module; the global learning module is used to capture the overall correlation within the slice and the correlation between slices.

[0125] The present application may also provide a medical image key point detection device, the device comprising a processor and a memory:

[0126] The memory is used to store program code and transmit the program code to the processor;

[0127] The processor is used to execute the steps of the above-mentioned medical image key point detection method according to the instructions in the program code.

[0128] like Figure 8 As shown, a medical image key point detection device provided by an embodiment of the present application may include: a processor 10, a memory 11, a communication interface 12 and a communication bus 13. The processor 10, the memory 11, and the communication interface 12 all communicate with each other through the communication bus 13.

[0129] In the embodiment of the present application, the processor 10 may be a central processing unit (CPU), an application specific integrated circuit, a digital signal processor, a field programmable gate array or other programmable logic devices, etc.

[0130] The processor 10 may call a program stored in the memory 11. Specifically, the processor 10 may execute operations in an embodiment of the medical image key point detection method.

[0131] The memory 11 is used to store one or more programs, which may include program codes, and the program codes include computer operation instructions. In the embodiment of the present application, the memory 11 at least stores programs for implementing the following functions:

[0132] Acquire a CT image to be processed, where the CT image to be processed is a CT image of a patient at a joint position to be detected;

[0133] Preprocessing the CT image to be processed to obtain a CT image of the joint to be detected with impurities removed; the impurities include artificial prosthesis implants or bone fragments;

[0134] Inputting the joint CT image to be detected into the key point detection network model to obtain the key point positions of the CT image to be processed;

[0135] Wherein, the key point detection network model includes a coarse detection network, an image slicing module and a fine detection network;

[0136] The coarse detection network includes a personalized key point distribution learning module and a local bone segmentation module; the personalized key point distribution learning module is used to fuse the general key point distribution information with the features of the joint CT image to be detected to obtain personalized key point distribution information; the local bone segmentation module is used to extract features from the joint CT image to be detected and output a bone local area prediction mask containing key points in combination with the personalized key point distribution information;

[0137] The image slicing module is used to slice the bone local area prediction mask to obtain a series of local image slices;

[0138] The precision detection network is used to process a series of input local image slices to obtain the key point positions of the CT image to be processed; the precision detection network includes a local learning module and a global learning module; the global learning module is used to capture the overall correlation within the slice and the correlation between slices.

[0139] In one possible implementation, the memory 11 may include a program storage area and a data storage area, wherein the program storage area can store an operating system and application programs required for at least one function (such as a file creation function, a data reading and writing function), etc.; the data storage area can store data created during use, such as initialization data, etc.

[0140] In addition, the memory 11 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device or other volatile solid-state storage device.

[0141] The communication interface 12 may be an interface of a communication module, and is used to connect to other devices or systems.

[0142] Of course, it should be noted that Figure 8 The structure shown does not constitute a limitation on the medical image key point detection device in the embodiment of the present application. In actual applications, the medical image key point detection device may include Figure 8 More or fewer components than shown, or combinations of certain components.

[0143] The embodiment of the present application may also provide a computer-readable storage medium, wherein the computer-readable storage medium is used to store program code, and the program code is used to execute the steps of the above-mentioned medical image key point detection method.

[0144] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0145] It can be known from the description of the above implementation methods that those skilled in the art can clearly understand that the present application can be implemented by means of software plus a necessary general hardware platform. Based on such an understanding, the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product can be stored in a storage medium such as ROM / RAM, a magnetic disk, an optical disk, etc., and includes several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in the various embodiments of the present application or certain parts of the embodiments.

[0146] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can refer to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system or system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can refer to the partial description of the method embodiment. The system and system embodiments described above are merely schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without creative work.

[0147] The above description is only a preferred embodiment of the present invention and is not intended to limit the protection scope of the present invention. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention are included in the protection scope of the present invention.

Claims

1. A method for detecting key points in a medical image, characterized in that: include: Acquire a CT image to be processed, where the CT image to be processed is a CT image of a patient at a joint position to be detected; Preprocessing the CT image to be processed to obtain a CT image of the joint to be detected with impurities removed; the impurities include artificial prosthesis implants or bone fragments; Inputting the joint CT image to be detected into the key point detection network model to obtain the key point positions of the CT image to be processed; Wherein, the key point detection network model includes a coarse detection network, an image slicing module and a fine detection network; The coarse detection network includes a personalized key point distribution learning module and a local bone segmentation module; the personalized key point distribution learning module is used to fuse the general key point distribution information with the features of the joint CT image to be detected to obtain personalized key point distribution information; the local bone segmentation module is used to extract features from the joint CT image to be detected and output a bone local area prediction mask containing key points in combination with the personalized key point distribution information; The image slicing module is used to slice the bone local area prediction mask to obtain a series of local image slices; The fine detection network is used to process a series of local image slices input to obtain key point positions of the CT image to be processed; the fine detection network includes a local learning module and a global learning module; The global learning module is used to capture the overall correlation within the slices and the correlation between slices.

2. The method for detecting key points in medical images according to claim 1, characterized in that: The personalized key point distribution learning module learns and obtains the general key point distribution information from a pre-established key point distribution data set corresponding to the joint to be detected; The personalized key point distribution information is obtained by extracting features of the universal key point distribution information of the joint to be detected through a convolutional neural network and fusing the features with the CT image features of the joint to be detected.

3. The method for detecting key points of medical images according to claim 2, characterized in that: The personalized key point distribution learning module learns to obtain general key point distribution information from a pre-established key point distribution data set corresponding to the joint to be detected, including: A key point coordinate distribution is randomly selected from the key point distribution data set as a template, and the remaining key point distributions are aligned to the approximate position of the key point distribution using a rigid registration method to construct a key point coordinate data set; a volume data set is constructed for each key point distribution, and the size is the same as the CT joint image after subsequent preprocessing, and each voxel in the volume data is assigned a value to obtain the general key point distribution information, and the formula is as follows: In the formula, v j represents the voxel value of the voxel position to be calculated, C j voxel represents the coordinates of the voxel, C i ld is the coordinate of the i-th joint key point, dist(·) represents the calculation of the Euclidean distance, and ln represents the logarithmic function with e as the base.

4. The method for detecting key points in medical images according to claim 1, characterized in that: The skeleton local area prediction mask includes the skeleton area of ​​the target range around each key point in the personalized key point distribution information.

5. The method for detecting key points in medical images according to claim 1, characterized in that: The slicing process includes taking the center of the mask area of ​​the local bone area prediction mask as the center of the sphere, expanding outward with a radius ε, and finding the optimal ε. The formula is as follows: In the formula, ρ is a minimum radius, ζ represents the function of finding the optimal ε, N represents the function of calculating the number of given conditions, and v represents the voxel value in the three-dimensional image; An outer bounding box is taken for the spherical area, and the image pixels within the outer bounding box are resampled to a target size to obtain a series of local image slices.

6. The method for detecting key points of medical images according to claim 1, characterized in that: During the training process of the coarse detection network, the network parameters are optimized by supervising the loss function, and the loss function formula is as follows: In the formula, The segmentation loss between the local skeleton segmentation mask of the output result and the local skeleton segmentation mask of the gold standard is calculated, which is composed of soft Dice loss; The difference between the predicted values ​​of the predicted region and the gold standard is calculated, which is composed of the MSE loss.

7. The method for detecting key points in medical images according to claim 1, characterized in that: During the training process of the precision detection network, the network parameters are optimized by supervising the loss function, and the loss function formula is as follows: In the formula, ω i Represents a weight map based on the number of iterations. The weights for the bone area and the non-bone area are different, and the sum of the two is 1; v i With v i ′ They represent the voxel values ​​of the gold standard heat map and the voxel values ​​of the predicted heat map, respectively.

8. A medical image key point detection device, characterized in that: Used to execute the medical image key point detection method according to any one of claims 1 to 7, the device comprising: An image acquisition unit, used for acquiring a CT image to be processed, wherein the CT image to be processed is a CT image of a patient at a joint position to be detected; An image preprocessing unit, used for preprocessing the CT image to be processed to obtain a CT image of the joint to be detected with impurities removed; the impurities include artificial prosthesis implants or bone fragments; A key point detection unit, used for inputting the joint CT image to be detected into the key point detection network model to obtain the key point positions of the CT image to be processed; Wherein, the key point detection network model includes a coarse detection network, an image slicing module and a fine detection network; The coarse detection network includes a personalized key point distribution learning module and a local bone segmentation module; the personalized key point distribution learning module is used to fuse the general key point distribution information with the features of the joint CT image to be detected to obtain personalized key point distribution information; the local bone segmentation module is used to extract features from the joint CT image to be detected and output a bone local area prediction mask containing key points in combination with the personalized key point distribution information; The image slicing module is used to slice the bone local area prediction mask to obtain a series of local image slices; The precision detection network is used to process a series of input local image slices to obtain the key point positions of the CT image to be processed; the precision detection network includes a local learning module and a global learning module; the global learning module is used to capture the overall correlation within the slice and the correlation between slices.

9. A medical image key point detection device, characterized in that: The device comprises a processor and a memory: The memory is used to store program code and transmit the program code to the processor; The processor is used to execute the medical image key point detection method according to any one of claims 1-7 according to the instructions in the program code.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium is used to store program code, and the program code is used to execute the medical image key point detection method according to any one of claims 1-7.

Images