Medical image key point detection method, device, equipment and medium

Abstract

The application discloses a medical image key point detection method, device, equipment and medium, and is based on the skeleton anatomy prior information and the deep neural network for the detection of the hip joint key point in the medical CT image. The method considers the common bone structure missing or damage situation in the clinical image data, introduces the anatomy prior information to realize the key point detection for the bone damage part. And the commonly used detection deep neural network structure is improved, the balance between the detection precision and the detection efficiency is realized. In addition, the loss function of the detection network is improved, the skeleton region mask is used to correct the predicted key point position, so that the position result of the key point prediction is more consistent with the normal cognition.

Description

Technical Field

[0001] This invention relates to the field of key point detection technology, and in particular to a method, apparatus, equipment and medium for key point detection in medical images based on prior anatomical information. Background Technology

[0002] In modern computer-assisted orthopedic surgery, key points of the hip joint are crucial for surgical planning in this modern surgical approach. In routine clinical practice, these key points are manually marked by experts on CT images, a process that consumes significant time and manpower.

[0003] Furthermore, manual annotation of key points not only heavily relies on the experience and skills of experts but also suffers from significant individual variations in annotation results. Therefore, there is an urgent clinical need for an efficient and accurate automated method for detecting key points in the hip joint.

[0004] However, due to the high resolution of hip CT images, the detection of key points on high-resolution images suffers from significant errors. Furthermore, since clinical hip data often contains bone defects, the missing bone structural information greatly reduces the effectiveness of automated methods in detecting hip key points. Therefore, accurate real-time detection of hip key points under conditions of bone defects remains a challenge.

[0005] Current methods for automatic detection of hip joint key points include atlas-based and learning-based methods, but both have limitations. First, the detection performance of atlas-based methods is greatly affected by the constructed atlas, and the expressive power of atlases is limited, making it difficult to represent the distribution of features not present in the atlas. Second, reinforcement learning-based methods, due to their agent training model, struggle to detect multiple target key points simultaneously, and their low detection efficiency significantly limits their application in clinical scenarios. Learning-based methods generally offer better detection performance; commonly used methods include Mask R-CNN and U-shaped network methods. However, the high resolution of hip CT images leads to increased training parameters for the Mask R-CNN method using anchor boxes. Furthermore, most researchers directly learn features from CT images for key point detection, but they overlook the correlation between these hip joint key points and the bone structure, resulting in a key point distribution that does not conform to objective laws, leading to low key point detection accuracy. Summary of the Invention

[0006] In view of the above problems, the present invention provides a method, apparatus, device and medium for detecting key points in medical images to overcome the above problems or at least partially solve the above problems.

[0007] This invention provides the following solution:

[0008] A method for detecting key points in medical images, comprising:

[0009] Acquire a CT image to be processed, wherein the CT image to be processed is a CT image of the joint location to be detected on the patient;

[0010] The CT image to be processed is preprocessed to obtain a CT image of the joint to be detected after removing impurities; the impurities include artificial prosthesis implants or bone fragments.

[0011] The key point positions of the CT image to be processed are obtained by inputting the CT image of the joint to be detected into the key point detection network model.

[0012] The key point detection network model includes a coarse detection network, an image slicing module, and a fine detection network.

[0013] The coarse detection network includes a personalized keypoint distribution learning module and a local bone segmentation module. The personalized keypoint distribution learning module is used to fuse general keypoint distribution information with the features of the CT image of the joint to be detected to obtain personalized keypoint distribution information. The local bone segmentation module is used to extract features from the CT image of the joint to be detected and combine them with the personalized keypoint distribution information to output a predicted mask of the local bone region containing keypoints.

[0014] The image slicing module is used to slice the predicted mask of the local bone region to obtain a series of local image slices.

[0015] The fine detection network is used to process a series of local image slices to obtain the key point locations of the CT image to be processed; the fine detection network includes a local learning module and a global learning module; the global learning module is used to capture the overall correlation within a slice and the correlation between slices.

[0016] Preferably, the personalized keypoint distribution learning module learns the general keypoint distribution information from a pre-established keypoint distribution dataset corresponding to the joint to be detected;

[0017] The personalized key point distribution information is obtained by fusing the general key point distribution information of the joint to be detected with the CT image features of the joint to be detected through a convolutional neural network.

[0018] Preferably, the personalized keypoint distribution learning module learns general keypoint distribution information from a pre-established keypoint distribution dataset corresponding to the joint to be detected, including:

[0019] A keypoint coordinate distribution is randomly selected from the keypoint distribution dataset as a template. The remaining keypoint distributions are aligned to the approximate position of this template using a rigid registration method, thus constructing a keypoint coordinate dataset. For each keypoint distribution, a volumetric dataset is constructed with the same size as the subsequently preprocessed CT joint image. Values ​​are assigned to each voxel within the volumetric dataset to obtain the general keypoint distribution information, as shown in the following formula:

[0020]

[0021] In the formula, v j C represents the voxel value of the voxel location to be calculated. j voxel represents the coordinates of a voxel, C i ld represents the coordinates of the i-th key point, dist(·) represents the calculation of Euclidean distance, and ln represents the logarithmic function with base e.

[0022] Preferably, the skeletal local region prediction mask includes the skeletal region of the target range around each keypoint in the personalized keypoint distribution information.

[0023] Preferably, the slicing process includes expanding outwards with a radius ε, using the center of the mask region of the predicted local bone region as the center of a sphere, as the formula below:

[0024]

[0025] In the formula, ρ is a minimum radius, ζ represents the function for finding the optimal ε, N represents the function for calculating the number of given conditions, and v represents the voxel value in the three-dimensional image.

[0026] Take an outer bounding box for the spherical region, and resample the image pixels within the outer bounding box to the target size to obtain a series of local image slices.

[0027] Preferably, the network parameters are optimized during the training of the coarse detection network using a supervised loss function, the formula of which is as follows:

[0028]

[0029] In the formula, The calculation is the segmentation loss between the local skeleton segmentation mask of the output result and the gold standard local skeleton segmentation mask, which is composed of soft Dice loss; The calculation measures the difference between the predicted value of the predicted region and the gold standard, which is composed of MSE loss.

[0030] Preferably, the network parameters are optimized using a supervised loss function during the training process of the precision detection network. The loss function formula is as follows:

[0031]

[0032] In the formula, ω i This represents a weighted map based on the number of iterations, where the weights for skeletal and non-skeletal regions are different, and the sum of the weights is 1; v i With v i ′ These represent the voxel values ​​of the gold standard heatmap and the predicted heatmap, respectively.

[0033] A medical image key point detection device, used to perform the above-described medical image key point detection method, the device comprising:

[0034] An image acquisition unit is used to acquire a CT image to be processed, wherein the CT image to be processed is a CT image of the joint location to be detected on the patient.

[0035] An image preprocessing unit is used to preprocess the CT image to be processed to obtain a CT image of the joint to be detected after removing impurities; the impurities include artificial prosthesis implants or bone fragments.

[0036] The key point detection unit is used to input the CT image of the joint to be detected into the key point detection network model to obtain the key point positions of the CT image to be processed.

[0037] The key point detection network model includes a coarse detection network, an image slicing module, and a fine detection network.

[0038] The coarse detection network includes a personalized keypoint distribution learning module and a local bone segmentation module. The personalized keypoint distribution learning module is used to fuse general keypoint distribution information with the features of the CT image of the joint to be detected to obtain personalized keypoint distribution information. The local bone segmentation module is used to extract features from the CT image of the joint to be detected and combine them with the personalized keypoint distribution information to output a predicted mask of the local bone region containing keypoints.

[0039] The image slicing module is used to slice the predicted mask of the local bone region to obtain a series of local image slices.

[0040] The fine detection network is used to process a series of local image slices to obtain the key point locations of the CT image to be processed; the fine detection network includes a local learning module and a global learning module; the global learning module is used to capture the overall correlation within a slice and the correlation between slices.

[0041] A medical image key point detection device, the device including a processor and a memory:

[0042] The memory is used to store program code and transmit the program code to the processor;

[0043] The processor is used to execute the above-described medical image key point detection method according to the instructions in the program code.

[0044] A computer-readable storage medium for storing program code for performing the above-described medical image key point detection method.

[0045] According to specific embodiments provided by the present invention, the present invention discloses the following technical effects:

[0046] This application provides a method, apparatus, device, and medium for detecting key points in medical images. The method constructs a "coarse detection-fine detection" network and integrates prior information about skeletal structure to achieve efficient and accurate joint key point detection. The fine detection network employs a multi-core global learning module, which can effectively extract image features from the local skeletal regions input to the coarse detection network. Simultaneously, a loss function based on skeletal shape regions is provided, making the predicted hip joint key point distribution more closely match the skeletal shape, eliminating many unreasonable prediction possibilities, thereby improving detection accuracy.

[0047] Of course, any product implementing this invention does not necessarily need to achieve all of the advantages described above at the same time. Attached Figure Description

[0048] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments will be briefly described below. Obviously, the drawings described below are merely some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without any creative effort.

[0049] Figure 1 This is a flowchart of the medical image key point detection method provided in the embodiments of the present invention;

[0050] Figure 2 This is a flowchart of a CT hip joint key point detection method based on skeletal prior information provided in an embodiment of the present invention;

[0051] Figure 3 This is a flowchart of the method for constructing personalized key point distribution provided in an embodiment of the present invention;

[0052] Figure 4 This is a flowchart of the process of assigning values ​​to each voxel within the volume data provided in the embodiments of the present invention;

[0053] Figure 5This is a local skeleton segmentation network diagram of the coarse detection network provided in this embodiment of the invention;

[0054] Figure 6 This is a local-global information extraction network diagram of the precision detection network provided in this embodiment of the invention;

[0055] Figure 7 This is a schematic diagram of the medical image key point detection device provided in an embodiment of the present invention;

[0056] Figure 8 This is a schematic diagram of the medical image key point detection device provided in an embodiment of the present invention. Detailed Implementation

[0057] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention are within the scope of protection of the present invention.

[0058] See Figure 1 This invention provides a method for detecting key points in medical images, such as... Figure 1 As shown, the method may include:

[0059] S101: Acquire the CT image to be processed, wherein the CT image to be processed is the CT image of the joint location to be detected on the patient;

[0060] S102: Preprocess the CT image to be processed to obtain a CT image of the joint to be detected after removing impurities; the impurities include artificial prosthesis implants or bone fragments;

[0061] S103: Input the CT image of the joint to be detected into the key point detection network model to obtain the key point positions of the CT image to be processed;

[0062] The key point detection network model includes a coarse detection network, an image slicing module, and a fine detection network.

[0063] The coarse detection network includes a personalized keypoint distribution learning module and a local bone segmentation module. The personalized keypoint distribution learning module is used to fuse general keypoint distribution information with the features of the CT image of the joint to be detected to obtain personalized keypoint distribution information. The local bone segmentation module is used to extract features from the CT image of the joint to be detected and combine them with the personalized keypoint distribution information to output a predicted mask of the local bone region containing keypoints.

[0064] In a specific implementation, the embodiments of this application may provide the personalized keypoint distribution learning module to learn the general keypoint distribution information from the pre-established keypoint distribution dataset corresponding to the joint to be detected;

[0065] The personalized key point distribution information is obtained by fusing the general key point distribution information of the joint to be detected with the CT image features of the joint to be detected through a convolutional neural network.

[0066] Furthermore, the personalized keypoint distribution learning module learns general keypoint distribution information from a pre-established keypoint distribution dataset corresponding to the joint to be detected, including:

[0067] A keypoint coordinate distribution is randomly selected from the keypoint distribution dataset as a template. The remaining keypoint distributions are aligned to the approximate position of this template using a rigid registration method, thus constructing a keypoint coordinate dataset. For each keypoint distribution, corresponding volume data is constructed with the same size as the subsequently preprocessed CT joint image. Values ​​are assigned to each voxel within the volume data to obtain the general keypoint distribution information, as shown in the following formula:

[0068]

[0069] In the formula, v j C represents the voxel value of the voxel location to be calculated. j voxel represents the coordinates of a voxel, C i ld represents the coordinates of the i-th key point, dist(·) represents the calculation of Euclidean distance, and ln represents the logarithmic function with base e.

[0070] The skeletal local region prediction mask includes the skeletal region of the target range around each keypoint in the personalized keypoint distribution information.

[0071] The image slicing module is used to slice the predicted mask of the local bone region to obtain a series of local image slices. In a specific implementation, the slicing process in this application embodiment may include expanding outward with a radius ε, using the center of the mask region of the predicted mask of the local bone region as the center of a sphere, as the formula below:

[0072]

[0073] In the formula, ρ is a minimum radius, ζ represents the function for finding the optimal ε, N represents the function for calculating the number of given conditions, and v represents the voxel value in the three-dimensional image.

[0074] Take an outer bounding box for the spherical region, and resample the image pixels within the outer bounding box to the target size to obtain a series of local image slices.

[0075] The fine detection network is used to process a series of local image slices to obtain the key point locations of the CT image to be processed; the fine detection network includes a local learning module and a global learning module; the global learning module is used to capture the overall correlation within a slice and the correlation between slices.

[0076] To further improve the network training effect, embodiments of this application can provide a supervised loss function to optimize network parameters during the training process of the coarse detection network. The formula for the loss function is as follows:

[0077]

[0078] In the formula, The calculation is the segmentation loss between the local skeleton segmentation mask of the output result and the gold standard local skeleton segmentation mask, which is composed of soft Dice loss; The calculation measures the difference between the predicted value of the predicted region and the gold standard, which is composed of MSE loss.

[0079] The precision detection network is trained using a supervised loss function to optimize network parameters. The loss function formula is as follows:

[0080]

[0081] In the formula, ω i This represents a weighted map based on the number of iterations, where the weights for skeletal and non-skeletal regions are different, and the sum of the weights is 1; v i With v i ′ These represent the voxel values ​​of the gold standard heatmap and the predicted heatmap, respectively.

[0082] The medical image key point detection method provided in this application addresses the challenge of detecting joint key points in current CT images. It primarily achieves efficient and accurate hip joint key point detection by constructing a "coarse detection-fine detection" network and fusing prior information about skeletal structure. The network model uses the U-Net network as its main architecture at each stage.

[0083] The coarse detection network consists of two core modules: a personalized keypoint distribution learning module, which learns general keypoint distribution information from the constructed keypoint distribution dataset and transforms it into personalized keypoint distribution information after processing by a convolutional neural network; and a local skeleton segmentation module, which extracts features from the input CT image and combines them with the personalized keypoint distribution information to predict the skeletal regions containing keypoints.

[0084] The precision detection network consists of a local learning module and a global learning module. Here, a multi-kernel global learning module is proposed, which can effectively extract image features from the input local skeletal region. Furthermore, this paper designs a loss function based on the skeletal shape region, making the predicted joint keypoint distribution more closely match the skeletal shape, eliminating many unreasonable predictions, and thus improving detection accuracy.

[0085] The following section uses the detection of key points in the hip joint as an example to provide a detailed description of the medical image key point detection method provided in this application.

[0086] The CT hip joint key point detection method based on skeletal prior information provided in this application embodiment, such as Figure 2 As shown, the main steps include:

[0087] First, the collected data is preprocessed, and the coordinate labels of the corresponding key points are also preprocessed. Both are then input into the CT hip joint key point detection network model and the personalized key point distribution learning model, respectively. The result of the personalized key point distribution learning model is fused into the CT hip joint key point detection network model as part of the network's calculation.

[0088] Next, the network model is trained to obtain a trained network model.

[0089] Finally, the test image is input into the trained network model, which outputs the hip joint key point detection results of the test image.

[0090] 1. Personalized keypoint distribution learning model.

[0091] The process of constructing a personalized keypoint distribution is as follows: Figure 3 As shown in the diagram, the coordinates of key points on the CT hip joint are first preprocessed. A random key point coordinate distribution is selected as a template, and the remaining key point distributions are aligned to the approximate position of this template using a rigid registration method, thus constructing a key point coordinate dataset. Figure 4 As shown, volumetric data is constructed for each keypoint distribution, with the same size as the subsequently preprocessed CT hip joint image. Values ​​are assigned to each voxel within the volumetric data using the following formula:

[0092]

[0093] Among them, v j C represents the voxel value of the voxel location to be calculated. j voxel is the coordinate of the voxel, C i ld represents the coordinates of the i-th hip joint key point, and dist(·) represents the calculation of the Euclidean distance.

[0094] This leads to the construction of a general keypoint distribution. Subsequently, this general keypoint distribution is fed into a convolutional neural network for feature extraction. After being fused with the features of the input case image, it becomes a personalized keypoint distribution.

[0095] 2. CT hip joint key point detection network model based on skeletal prior information.

[0096] First, the acquired hip CT images are preprocessed using threshold segmentation to remove metal implants. Simultaneously, large bone fragments are manually delineated to remove impurities, resulting in a clean hip CT image. The image is then resampled to a fixed size and fed into the training network.

[0097] ① Coarse detection network.

[0098] The structure of the coarse detection network is as follows: Figure 5 As shown, the coarse detection network employs a task transformation approach. Since finding points of interest (POIs) in high-resolution images is more difficult than finding regions of interest (ROIs), and training is more efficient after transforming it into a region-of-interest (ROI) search task, the coarse detection network uses a local skeleton segmentation network. The input is a hip joint CT image, and the output is a skeletal region within a certain range around each POI. This transforms the direct POI detection task into a region segmentation task, simplifying the task and improving training performance. Compared to current mainstream networks that use anchor boxes for region prediction, this significantly reduces training parameters and training time while maintaining region-finding accuracy.

[0099] ② Image slicing module.

[0100] After passing through a coarse detection network, the image yields a skeletal local region prediction mask. This mask, after passing through an image slicing module, produces a series of local image slices. Using the center of the skeletal local region prediction mask as the center of a sphere, the image is expanded outwards with a radius ε. During this process, the optimal ε is continuously sought, as shown in the following formula:

[0101]

[0102] Where ρ is a minimum radius.

[0103] Then, an outer bounding box is taken for the spherical region, and the image pixels within this outer bounding box are resampled to a specified size, thereby forming a local image slice.

[0104] ③ Precision detection network.

[0105] Precision detection network structure such as Figure 6 As shown, the precision detection network adopts a "local-global" modular design, consisting of a local learning module and a global learning module. The network input is a sequence of local image slices. Since these local image slice sequences only contain local information and have sparse texture, it is difficult to extract effective information to predict keypoint locations directly using these slice sequences. Therefore, a global learning module is introduced here. The global learning module is used to capture the overall correlation within a slice and the correlation between slices. It adopts a multi-kernel convolution design to expand the receptive field and improve the information extraction effect.

[0106] 3. Train the CT hip joint key point detection network model.

[0107] The preprocessed CT images and the general hip joint key point distribution are simultaneously input into the coarse detection network of the CT hip joint key point detection network model for training. The network parameters are optimized using a supervised loss function, as shown in the following formula:

[0108]

[0109] in, The calculation is the segmentation loss between the local skeleton segmentation mask of the output result and the gold standard local skeleton segmentation mask, which is composed of soft Dice loss; The calculation measures the difference between the predicted value of the predicted region and the gold standard, which is composed of MSE loss.

[0110] Subsequently, the fine detection network is trained by inputting the local skeletal region segmentation mask output by the coarse detection network into the image slicing module to obtain an image slice sequence, which is then used as input to train the fine detection network. The network parameters are optimized using a supervised loss function, as shown in the following formula:

[0111]

[0112] Where, ω i This is a weighted map based on the number of iterations, with different weights for skeletal and non-skeletal regions, and the sum of the weights for the two is 1. i With v i ′ These are the voxel values ​​of the gold standard heatmap and the predicted heatmap, respectively.

[0113] This application constructs a loss function based on bone shape, applying weights to skeletal and non-skeletal regions based on the number of training iterations to the calculated MSE loss, thereby constraining the distribution of hip joint key points. Since this weighting is based on the number of training iterations, at the beginning of training, with fewer iterations, the weight difference between skeletal and non-skeletal regions is not significant. Especially in the first round, the weights for skeletal and non-skeletal regions are the same. This design effectively avoids overfitting and improves training performance.

[0114] 4. Perform segmentation preprocessing on the test set.

[0115] First, the test dataset is preprocessed. Then, the processed CT images are input into the trained hip joint keypoint network model for segmentation to obtain the keypoint locations of the test hip joint images.

[0116] In summary, the medical image keypoint detection method provided in this application achieves efficient and accurate joint keypoint detection by constructing a "coarse detection-fine detection" network and fusing prior information about skeletal structure. The fine detection network employs a multi-core global learning module, which can effectively extract image features from the local skeletal regions input to the coarse detection network. Simultaneously, a loss function based on skeletal shape regions is provided, making the predicted hip joint keypoint distribution more closely match the skeletal shape, eliminating many unreasonable predictions, thereby improving detection accuracy.

[0117] See Figure 7 This application embodiment can also provide a medical image key point detection device, such as... Figure 7 As shown, the device may include:

[0118] The image acquisition unit 701 is used to acquire a CT image to be processed, wherein the CT image to be processed is a CT image of the joint location to be detected on the patient.

[0119] The image preprocessing unit 702 is used to preprocess the CT image to be processed to obtain a CT image of the joint to be detected after removing impurities; the impurities include artificial prosthesis implants or bone fragments.

[0120] The key point detection unit 703 is used to input the CT image of the joint to be detected into the key point detection network model to obtain the key point positions of the CT image to be processed.

[0121] The key point detection network model includes a coarse detection network, an image slicing module, and a fine detection network.

[0122] The coarse detection network includes a personalized keypoint distribution learning module and a local bone segmentation module. The personalized keypoint distribution learning module is used to fuse general keypoint distribution information with the features of the CT image of the joint to be detected to obtain personalized keypoint distribution information. The local bone segmentation module is used to extract features from the CT image of the joint to be detected and combine them with the personalized keypoint distribution information to output a predicted mask of the local bone region containing keypoints.

[0123] The image slicing module is used to slice the predicted mask of the local bone region to obtain a series of local image slices.

[0124] The fine detection network is used to process a series of local image slices to obtain the key point locations of the CT image to be processed; the fine detection network includes a local learning module and a global learning module; the global learning module is used to capture the overall correlation within a slice and the correlation between slices.

[0125] This application embodiment can also provide a medical image key point detection device, the device including a processor and a memory:

[0126] The memory is used to store program code and transmit the program code to the processor;

[0127] The processor is used to execute the steps of the above-described medical image key point detection method according to the instructions in the program code.

[0128] like Figure 8 As shown in the figure, a medical image key point detection device provided in this application embodiment may include: a processor 10, a memory 11, a communication interface 12, and a communication bus 13. The processor 10, memory 11, and communication interface 12 all communicate with each other through the communication bus 13.

[0129] In the embodiments of this application, the processor 10 may be a central processing unit (CPU), an application-specific integrated circuit, a digital signal processor, a field-programmable gate array, or other programmable logic devices.

[0130] The processor 10 can call programs stored in the memory 11. Specifically, the processor 10 can execute operations in the embodiments of the medical image key point detection method.

[0131] The memory 11 is used to store one or more programs. The programs may include program code, which includes computer operation instructions. In this embodiment, the memory 11 stores at least a program for implementing the following functions:

[0132] Acquire a CT image to be processed, wherein the CT image to be processed is a CT image of the joint location to be detected on the patient;

[0133] The CT image to be processed is preprocessed to obtain a CT image of the joint to be detected after removing impurities; the impurities include artificial prosthesis implants or bone fragments.

[0134] The key point positions of the CT image to be processed are obtained by inputting the CT image of the joint to be detected into the key point detection network model.

[0135] The key point detection network model includes a coarse detection network, an image slicing module, and a fine detection network.

[0136] The coarse detection network includes a personalized keypoint distribution learning module and a local bone segmentation module. The personalized keypoint distribution learning module is used to fuse general keypoint distribution information with the features of the CT image of the joint to be detected to obtain personalized keypoint distribution information. The local bone segmentation module is used to extract features from the CT image of the joint to be detected and combine them with the personalized keypoint distribution information to output a predicted mask of the local bone region containing keypoints.

[0137] The image slicing module is used to slice the predicted mask of the local bone region to obtain a series of local image slices.

[0138] The fine detection network is used to process a series of local image slices to obtain the key point locations of the CT image to be processed; the fine detection network includes a local learning module and a global learning module; the global learning module is used to capture the overall correlation within a slice and the correlation between slices.

[0139] In one possible implementation, the memory 11 may include a program storage area and a data storage area. The program storage area may store the operating system and applications required for at least one function (such as file creation or data read / write). The data storage area may store data created during use, such as initialization data.

[0140] In addition, memory 11 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device or other volatile solid-state storage device.

[0141] Communication interface 12 can be an interface for the communication module, used to connect with other devices or systems.

[0142] Of course, it should be noted that, Figure 8 The structure shown does not constitute a limitation on the medical image key point detection device in the embodiments of this application. In practical applications, the medical image key point detection device may include more than Figure 8 More or fewer components as shown, or combinations of certain components.

[0143] This application embodiment may also provide a computer-readable storage medium for storing program code for performing the steps of the above-described medical image key point detection method.

[0144] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0145] As can be seen from the above description of the embodiments, those skilled in the art can clearly understand that this application can be implemented by means of software plus necessary general-purpose hardware platforms. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in various embodiments or some parts of the embodiments of this application.

[0146] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, for system or system embodiments, since they are basically similar to method embodiments, the description is relatively simple, and relevant parts can be referred to the descriptions in the method embodiments. The systems and system embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0147] The above description is merely a preferred embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention are included within the scope of protection of the present invention.

Claims

1. A method for detecting key points in medical images, characterized in that, include: Acquire a CT image to be processed, wherein the CT image to be processed is a CT image of the joint location to be detected on the patient; The CT image to be processed is preprocessed to obtain a CT image of the joint to be detected after removing impurities; the impurities include artificial prosthesis implants or bone fragments. The key point positions of the CT image to be processed are obtained by inputting the CT image of the joint to be detected into the key point detection network model. The key point detection network model includes a coarse detection network, an image slicing module, and a fine detection network. The coarse detection network includes a personalized keypoint distribution learning module and a local bone segmentation module. The personalized keypoint distribution learning module is used to fuse general keypoint distribution information with the features of the CT image of the joint to be detected to obtain personalized keypoint distribution information. The local bone segmentation module is used to extract features from the CT image of the joint to be detected and combine them with the personalized keypoint distribution information to output a predicted mask of the local bone region containing keypoints. The image slicing module is used to slice the predicted mask of the local bone region to obtain a series of local image slices. The fine detection network is used to process a series of local image slices to obtain the key point locations of the CT image to be processed; the fine detection network includes a local learning module and a global learning module; the global learning module is used to capture the overall correlation within a slice and the correlation between slices; The personalized keypoint distribution learning module learns the general keypoint distribution information from a pre-established keypoint distribution dataset corresponding to the joint to be detected. The personalized key point distribution information is obtained by fusing the general key point distribution information with the features of the CT image of the joint to be detected after feature extraction by a convolutional neural network. The method for obtaining general keypoint distribution information includes: randomly selecting a keypoint coordinate distribution from the keypoint distribution dataset as a template; using a rigid registration method to align the remaining keypoint distributions to the approximate position of the original keypoint distribution to construct a keypoint coordinate dataset; constructing a volumetric dataset for each keypoint distribution, with the same size as the subsequently preprocessed CT joint image; and assigning values ​​to each voxel within the volumetric dataset to obtain the general keypoint distribution information, as shown in the following formula: In the formula, The voxel value representing the voxel position to be calculated. Coordinates representing voxels For the first Coordinates of key points of each joint This represents the calculation of Euclidean distance. This represents the logarithmic function with base e.

2. The medical image key point detection method according to claim 1, characterized in that, The skeletal local region prediction mask includes the skeletal region of the target range around each keypoint in the personalized keypoint distribution information.

3. The medical image key point detection method according to claim 1, characterized in that, The slicing process includes using the center of the mask region of the predicted local mask of the bone as the center of a sphere, and a radius of... Expand outwards to find the optimal solution. The formula is as follows: In the formula, For a minimum value radius, Indicates the search for optimization. The function, A function that calculates the quantity given conditions. Represents the voxel value in a 3D image; Take an outer bounding box for the spherical region, and resample the image pixels within the outer bounding box to the target size to obtain a series of local image slices.

4. The medical image key point detection method according to claim 1, characterized in that, The coarse detection network is trained using a supervised loss function to optimize network parameters. The formula for the loss function is as follows: In the formula, The calculation is the segmentation loss between the local skeleton segmentation mask of the output result and the gold standard local skeleton segmentation mask, which is composed of soft Dice loss; The calculation measures the difference between the predicted value of the predicted region and the gold standard, which is composed of MSE loss.

5. The medical image key point detection method according to claim 1, characterized in that, The network parameters are optimized during the training process of the precision detection network using a supervised loss function, the formula of which is as follows: In the formula, This represents a weighted map based on the number of iterations, where the weights for skeletal and non-skeletal regions are different, and the sum of the two is 1. and These represent the voxel values ​​of the gold standard heatmap and the predicted heatmap, respectively.

6. A medical image key point detection device, characterized in that, The apparatus for performing the medical image key point detection method according to any one of claims 1-5, the apparatus comprising: An image acquisition unit is used to acquire a CT image to be processed, wherein the CT image to be processed is a CT image of the joint location to be detected on the patient. An image preprocessing unit is used to preprocess the CT image to be processed to obtain a CT image of the joint to be detected after removing impurities; the impurities include artificial prosthesis implants or bone fragments. The key point detection unit is used to input the CT image of the joint to be detected into the key point detection network model to obtain the key point positions of the CT image to be processed. The key point detection network model includes a coarse detection network, an image slicing module, and a fine detection network. The coarse detection network includes a personalized keypoint distribution learning module and a local bone segmentation module. The personalized keypoint distribution learning module is used to fuse general keypoint distribution information with the features of the CT image of the joint to be detected to obtain personalized keypoint distribution information. The local bone segmentation module is used to extract features from the CT image of the joint to be detected and combine them with the personalized keypoint distribution information to output a predicted mask of the local bone region containing keypoints. The image slicing module is used to slice the predicted mask of the local bone region to obtain a series of local image slices. The fine detection network is used to process a series of local image slices to obtain the key point locations of the CT image to be processed; the fine detection network includes a local learning module and a global learning module; the global learning module is used to capture the overall correlation within a slice and the correlation between slices.

7. A medical image key point detection device, characterized in that, The device includes a processor and a memory: The memory is used to store program code and transmit the program code to the processor; The processor is used to execute the medical image key point detection method according to any one of claims 1-5 according to the instructions in the program code.

Images