Key point detection method and device, equipment, storage medium and program product

By introducing the topological relationship of reference key points in the detection of key points in medical image key points, combining texture features and probability map adjustment models, the accuracy and efficiency of key point detection are improved, and the problem of low accuracy in the prior art is solved.

CN118968206APending Publication Date: 2024-11-15UNITED IMAGING RES INST OF INTELLIGENT IMAGING
View PDF 0 Cites 1 Cited by

Patent Information

Application Number
CN202411204109.7
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-29
Publication Date
2024-11-15

AI Technical Summary

Technical Problem

The prior art has low accuracy in the detection of key points of medical images, and insufficient detection efficiency and accuracy, which cannot meet the requirements of high-throughput medical image analysis.

Method used

By obtaining the topological relationship between the medical image and the reference key points, input it into the preset key point detection model, using the feature recognition model, the first adjustment model and the second adjustment model, combining the texture feature information and the pixel point probability map, Identify and adjust the coordinates of key points, consider the topological relationship between key points to improve detection accuracy.

Benefits of technology

The accuracy and efficiency of key point detection are significantly improved, and the accuracy of detection results is enhanced by comprehensively considering local and global characteristics.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118968206A_ABST
    Figure CN118968206A_ABST
Patent Text Reader

Abstract

The invention relates to a key point detection method and device, equipment, a storage medium and a program product. The method comprises the following steps: acquiring a topological relation between a medical image of a target part and a reference key point in the target part; and inputting the topological relation between the medical image and the reference key points into a preset key point detection model, and identifying a plurality of key points in the medical image based on the topological relation between the reference key points to obtain coordinate information of the plurality of key points in the medical image. By adopting the method, the key point detection accuracy can be improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a key point detection method, device, equipment, storage medium and program product. Background Art

[0002] With the continuous development of artificial intelligence technology, the processing of medical images is becoming more and more intelligent.

[0003] In the related art, when performing key point detection on a medical image, doctors usually observe the medical image using their own experience and mark the positions where key points are most likely to appear as key points.

[0004] However, the related art methods have the problem of low accuracy in key point detection. Summary of the invention

[0005] Based on this, it is necessary to provide a key point detection method, device, equipment, storage medium and program product to improve the accuracy of key point detection in response to the above technical problems.

[0006] In a first aspect, the present application provides a key point detection method, the method comprising:

[0007] Acquire a topological relationship between a medical image of a target part and reference key points in the target part;

[0008] The topological relationship between the medical image and the reference key points is input into a preset key point detection model, and multiple key points in the medical image are identified based on the topological relationship between the reference key points to obtain the coordinate information of the multiple key points in the medical image.

[0009] In one embodiment, the key point detection model includes a feature recognition model, a first adjustment model, and a second adjustment model, and identifies multiple key points in the medical image based on the topological relationship between the reference key points, and obtains coordinate information of the multiple key points in the medical image, including:

[0010] The feature recognition model is used to extract the texture feature information and pixel probability map of the medical image; the probability in the pixel probability map indicates the probability that the pixel is a key point;

[0011] Analyzing the texture feature information and the pixel probability map using the first adjustment model to obtain initial coordinate information of multiple key points in the medical image;

[0012] By referring to the topological relationship between the key points and the second adjustment model, each initial coordinate information is adjusted to obtain the coordinate information of each key point.

[0013] In one embodiment, the texture feature information and the pixel probability map are analyzed using the first adjustment model to obtain initial coordinate information of multiple key points in the medical image, including:

[0014] According to the pixel probability map and the preset probability threshold, multiple key points in the medical image are obtained;

[0015] The position of each key point in the medical image and the corresponding offset value are used as the coordinate feature information of each key point;

[0016] Based on each coordinate feature information and each texture feature information, the pixel point probability map is updated, and the initial coordinate information of each key point is extracted from the updated pixel point probability map.

[0017] In one embodiment, by referring to the topological relationship between the key points and the second adjustment model, each initial coordinate information is adjusted to obtain the coordinate information of each key point, including:

[0018] Obtain the weight between any two reference key points based on the feature information of each key point;

[0019] The product of the topological relationship between any two reference key points and the corresponding weight is used as a topological relationship probability map;

[0020] The probability of each pixel is extracted from the topological relationship probability map, and the coordinate information of the pixel points greater than the preset probability threshold is used as the coordinate information of each key point.

[0021] In one embodiment, obtaining the weight between any two reference key points based on the feature information of each key point includes:

[0022] For any two reference key points, obtain the initial weights of the two reference key points;

[0023] The initial weights are adjusted using the feature information of the two reference key points to obtain the weights of the reference key points.

[0024] In one embodiment, the training process of the key point detection model includes:

[0025] Obtaining sample medical images of the target part, sample topological relationships between reference key points, and coordinate information of corresponding multiple key points;

[0026] Input the sample medical image and sample topological relationship into the initial key point detection model to obtain the predicted coordinate information of multiple key points;

[0027] The parameters of the initial key point detection model are adjusted based on the predicted coordinate information of multiple key points to obtain a key point detection model.

[0028] In a second aspect, the present application also provides a key point detection device, the device comprising:

[0029] An acquisition module, used for acquiring a medical image of a target part and a topological relationship between reference key points in the target part;

[0030] The recognition module is used to input the topological relationship between the medical image and the reference key points into a preset key point detection model, identify multiple key points in the medical image based on the topological relationship between the reference key points, and obtain the coordinate information of multiple key points in the medical image.

[0031] In a third aspect, the present application further provides a computer device, which includes a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the content of any one embodiment of the key point detection method in the first aspect is implemented.

[0032] In a fourth aspect, the present application further provides a computer-readable storage medium, wherein a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, the content of any one embodiment of the key point detection method in the first aspect is implemented.

[0033] In a fifth aspect, the present application further provides a computer program product, including a computer program, which implements the content of any one embodiment of the key point detection method in the first aspect when executed by a processor.

[0034] The above-mentioned key point detection method, device, equipment, storage medium and program product obtain the topological relationship between the medical image of the target part and the reference key points in the target part; input the topological relationship between the medical image and the reference key points into a preset key point detection model, identify multiple key points in the medical image based on the topological relationship between the reference key points, and obtain the coordinate information of multiple key points in the medical image. In the process of detecting the key points of the medical image, this method introduces the topological relationship between the reference key points. The topological relationship is the connection relationship between a reference key point and other reference key points. That is to say, in the detection process, not only the characteristic information of the key point itself is considered, but also the influence of the surrounding key points. In this way, the key point detection process considers more comprehensive features and the key point detection results obtained will be more accurate. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 A diagram showing an application environment of a key point detection method in an embodiment;

[0036] Figure 2 Schematic diagram of a key point detection method in one embodiment;

[0037] Figure 3 is a schematic diagram of a medical image in one embodiment;

[0038] Figure 4 is a schematic diagram of a topological relationship between reference key points in one embodiment;

[0039] Figure 5 Schematic diagram of a key point detection method in one embodiment;

[0040] Figure 6 is a schematic diagram of multiple key points in a medical image in one embodiment;

[0041] Figure 7 Schematic diagram of a key point detection method in one embodiment;

[0042] Figure 8 Schematic diagram of a key point detection method in one embodiment;

[0043] Fig. 9 Schematic diagram of a key point detection method in one embodiment;

[0044] Fig.10 Schematic diagram of a key point detection method in one embodiment;

[0045] Fig.11 Schematic diagram of a key point detection method in one embodiment;

[0046] Fig.12 Schematic diagram of a key point detection method in one embodiment;

[0047] Fig.13 Schematic diagram of a key point detection method in one embodiment;

[0048] Fig.14 Schematic diagram of a key point detection method in one embodiment;

[0049] Fig.15 Schematic diagram of a key point detection method in one embodiment;

[0050] Fig.16 Schematic diagram of a key point detection method in one embodiment;

[0051] Fig.17 Schematic diagram of a key point detection method in one embodiment;

[0052] Fig.18 4 is a structural block diagram of a key point detection device in an embodiment. DETAILED DESCRIPTION

[0053] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0054] Before introducing the technical solution of the present application in detail, a brief introduction to the technical background of the present application is first given.

[0055] Medical image analysis is a cross-analysis method that combines digital image processing with artificial intelligence. The analysis results of medical images can provide auxiliary reference for the clinical treatment of various diseases to improve the efficiency of clinical diagnosis.

[0056] Taking medical image analysis as an example, the key point detection of medical images is used for explanation. The key point in the medical image refers to the coordinate position with iconic features. In one scenario, the measurement of the force line of the lower limbs after total knee replacement is very critical, and the accuracy of the force line of the lower limbs is crucial to the success of the prosthesis placement. If the prosthesis is improperly positioned, it will lead to limited joint movement, and even loosening, wear and instability of the joint. Therefore, the medical image can be a lower limb image, and the key point detection of the lower limb image can measure the force line of the lower limb, which can include various mechanical parameters such as the length and angle of the lower limb. In another scenario, the change of the topological structure of the fundus blood vessels can be used for the diagnosis of retinal diseases such as proliferative diabetic retinopathy and hypertensive retinopathy. Therefore, the medical image can be a retinal image, and the key point detection of the retinal image is helpful for the registration of the vascular structure and the detection and treatment of the progression of vascular lesions. Alternatively, the medical image can also be a lumbar spine image or an ankle image, and the corresponding key point detection is lumbar spine key point detection and ankle key point detection.

[0057] In the related art, one method is that doctors use their own experience to observe medical images and mark the locations where key points are most likely to appear as key points. However, due to the complexity of medical images, this method takes a long time to detect key points, resulting in low key point detection efficiency. In addition, the accuracy of the detection results is also low during the manual detection process, which cannot meet the requirements of high-throughput medical image analysis.

[0058] In one method, you can also set the gold standard for network training by constructing a Gaussian distribution for the key point positions, output the feature heat map of the key points through the neural network, find the maximum value in the heat map, that is, the coordinates with the highest probability of the key point appearing, and use the coordinates with the highest probability as the coordinates of the key point. Alternatively, you can directly regress the key point coordinates through a full connection method, use the manually annotated key point coordinates as the gold standard, and directly output the key point coordinates through the neural network. However, both the model and regression methods focus on the positional features of a single key point, which only have local features, and do not introduce the topological structure between global key points into the neural network calculation. There is a lack of information on the connection relationship between key points, which leads to problems such as large-scale identification coordinate offsets during key point recognition, affecting the accuracy of subsequent calculations.

[0059] In view of the above problems, the present application provides a key point detection method, device, equipment, storage medium and program product, which can improve the accuracy of key point detection. Of course, the technical solution provided in the embodiments of the present application is not limited to solving only the above problems, but also has other technical effects, which can be specifically described in the following embodiments. The technical solution of the present application is described in detail below.

[0060] The key point detection method provided in the embodiment of the present application can be applied to Figure 1 In the application environment shown. The computer device in the application environment can be a server, a personal computer, a laptop computer, a smart phone, a tablet computer, a smart mobile phone, etc. The computer device may include a processor, a memory and a network interface connected via a system bus or wirelessly. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device may include a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store data in the key point detection process. The network interface of the computer device is used to communicate with an external terminal through a network connection, and the computer program is executed by the processor to implement a key point detection method.

[0061] In one embodiment, Figure 2 As shown, a key point detection method is provided, which is applied to Figure 1 The computer device in the example is used to illustrate, including the following steps:

[0062] S201, obtaining a topological relationship between a medical image of a target part and reference key points in the target part.

[0063] The target part refers to the part where the medical scan is performed, for example, the part may be the lumbar spine, lower limbs, ankles, fundus retina, etc. The medical image may be a computed tomography (CT) image, a magnetic resonance (MRI) image, a positron emission tomography (PET) image, a functional magnetic resonance (fMRI) image, etc. Figure 3 is a schematic diagram of a medical image. It can be seen from the figure that the target part corresponding to the medical image is the lower limbs.

[0064] In an embodiment of the present application, a computer device may send an image acquisition instruction to a medical device, and the image acquisition instruction carries the part of the image acquisition, that is, the target part. After receiving the image acquisition instruction, the medical device scans the target part, obtains a medical image of the target part, and sends the medical image of the target part to the computer device. Alternatively, the computer device may also obtain the medical image of the target part from a picture archiving and communication system (PACS). The embodiment of the present application does not limit the method of obtaining the medical image of the target part.

[0065] The above-mentioned reference key points refer to the standard reference points of the target part. For example, when the target part is the femur, the standard reference points refer to the standard lateral malleolus, medial malleolus of the femur and intermalleolar fossa. The topological relationship between the reference key points refers to the connection relationship between the standard lateral malleolus, medial malleolus of the femur and intermalleolar fossa. Figure 4 It is a schematic diagram of the topological relationship between reference key points. The dark pixels on the right side of the figure refer to multiple reference key points, and the left side refers to the interconnected lines between multiple reference key points, that is, the topological relationship between multiple reference points.

[0066] For example, taking the medical image as a digital radiography (DR) image and the target part as the lower limb, the DR image dataset is , where H and W represent the input size of the DR image; K is the number of key points in the DR image; the coordinates of the key points can be expressed as The size of the DR image can be 768*256*3, and the corresponding number of key points is 20, including 10 feature points each of the left and right femoral heads, greater trochanter, lateral femoral malleolus, medial femoral malleolus, intermalleolar fossa, lateral tibial malleolus, medial tibial malleolus, intermalleolar eminence, lateral ankle joint point, and medial ankle key point.

[0067] In the embodiment of the present application, the computer device can search the topological relationship between the reference key points corresponding to the identification information from the topological relationship database based on the identification information of the target part. Alternatively, the computer device can also connect the reference key points in the reference image of the target part based on prior knowledge, and use the connection relationship between the reference key points as the topological relationship.

[0068] S202, inputting the topological relationship between the medical image and the reference key points into a preset key point detection model, identifying multiple key points in the medical image based on the topological relationship between the reference key points, and obtaining coordinate information of the multiple key points in the medical image.

[0069] Among them, the key point detection model is a model used to detect key points in medical images, which can be a dynamic neural network model (DNN), a recurrent neural network model (RNN) or a convolutional neural network model (CNN), etc. The CNN network can be a residual (ResNet) series network, a CPN network, a dual propagation (SimpleBaseline) network, a Posefix network, a high-resolution (HRNet) network or a human posture estimation algorithm based on deep learning (HigherHRNet) network, etc. The key point detection model can be a model composed of one network or a model composed of multiple networks.

[0070] In an embodiment of the present application, after obtaining the topological relationship between the medical image and the key points, the computer device can use the topological relationship between the medical image and the key points as the input signal of the key point detection model and input it into the key point detection model. The key point detection model can first identify multiple key points in the medical image and determine the initial coordinate information of the multiple key points. And adjust the initial coordinate information of the multiple key points by referring to the topological relationship between the key points to obtain the coordinate information of the multiple key points in the medical image. Alternatively, the key point detection model can also extract the topological features in the topological relationship between the reference key points, and extract the features of the multiple key points in the medical image, and fuse the topological features and the features of the multiple key points. And determine the coordinate information of the multiple key points based on the fused features. The embodiment of the present application does not limit the method of identifying multiple key points in the medical image based on the topological relationship between the reference key points.

[0071] In the above key point detection method, the topological relationship between the medical image of the target part and the reference key points in the target part is obtained; the topological relationship between the medical image and the reference key points is input into a preset key point detection model, and multiple key points in the medical image are identified based on the topological relationship between the reference key points to obtain the coordinate information of the multiple key points in the medical image. In the process of detecting the key points of the medical image, this method introduces the topological relationship between the reference key points. The topological relationship is a connection relationship between a reference key point and other reference key points. That is to say, the detection process not only considers the characteristic information of the key point itself, but also considers the influence of the surrounding key points. In this way, the key point detection process considers more comprehensive features, and the key point detection results obtained will be more accurate.

[0072] The above embodiment introduces the process of key point detection by the key point detection model from an overall perspective. Assuming that the key point detection model is a large model composed of three models, and the large model includes a feature recognition model, a first adjustment model, and a second adjustment model, then, in one embodiment, Figure 5 As shown, the specific process of identifying multiple key points in the medical image based on the topological relationship between the reference key points to obtain the coordinate information of the multiple key points in the medical image includes:

[0073] S301, extracting texture feature information and a pixel probability map of a medical image using a feature recognition model; the probability in the pixel probability map represents the probability that the pixel is a key point.

[0074] Since the surface properties of different parts of the human body are different, texture feature information refers to the surface feature information of the location of a key point. The pixel probability map refers to the probability that each pixel in the medical image is a key point. For example, the probability that a pixel is a key point can be 20%, 50%, or 90%. Figure 6 This is a schematic diagram of multiple key points in a medical image. Each lower limb image contains a key point. The key points contained in different lower limb images are different. The probability value of the key points reflected in the figure is relatively large.

[0075] In an embodiment of the present application, a computer device can input a medical image into a feature recognition model, and the feature recognition model can identify the texture feature information and pixel feature information of each pixel in the medical image. According to the pixel features of each pixel, the probability of each pixel being a key point is predicted, and the probabilities of all pixels being key points are combined into a pixel probability map.

[0076] In the process of extracting texture feature information, if the feature recognition model is a convolutional neural network based on the HRNet network, then the feature extraction process includes four stages, each of which includes multiple standard convolution kernels and channel-by-channel convolution kernels. Specifically, after the medical image is input into the convolution operation of the first stage, the input features of the second stage are obtained. The input features of the second channel are convolved to obtain the input features of the third stage. Similarly, the features of the first stage, second stage, third stage and fourth stage can be obtained respectively, and the features of the four stages are fused to obtain the final texture feature vector of the key point. ,in, , represents the size of the texture feature vector obtained by convolution, d represents the number of channels of the feature vector, and R represents the texture feature set of all pixels in the medical image. Then the texture feature vector is convolved to obtain texture feature information. The texture feature information can be expressed as:

[0077]

[0078] in, represents the convolution operation, Represents the size of the texture feature vector after convolution.

[0079] S302: Analyze the texture feature information and the pixel probability map using the first adjustment model to obtain initial coordinate information of multiple key points in the medical image.

[0080] The initial coordinate information of the multiple key points refers to the coordinate information of each key point in the medical image coordinate system. The medical image coordinate system can be based on the center point of the medical image as the coordinate origin, or it can be based on the four vertices of the medical image as the coordinate origin, which is not limited in the present embodiment.

[0081] In the embodiment of the present application, since the pixel probability map only contains the coordinate feature information of the pixel, the coordinate information determined based on the coordinate feature information may be inaccurate. Therefore, it is necessary to fuse the texture feature information of each key point to obtain more accurate initial coordinate information of multiple key points.

[0082] Specifically, the computer device can input each texture feature information and the pixel probability map into the first adjustment model, and the first adjustment model can update the pixel probability map based on the texture feature information of each key point. Then, according to the updated pixel probability map, multiple key points are screened from all the pixels of the medical image, and the initial coordinate information of the multiple key points is obtained.

[0083] S303, adjusting each initial coordinate information by referring to the topological relationship between the key points and the second adjustment model to obtain the coordinate information of each key point.

[0084] In an embodiment of the present application, on the basis of determining the initial coordinate information through texture feature information and coordinate feature information, the topological relationship between multiple key points can be integrated from a global perspective. In this way, by comprehensively considering global features and local features, the coordinate information obtained will be more accurate.

[0085] Specifically, the computer device can input the topological relationship between the reference key points and the initial coordinate information of multiple key points into the second adjustment model. The second adjustment model can integrate the topological relationship between the reference key points into the key point detection process to adjust the initial coordinate information of multiple key points to obtain the coordinate information of multiple key points.

[0086] In the above key point detection method, a feature recognition model is used to extract texture feature information and pixel probability maps of medical images; the probability in the pixel probability map represents the probability that a pixel is a key point; the texture feature information and the pixel probability map are analyzed using a first adjustment model to obtain the initial coordinate information of multiple key points in the medical image; each initial coordinate information is adjusted by referring to the topological relationship between the key points and the second adjustment model to obtain the coordinate information of each key point. In the process of key point detection, this method makes two adjustments through the topological relationship between the texture feature information and the reference key points, so that the key point detection process introduces texture features and topological relationships, the feature information considered is more comprehensive and detailed, and the coordinate information of multiple key points obtained is also more accurate.

[0087] The above embodiment briefly introduces the two adjustment processes. Next, the specific processes of the first adjustment model and the second adjustment model are introduced.

[0088] First, the specific process of the first adjustment model is described. In one embodiment, Figure 7 As shown, the specific adjustment process in the above first adjustment model includes the following steps:

[0089] S401, acquiring a plurality of key points in a medical image according to a pixel probability map and a preset probability threshold.

[0090] In the embodiment of the present application, for any pixel point in the medical image, the computer device can compare the probability of the pixel point in the probability map with the preset probability threshold. If it is greater than the preset probability threshold, it means that the probability of the pixel point being a key point is relatively high, and the pixel point is taken as the key point; if it is less than or equal to the preset probability threshold, it means that the probability of the pixel point being a key point is relatively low, and the pixel point is a background point, and the pixel point is ignored. After all the pixel points are compared, the key points in the medical image can be determined.

[0091] S402: The position of each key point in the medical image and the corresponding offset value are used as coordinate feature information of each key point.

[0092] The offset value refers to the difference between the coordinate information of the key point and the center of the medical image. The interpolation includes interpolation in the X direction and interpolation in the Y direction. For example, the difference between the coordinate information of the key point and the center of the medical image can be expressed as ( ).

[0093] In an embodiment of the present application, after acquiring the key points in the medical image, the computer device can determine the coordinates of each key point in the medical image coordinate system and use the coordinates as the position of the key point in the medical image.

[0094] In addition, the computer device can obtain the position of the center point of the medical image in the medical image coordinate system. For each key point, the computer device can calculate the difference between the key point and the center point, and use the difference as the offset value between the key point and the center of the medical image. It should be emphasized that if the origin of the medical image coordinate system is the center point of the medical image, the coordinates of the key point are the offset value between the key point and the center of the medical image.

[0095] Furthermore, after the computer device obtains the position and corresponding offset value of each key point, it can construct the position information vector of the key point. For example, the position information vector of the key point can be expressed as ,in, represents the position of the key point. Then the first adjustment model performs a convolution operation on the position information vector and converts it into a dimension of The characteristic vector of is the coordinate characteristic information. The coordinate characteristic information can be expressed as:

[0096]

[0097] in, represents the convolution operation, Represents the size of the feature vector after convolution.

[0098] S403: Based on each coordinate feature information and each texture feature information, the pixel point probability map is updated, and initial coordinate information of each key point is extracted from the updated pixel point probability map.

[0099] In an embodiment of the present application, for any key point, the computer device can fuse the coordinate feature information of the key point with the corresponding texture feature information to obtain the fused feature information of the key point. And based on the fused feature information of each key point, predict the new probability of the key point. And determine the historical probability corresponding to the key point from the pixel point probability map, compare the historical probability with the new probability, if the historical probability is the same as the new probability, continue to identify the next key point. If the historical probability is different from the new probability, replace the historical probability in the pixel point probability map with the new probability, and continue to identify the next key point until all key points are replaced, and obtain an updated pixel point probability map.

[0100] Furthermore, after obtaining the updated pixel probability map, the computer device can compare the probability of each pixel in the pixel probability map with a preset probability threshold, and use the pixel points that are greater than the preset probability threshold as key points. The coordinate position of each key point in the medical image coordinate system is determined, and the coordinate position in the medical image coordinate system is used as the initial coordinate information of multiple key points.

[0101] In the above key point detection method, multiple key points in the medical image are obtained according to the pixel point probability map and the preset probability threshold; the position of each key point in the medical image and the corresponding offset value are used as the coordinate feature information of each key point; based on each coordinate feature information and each texture feature information, the pixel point probability map is updated, and the initial coordinate information of each key point is extracted from the updated pixel point probability map. This method determines the coordinate feature information of the key point by analyzing the pixel point probability map and the preset probability threshold, and comprehensively considers the coordinate feature information and the texture feature information, so that the coordinate feature information of the key point can be accurately determined, so that the pixel point probability map can be updated more accurately, so that the initial coordinate information of the key point is more accurate.

[0102] In one embodiment, Figure 8 As shown, the specific content of updating the pixel point probability map based on each coordinate feature information and each texture feature information is introduced, and the specific content includes the following steps:

[0103] S501, convert each coordinate feature information and each texture feature information into the same dimension to obtain feature information of each key point.

[0104] In the embodiment of the present application, the computer device can perform dimension conversion on each coordinate feature information and texture feature information to convert each coordinate feature information and texture feature information to the same dimension. And each coordinate feature information and corresponding texture feature information under the same dimension are merged to obtain feature information of each key point. The feature information of each key point can be expressed as:

[0105]

[0106] in, Represents the texture feature information of each key point, Represents the coordinate feature information of each key point.

[0107] That is to say, the feature information of each key point includes coordinate feature information and texture feature information. It should be noted that each coordinate feature information can be converted to the dimension of each texture feature information, or each texture feature information can be converted to the dimension of each coordinate feature information, or each coordinate feature information and each texture feature information can be converted to a preset dimension.

[0108] S502: Determine the probability that each pixel point in the medical image is a key point through each feature information.

[0109] In an embodiment of the present application, the computer device can input the feature information of each key point into the probability recognition model, analyze the feature information of each key point through the probability recognition model, and output a new pixel probability map. And determine the probability of each pixel being a key point from the new pixel probability map. Among them, the new pixel probability map can be expressed as:

[0110]

[0111] in, represents the new pixel probability map, represents the probability recognition model, Represents the parameters of the probabilistic recognition model.

[0112] S503, replacing the probability in the pixel probability map based on the probability that each pixel is a key point, to obtain an updated pixel probability map.

[0113] In an embodiment of the present application, for each pixel point, the computer device can compare the probability that the pixel point is a key point with the corresponding probability in the pixel point probability map. If the two are the same, no replacement is required; if the two are different, the probability of the pixel point is used to replace the probability of the corresponding position in the pixel point probability map to obtain an updated pixel point probability map.

[0114] In the above key point detection method, each coordinate feature information and each texture feature information are converted to the same dimension to obtain the feature information of each key point; the probability of each pixel point in the medical image being a key point is determined by each feature information; the probability in the pixel point probability map is replaced based on the probability that each pixel point is a key point to obtain an updated pixel point probability map. This method is based on the fusion result of the coordinate feature information and the texture feature information, so that the probability of each pixel point being a key point is more accurate. Then, the pixel point probability map is updated by the probability, and the updated pixel point probability map is more accurate.

[0115] Next, the specific process of the second adjustment model is described. In one embodiment, Fig. 9 As shown, the specific adjustment process in the above second adjustment model includes the following steps:

[0116] S601, obtaining the weight between any two reference key points based on the feature information of each key point.

[0117] In the embodiment of the present application, for any two key points, the computer device can use the improved sigmoid function to analyze the feature information of the two key points to determine the degree of connection between the two key points. And based on the degree of connection between the two key points, the weight between the two reference key points is determined. The weight between any two reference key points can be expressed as:

[0118]

[0119] in, represents a hyperparameter, which can be set to 100; Represents the initial weight between any two reference key points; It means the weight between two reference key points; represents the sigmoid function; K represents the number of key points; D represents the dimension of the feature vector; R represents the feature set of two reference key points.

[0120] S602: taking the product of the topological relationship between any two reference key points and the corresponding weight as a topological relationship probability map.

[0121] Among them, the topological relationship between the reference key points can be expressed as: ,in, represents K key points, represents L edges, and E represents the set of all key points and edges, including the connection relationship between two reference key points.

[0122] In the embodiment of the present application, the topological relationship between any two reference key points can be represented by an adjacency matrix, which can be represented as: The computer device can multiply the topological relationship of any two reference key points with the corresponding weights, and then perform a convolution operation on the multiplication result to obtain a topological relationship probability map. The specific process can be expressed as:

[0123]

[0124] in, Represents the topological relationship between key points.

[0125] Next, Perform multiple nonlinear convolution operations to obtain:

[0126]

[0127] in, Represents the topological relationship after multiple nonlinear convolution operations. represents the LeakyReLu nonlinear activation function, and Represents the parameters of multiple nonlinear convolution operations.

[0128] Then combine the feature information of each key point with multiple nonlinear convolution operations to obtain:

[0129]

[0130] in, The feature vector representing the combination of the feature information of each key point and multiple nonlinear convolution operations. represents the convolution operation, Represents the feature information of each key point, , It represents the size of the feature vector after the feature information of each key point is combined with multiple nonlinear convolution operations, and D represents the number of channels of the feature vector.

[0131] Finally, the above combined results are deconvolved to obtain a topological relationship probability map. This process can be expressed as:

[0132]

[0133] in, represents a topological relationship probability graph, represents the feature encoding model, Represents the parameters of the feature encoding model.

[0134] S603: extract the probability of each pixel from the topological relationship probability map, and use the coordinate information of the pixel points that are greater than a preset probability threshold as the coordinate information of each key point.

[0135] Since the topological relationship probability map is a probability map containing the connection relationship of key points, the topological relationship probability map includes the probability of each pixel in the medical image, and the probability of the topological relationship probability map corresponds to each pixel in the medical image. Therefore, the computer device can directly extract the probability of each pixel from the topological relationship probability map.

[0136] In an embodiment of the present application, the computer device can compare the probability of each pixel point in the topological relationship probability map with a preset probability threshold. If it is greater than the preset probability threshold, the coordinate information of the pixel points greater than the preset probability threshold is obtained, and the coordinate information is used as the coordinate information of each key point; if it is less than or equal to the preset probability threshold, the pixel points less than or equal to the preset probability threshold are ignored.

[0137] In the above key point detection method, the topological relationship between the reference key points is analyzed by the second adjustment model to obtain a topological relationship probability map; the probability of each pixel point is extracted from the topological relationship probability map; the coordinate information of the pixel points greater than the preset probability threshold is used as the coordinate information of each key point. This method introduces the topological relationship between the reference key points in the process of determining the probability, so that the characteristics of the key points and the influence of the surrounding key points can be integrated, and the accuracy of the obtained topological relationship probability map is higher, and the coordinate information of each key point obtained through the topological probability map is also more accurate.

[0138] In the process of topological relationship analysis, weights can also be introduced for analysis. Fig.10 As shown, the specific content of introducing weights for analysis is introduced next.

[0139] S701, for any two reference key points, obtaining initial weights of the two reference key points.

[0140] In an embodiment of the present application, the computer device may receive the weight annotations of the multiple reference key points by the user, and use the annotated weights as the initial weights of the multiple reference key points in the medical image. Alternatively, the computer device may also input the multiple key points in the medical image into the weight determination model, analyze the multiple key points through the weight determination model, and determine the initial weight between any two reference key points.

[0141] S702, adjusting the initial weights using feature information of the two reference key points to obtain the weights of the reference key points.

[0142] The feature information of two key points indicates the degree of connection between the two key points. For example, the closer the textures of the two key points are, the closer the connection between the two key points is; and the smaller the difference in coordinates of the two key points is, the closer the connection between the two key points is. The closer the connection between the two key points is, the higher the weight value of the reference key point is, and the initial weight needs to be increased; the looser the connection between the two key points is, the lower the weight value of the reference key point is, and the initial weight needs to be decreased.

[0143] In the embodiment of the present application, after obtaining the initial weights of any two reference key points, the computer device can predict the weight between the two reference key points based on the closeness of the connection between the two reference key points to obtain the predicted weights of the two reference key points. And based on the weight difference between the predicted weight and the initial weight, the initial weight is adjusted to obtain the weight of the reference key point.

[0144] In the above key point detection method, for any two reference key points, the initial weights of the two reference key points are obtained; the initial weights are adjusted using the feature information of the two reference key points to obtain the weights of the reference key points. For any two reference key points, this method uses the feature information of the two reference key points to adjust the initial weights of the two reference key points to accurately obtain the weights of the reference key points.

[0145] All of the above embodiments are introductions to the actual application process of the key point detection model. Next, the training process of the key point detection model will be introduced through several embodiments.

[0146] In one embodiment, Fig.11 As shown in Figure 1, the training process of the key point detection model includes:

[0147] S801, obtaining a sample medical image of a target part, a sample topological relationship between reference key points, and coordinate information of a plurality of corresponding key points.

[0148] In the embodiment of the present application, the computer device can obtain a sample medical image with the same identification information as the target part from the PACS, and obtain coordinate information corresponding to the sample medical image. In addition, the computer device can search for a sample topological relationship between reference key points corresponding to the identification information from a topological relationship database based on the identification information of the target part.

[0149] S802, inputting the sample medical image and the sample topological relationship into the initial key point detection model to obtain the predicted coordinate information of multiple key points.

[0150] In an embodiment of the present application, a computer device can simultaneously input a sample medical image and a sample topological relationship into an initial key point detection model. The initial key point detection model can identify the coordinate information of multiple key points in the sample medical image based on the sample topological relationship, and use the coordinate information of the identified multiple key points as predicted coordinate information of the multiple key points.

[0151] S803, adjusting the parameters of the initial key point detection model based on the predicted coordinate information of multiple key points until the error between the predicted key point coordinate information and the corresponding coordinate information is less than a preset error threshold, thereby obtaining a key point detection model.

[0152] In an embodiment of the present application, after obtaining the predicted coordinate information of multiple key points, the computer device can obtain the error value between the predicted coordinate information of the multiple key points and the corresponding coordinate information, and compare the error value with the preset error threshold. If the error value is greater than the preset error threshold, it means that the predicted coordinate information is inaccurate, and the model parameters need to be adjusted, and then the initial key point detection model after parameter adjustment is used to continue prediction to obtain new predicted coordinate information. And continue to compare the error value between the new predicted coordinate information and the corresponding coordinate information with the preset probability threshold. If the error value is still greater than the preset error threshold, continue to adjust the model parameters. If the error value is less than the preset error threshold, it means that the training process is completed and a trained key point detection model is obtained.

[0153] It should be noted that in the training process of the key point detection model, the objective function is constructed by the sample texture feature information and the sample topological relationship. The objective function L can be expressed as:

[0154]

[0155] in, represents a hyperparameter, which can be set to 0.01. represents the mean square error loss function, BCE represents the binary cross entropy loss function; K represents the number of key points; Represents the pixel probability map of the predicted key points; Pixel probability map representing the key points of the gold standard; Indicates the predicted sample topological relationship; Represents the sample topological relationship of the gold standard; Represents the loss function of key points; The loss function represents the topological relationship of the samples. In the above training process, the initial learning rate can be set to 0.001, the number of training times can be set to 200, and the number of samples for one training is 3.

[0156] In the above key point detection method, a sample medical image of the target part, a sample topological relationship between reference key points and the coordinate information of the corresponding multiple key points are obtained; the sample medical image and the sample topological relationship are input into the initial key point detection model to obtain the predicted coordinate information of the multiple key points; the parameters of the initial key point detection model are adjusted based on the predicted coordinate information of the multiple key points until the error between the predicted key point coordinate information and the corresponding coordinate information is less than the preset error threshold, thereby obtaining the key point detection model. In the process of training the initial key point detection model, the method simultaneously trains the sample medical image and the sample topological relationship, that is, during the training process, the model can not only learn the local features of the key points, but also learn the global features affected by other key points. The model is trained by combining local features and global features, so that the trained key point detection model is more accurate, and the key point detection is performed by the trained key point detection model, and the detection results obtained will also be more accurate.

[0157] In one embodiment, Fig.12 As shown, the specific contents of inputting the sample medical image and the sample topological relationship into the initial key point detection model to obtain the predicted coordinate information of multiple key points include the following steps:

[0158] S901, extracting features of a sample medical image using an initial feature recognition model to obtain sample texture feature information and a sample pixel probability map of a plurality of key points in the sample medical image.

[0159] In an embodiment of the present application, a computer device can input a sample medical image into an initial feature recognition model, and the sample texture feature information and sample pixel feature information of each pixel in the sample medical image can be identified through the initial feature recognition model. According to the sample pixel features of each pixel, the probability of each pixel being a key point is predicted, and the probabilities of all pixels being key points are combined into a sample pixel probability map. The pixels with higher probabilities in the sample pixel probability map are taken as key points, and the sample texture feature information of multiple key points is determined from the sample texture feature information of all pixels.

[0160] S902: Input the texture feature information of each sample and the probability map of the sample pixel points into the first initial adjustment model to obtain the initial predicted coordinate information of each key point.

[0161] In the embodiment of the present application, the computer device can input each sample texture feature information and the sample pixel probability map into the first initial adjustment model, and the first initial adjustment model can update the sample pixel probability map based on the sample texture feature information of each key point. Afterwards, based on the updated sample pixel probability map, multiple key points are screened out from all the pixels of the sample medical image, and the initial predicted coordinate information of the multiple key points is obtained.

[0162] S903: Based on the sample topological relationship between the reference key points and the second initial adjustment model, the initial predicted coordinate information is adjusted to obtain predicted coordinate information of each key point.

[0163] Specifically, the computer device can input the sample topological relationship between the reference key points and the initial coordinate information of multiple key points into the second initial adjustment model. The second initial adjustment model can integrate the sample topological relationship between the reference key points into the training process of the second adjustment model to adjust the initial coordinate information of multiple key points to obtain the predicted coordinate information of multiple key points.

[0164] In the above key point detection method, the features of the sample medical image are extracted using the initial feature recognition model to obtain sample texture feature information and sample pixel probability maps of multiple key points in the sample medical image; each sample texture feature information and sample pixel probability map is input into the first initial adjustment model to obtain the initial predicted coordinate information of each key point; based on the sample topological relationship between the reference key points and the second initial adjustment model, the initial predicted coordinate information is adjusted to obtain the predicted coordinate information of each key point. In the process of model training, this method makes two adjustments through the sample texture feature information and the sample topological relationship between the reference key points, so that the model training process introduces the sample texture features and sample topological relationship, the sample feature information considered is more comprehensive and detailed, and the predicted coordinate information obtained is more accurate.

[0165] The above embodiment involves the training process of two initial adjustment models. Next, the training process of the first initial adjustment model and the training process of the second initial adjustment model are respectively described.

[0166] First, the training process of the first initial adjustment model is described. In one embodiment, Fig.13 As shown, the training process of the first initial adjustment model may include the following steps:

[0167] S1001, obtaining sample coordinate feature information of each key point from the sample pixel probability map.

[0168] In the embodiment of the present application, the computer device can compare the probability of each pixel in the sample pixel probability map with a preset probability threshold, and use the pixel points with a probability greater than the preset probability threshold as key points. The position of each key point is analyzed to determine the sample coordinate feature information of each key point.

[0169] S1002, based on the fusion result of each sample's texture feature information and each sample's coordinate feature information, update the sample pixel probability map.

[0170] In an embodiment of the present application, for any key point, the computer device can fuse the sample coordinate feature information of the key point with the corresponding sample texture feature information to obtain the fused sample feature information of the key point. And based on the fused sample feature information of each key point, predict the new probability of the key point. And determine the historical probability corresponding to the key point from the sample pixel point probability map, compare the historical probability with the new probability, and if the historical probability is the same as the new probability, continue to identify the next key point. If the historical probability is different from the new probability, replace the historical probability in the sample pixel point probability map with the new probability, and continue to identify the next key point until all key points are replaced, and obtain the updated sample pixel point probability map.

[0171] S1003: extracting initial predicted coordinate information of each key point from the updated sample pixel probability map.

[0172] In the embodiment of the present application, after obtaining the updated sample pixel probability map, the computer device can compare the probability of each pixel in the sample pixel probability map with the preset probability threshold, and use the pixel points greater than the preset probability threshold as key points. And determine the coordinate position of each key point in the medical image coordinate system, and use the coordinate position in the medical image coordinate system as the initial predicted coordinate information of multiple key points.

[0173] In the above key point detection method, the sample coordinate feature information of each key point is obtained from the sample pixel probability map; based on the fusion result of each sample texture feature information and each sample coordinate feature information, the sample pixel probability map is updated. The initial predicted coordinate information of each key point is extracted from the updated sample pixel probability map. This method analyzes the pixel probability map to determine the coordinate feature information of the key point, and comprehensively considers the coordinate feature information and texture feature information, so that the pixel probability map can be accurately updated, so that the initial predicted coordinate information of the key point is more accurate.

[0174] Next, the training process of the second initial adjustment model is described. In one embodiment, Fig.14As shown, the training process of the second initial adjustment model may include the following steps:

[0175] S1101, multiply the weights of any two reference key points by the corresponding sample topological relationship to obtain a sample topological relationship probability map.

[0176] In an embodiment of the present application, the computer device may multiply the topological relationship between any two reference key points by the corresponding weights, and then perform a convolution operation on the multiplication result to obtain a sample topological relationship probability map.

[0177] S1102, extracting the probability of each pixel from the sample topological relationship probability map.

[0178] Since the sample topology relationship probability map is a probability map containing the connection relationship of key points in the sample medical image, the sample topology relationship probability map includes the probability of each pixel in the sample medical image, and the probability of the sample topology relationship probability map corresponds to each pixel in the sample medical image. Therefore, the computer device can directly extract the probability of each pixel from the sample topology relationship probability map.

[0179] S1103: taking the coordinate information of the pixel points whose probability is greater than a preset probability threshold as the predicted coordinate information of each key point.

[0180] In an embodiment of the present application, the computer device can compare the probability of each pixel point in the sample topological relationship probability map with a preset probability threshold. If it is greater than the preset probability threshold, the coordinate information of the pixel points greater than the preset probability threshold is obtained, and the coordinate information is used as the predicted coordinate information of each key point; if it is less than or equal to the preset probability threshold, the pixel points less than or equal to the preset probability threshold are ignored.

[0181] In the above key point detection method, the weights of any two reference key points are multiplied by the corresponding sample topological relationship to obtain a sample topological relationship probability map; the probability of each pixel point is extracted from the sample topological relationship probability map; the coordinate information of the pixel points greater than the preset probability threshold is used as the predicted coordinate information of each key point. This method introduces the sample topological relationship between the reference key points in the process of determining the probability, so that the characteristics of the key points and the influence of the surrounding key points can be integrated, and the obtained sample topological relationship probability map has higher accuracy, and the predicted coordinate information of each key point obtained through the sample topological probability map is also more accurate.

[0182] Fig.15 : is a flowchart of a key point detection method, the key point detection process includes the following steps:

[0183] S1201: Extracting texture feature information and pixel probability map of the medical image using a feature recognition model;

[0184] See also Fig.16 The process of extracting features in the key point feature extraction model.

[0185] S1202: Acquire multiple key points in the medical image according to the pixel probability map and a preset probability threshold;

[0186] S1203: taking the position of each key point in the medical image and the corresponding offset value as the coordinate feature information of each key point;

[0187] See also Fig.16 The process from pixel probability map to coordinate feature information.

[0188] S1204: Based on each coordinate feature information and each texture feature information, the pixel point probability map is updated, and initial coordinate information of each key point is extracted from the updated pixel point probability map;

[0189] See also Fig.16 The process of determining initial coordinate information based on texture feature information and coordinate feature information.

[0190] S1205: Obtaining the weight between any two reference key points based on the feature information of each key point;

[0191] S1206: taking the product of the topological relationship between any two reference key points and the corresponding weights as a topological relationship probability map;

[0192] See also Fig.16 The process from texture feature information and weights to topological relationship probability map.

[0193] S1207: Extract the probability of each pixel from the topological relationship probability map, and use the coordinate information of the pixel points that are greater than a preset probability threshold as the coordinate information of each key point.

[0194] Fig.16 A flow chart showing a key point detection method, the method comprising: performing feature recognition on a medical image using a feature recognition model to obtain texture feature information and a pixel probability map; determining coordinate feature information of the medical image based on the pixel probability map; determining initial coordinate information based on the texture feature information and the coordinate feature information; generating a topological relationship probability map based on the texture feature information, weights, and initial coordinate information; and determining coordinate information of multiple key points in the medical image based on the topological relationship map.

[0195] Fig.17 The flowchart of the key point detection method is shown in the figure. It can be seen from the figure that the target part in the medical image is the lower limbs. The feature recognition model is used to perform feature recognition on the medical image containing the lower limbs to obtain feature information Ec , the characteristic information E c Including coordinate feature information b e and texture feature information f e . The coordinate feature information b e and texture feature information f e Input into the key point feature extractor to obtain the initial coordinate information of the key point. Then use the coordinate feature information b e and texture feature information f e Weights between reference key points in the topological relationship graph Adjust and use the graph convolutional feature extractor The adjusted topological relationship graph is subjected to feature extraction to obtain a topological relationship probability graph. Then, the initial coordinate information is adjusted according to the topological relationship graph to obtain the coordinate information of multiple key points in the medical image.

[0196] It should be understood that, although the steps in the flowcharts involved in the above embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps is not strictly limited in order, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above embodiments may include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0197] Based on the same inventive concept, the embodiment of the present application also provides a key point detection device for implementing the key point detection method involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in one or more key point detection device embodiments provided below can refer to the limitations on the key point detection method above, and will not be repeated here.

[0198] In one embodiment, Fig.18 As shown, a key point detection device is provided, including: an acquisition module 11 and a recognition module 12, wherein:

[0199] An acquisition module 11 is used to acquire a topological relationship between a medical image of a target part and a reference key point in the target part;

[0200] The recognition module 12 is used to input the topological relationship between the medical image and the reference key points into a preset key point detection model, recognize multiple key points in the medical image based on the topological relationship between the reference key points, and obtain the coordinate information of the multiple key points in the medical image.

[0201] In one embodiment, the identification module 12 includes: a feature extraction unit, an information acquisition unit and an information adjustment unit, wherein:

[0202] A feature extraction unit, used to extract texture feature information and a pixel probability map of a medical image using a feature recognition model; the probability in the pixel probability map indicates the probability that the pixel is a key point;

[0203] An information acquisition unit, used to analyze the texture feature information and the pixel probability map using the first adjustment model to acquire initial coordinate information of multiple key points in the medical image;

[0204] The information adjustment unit is used to adjust each initial coordinate information by referring to the topological relationship between the key points and the second adjustment model to obtain the coordinate information of each key point.

[0205] In one embodiment, the above-mentioned information acquisition unit is also used to obtain multiple key points in the medical image based on the pixel point probability map and a preset probability threshold; the position of each key point in the medical image and the corresponding offset value are used as the coordinate feature information of each key point; based on each coordinate feature information and each texture feature information, the pixel point probability map is updated, and the initial coordinate information of each key point is extracted from the updated pixel point probability map.

[0206] In one embodiment, the above-mentioned information adjustment unit is also used to obtain the weight between any two reference key points based on the feature information of each key point; the product of the topological relationship between any two reference key points and the corresponding weight is used as a topological relationship probability map; the probability of each pixel point is extracted from the topological relationship probability map, and the coordinate information of the pixel points greater than the preset probability threshold is used as the coordinate information of each key point.

[0207] In one embodiment, the information adjustment unit is further used to obtain initial weights of two reference key points for any two reference key points; and adjust the initial weights using feature information of the two reference key points to obtain weights of the reference key points.

[0208] In one embodiment, the key point detection device further includes: a sample acquisition module, a coordinate prediction module and a parameter adjustment module, wherein:

[0209] A sample acquisition module is used to acquire sample medical images of target parts, sample topological relationships between reference key points, and coordinate information of corresponding multiple key points;

[0210] A coordinate prediction module is used to input the sample medical image and the sample topological relationship into the initial key point detection model to obtain the predicted coordinate information of multiple key points;

[0211] The parameter adjustment module is used to adjust the parameters of the initial key point detection model based on the predicted coordinate information of multiple key points until the error between the predicted key point coordinate information and the corresponding coordinate information is less than a preset error threshold, thereby obtaining a key point detection model.

[0212] Each module in the above key point detection device can be implemented in whole or in part by software, hardware and a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute operations corresponding to each module above.

[0213] In one embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, the content of any one embodiment of the above-mentioned key point detection method is implemented.

[0214] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the content of any embodiment of the above-mentioned key point detection method is implemented.

[0215] In one embodiment, a computer program product is provided, including a computer program, which implements the content of any one embodiment of the above-mentioned key point detection method when executed by a processor.

[0216] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties.

[0217] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but are not limited to this.

[0218] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0219] The above embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A key point detection method, It is characterized in that The method comprises: Acquire a topological relationship between a medical image of a target part and reference key points in the target part; The topological relationship between the medical image and the reference key points is input into a preset key point detection model, and multiple key points in the medical image are identified based on the topological relationship between the reference key points to obtain coordinate information of the multiple key points in the medical image.

2. The method according to claim 1, It is characterized in that The key point detection model includes a feature recognition model, a first adjustment model and a second adjustment model. The identifying multiple key points in the medical image based on the topological relationship between the reference key points and obtaining coordinate information of the multiple key points in the medical image includes: The feature recognition model is used to extract texture feature information and a pixel probability map of the medical image; the probability in the pixel probability map represents the probability that the pixel is a key point; Analyzing the texture feature information and the pixel probability map using the first adjustment model to obtain initial coordinate information of a plurality of key points in the medical image; The initial coordinate information is adjusted through the topological relationship between the reference key points and the second adjustment model to obtain the coordinate information of each key point.

3. The method according to claim 2, It is characterized in that The using the first adjustment model to analyze the texture feature information and the pixel probability map to obtain initial coordinate information of a plurality of key points in the medical image includes: Acquire a plurality of key points in the medical image according to the pixel probability map and a preset probability threshold; Taking the position of each key point in the medical image and the corresponding offset value as the coordinate feature information of each key point; Based on each of the coordinate feature information and each of the texture feature information, the pixel point probability map is updated, and the initial coordinate information of each of the key points is extracted from the updated pixel point probability map.

4. The method according to claim 2 or 3, It is characterized in that The adjusting each of the initial coordinate information by using the topological relationship between the reference key points and the second adjustment model to obtain the coordinate information of each of the key points includes: Obtaining a weight between any two reference key points based on the feature information of each of the key points; The product of the topological relationship between any two reference key points and the corresponding weight is used as a topological relationship probability map; The probability of each pixel point is extracted from the topological relationship probability map, and the coordinate information of the pixel points greater than a preset probability threshold is used as the coordinate information of each key point.

5. The method according to claim 4, It is characterized in that The obtaining the weight between any two reference key points based on the feature information of each key point includes: For any two reference key points, obtaining initial weights of the two reference key points; The initial weight is adjusted using the feature information of the two reference key points to obtain the weight of the reference key point.

6. The method according to any one of claims 1 to 3, It is characterized in that The training process of the key point detection model includes: Obtaining sample medical images of the target part, sample topological relationships between reference key points, and coordinate information of corresponding multiple key points; Inputting the sample medical image and the sample topological relationship into an initial key point detection model to obtain predicted coordinate information of the multiple key points; The parameters of the initial key point detection model are adjusted based on the predicted coordinate information of the multiple key points to obtain the key point detection model.

7. A key point detection device, It is characterized in that The device comprises: An acquisition module, used to acquire a topological relationship between a medical image of a target part and a reference key point in the target part; An identification module is used to input the topological relationship between the medical image and the reference key points into a preset key point detection model, identify multiple key points in the medical image based on the topological relationship between the reference key points, and obtain coordinate information of the multiple key points in the medical image.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program. It is characterized in that When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, It is characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer program product comprising a computer program, It is characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Citation Information

Cited By

  • Multi-modal retina fundus image registration method and system

    CN119228861A