Postoperative lower limb key point detection method and device, electronic equipment and storage medium

Abstract

The application relates to the field of computer technology, and provides a postoperative lower limb key point detection method and device, electronic equipment and a storage medium, which comprise the following steps: acquiring a postoperative lower limb image to be detected currently and a preoperative lower limb image corresponding to the postoperative lower limb image; the postoperative lower limb image is an image taken after a knee replacement surgery on a lower limb; the preoperative lower limb image is associated with corresponding preoperative lower limb key point information; based on the postoperative lower limb image, key points in a prosthesis region of the postoperative lower limb image are detected to obtain prosthesis region key point information; based on the preoperative lower limb image and the prosthesis region key point information, key points in a non-prosthesis region of the postoperative lower limb image are detected to obtain non-prosthesis region key point information. The method provided in the embodiment of the application can realize accurate detection of postoperative lower limb X-ray image key points without establishing an additional deep learning model, and can improve the model deployment efficiency of a lower limb skeleton intelligent measurement related system.

Description

Technical Field

[0001] This invention relates to the field of computer technology applications, and in particular to a method, device, electronic device, and storage medium for detecting key points in the lower limbs after surgery. Background Technology

[0002] Lower limb alignment assessment is a core technology for orthopedic surgical planning and efficacy verification, and its measurement accuracy directly affects the clinical prognosis of surgeries such as joint replacement and osteotomy. Currently, in clinical practice, physicians need to manually annotate key anatomical landmarks such as the femoral mechanical axis positioning point and the tibial plateau geometric center on X-ray images to calculate the biomechanical parameters of the hip-knee-ankle joint. However, postoperative images commonly exhibit implant artifacts, skeletal morphological distortions, and blurred soft tissue layers, significantly weakening the reliability and consistency of manual annotation. With the deepening application of artificial intelligence technology in medical image analysis, while existing intelligent measurement methods can achieve automatic key point detection on X-ray images, they are forced to adopt independent model development strategies due to domain differences between preoperative physiological structures and complex postoperative features. The development processes for preoperative and postoperative models are usually very similar, involving a lot of repetitive work. This situation also leads to high redundancy in algorithm modules, restricting the efficiency of clinical deployment. There is an urgent need to construct a postoperative key point detection method that avoids repetitive modeling, eliminating model redundancy throughout the preoperative and postoperative processes in intelligent assistance systems while ensuring measurement accuracy, thereby improving development efficiency.

[0003] In recent years, keypoint detection technology based on deep learning has been gradually applied to the field of medical image analysis, and automatic detection methods for lower limb keypoints have begun to emerge, such as patent documents 1 (CN115345928A) and 2 (CN114782449A). Patent document 1 discloses a two-stage cascaded keypoint detection model, which uses coarse detection for initial localization followed by fine detection for refined localization, to achieve the detection of keypoints and calculation of mechanical measurement parameters in medical images. Patent document 2 mainly discloses a method and system for extracting keypoints from lower limb X-ray images. This method specifically uses a binary Gaussian model to generate directional heatmaps for edge points and utilizes a channel feature complementarity module to model the correlation between feature channels, enhancing image features and thus improving the localization accuracy of keypoints in lower limb X-ray images. However, these detection methods are all for keypoint detection in lower limb X-ray images without prostheses before surgery, and the focus is mainly on improving detection accuracy. There is a lack of a postoperative lower limb keypoint recognition method that can avoid the need for repeated model development. Summary of the Invention

[0004] To address the aforementioned technical problems, the technical solution adopted by this invention is as follows:

[0005] According to a first aspect of the present invention, a method for detecting key points in the lower limb after surgery is provided, the method comprising the following steps:

[0006] S100, acquire the postoperative lower limb image to be detected and the corresponding preoperative lower limb image; the postoperative lower limb image is an image taken after knee replacement surgery of the lower limb, and the preoperative lower limb image is associated with the corresponding preoperative lower limb key point information.

[0007] S200, based on the postoperative lower limb image, detect the key points of the prosthesis region in the postoperative lower limb image to obtain the key point information of the prosthesis region.

[0008] S300, based on the preoperative lower limb image and the key point information of the prosthesis region, detect the key points of the non-prosthesis region in the postoperative lower limb image to obtain the key point information of the non-prosthesis region.

[0009] S400, the key point information of the prosthesis region and the key point information of the non-prosthesis region are used as the key point information of the lower limb corresponding to the postoperative lower limb image that needs to be detected.

[0010] Optionally, the key point information of the prosthesis region includes the coordinates of the first key point to the sixth key point, wherein the first key point is the apex of the intercondylar fossa of the femur, the second key point is the lowest point of the lateral femoral condyle, the third key point is the lowest point of the medial femoral condyle, the fourth key point is the midpoint of the intercondylar ridge of the tibia, the fifth key point is the lowest point of the lateral tibial plateau, and the sixth key point is the lowest point of the medial tibial plateau.

[0011] S200 specifically includes:

[0012] S201, the prosthesis region in the postoperative lower limb image is segmented using an image segmentation algorithm to obtain the prosthesis region.

[0013] S202, the contours of the upper and lower regions of the prosthesis region are extracted using an image contour extraction method to obtain the upper region contour and the lower region contour.

[0014] S203, based on the upper region contour, obtain the coordinates of the first to third key points in the key point information of the prosthesis region, and based on the lower region contour, obtain the coordinates of the fourth to sixth key points in the key point information of the prosthesis region.

[0015] Optionally, in S203, the step of obtaining the coordinates of the first to third key points in the key point information of the prosthesis region based on the upper region contour specifically includes:

[0016] S2031, Sort the coordinates of all pixels corresponding to the upper region contour in ascending order of their ordinates to obtain the sorted upper coordinate set.

[0017] S2032, take the last k1 pixel coordinates of the sorted upper coordinate set as the first upper coordinate set.

[0018] S2033, sort the pixel coordinates in the first coordinate set in ascending order of their horizontal coordinates to obtain the second upper coordinate set.

[0019] S2034 divides the second upper coordinate set into three sub-coordinate sets: the first upper sub-coordinate set, the second upper sub-coordinate set, and the third upper sub-coordinate set.

[0020] S2035, take the pixel coordinates corresponding to the minimum ordinate value in the second sub-coordinate set as the coordinates of the first key point.

[0021] S2036, using the x-coordinate of the first key point as the dividing point, the second upper coordinate set is divided into two sub-coordinate sets, namely the fourth upper sub-coordinate set and the fifth upper sub-coordinate set, wherein the maximum x and y coordinate values ​​in the fourth upper sub-coordinate set are less than the minimum x-coordinate value in the fifth upper sub-coordinate set.

[0022] S2037, the pixel coordinates corresponding to the maximum ordinate value in the fourth sub-coordinate set are used as the coordinates of the second key point, and the pixel coordinates corresponding to the maximum ordinate value in the fifth sub-coordinate set are used as the coordinates of the third key point.

[0023] Optionally, in S203, the step of obtaining the coordinates of the fourth to sixth key points in the key point information of the prosthesis region based on the lower region contour specifically includes:

[0024] S10, sort all pixel coordinates corresponding to the lower region contour in ascending order of ordinate to obtain the sorted lower sorted coordinate set.

[0025] S11, take the coordinates of the first k2 pixels in the sorted lower coordinate set as the first lower coordinate set.

[0026] S12, sort the pixel coordinates in the first lower coordinate set in ascending order of their horizontal coordinates to obtain the second lower coordinate set.

[0027] S13 divides the second lower coordinate set into three sub-coordinate sets: the first lower coordinate set, the second lower sub-coordinate set, and the third lower sub-coordinate set.

[0028] S14, take the pixel coordinates corresponding to the maximum ordinate value in the second sub-coordinate set as the coordinates of the fourth key point.

[0029] S15, using the x-coordinate of the fourth key point as the dividing point, the second lower coordinate set is divided into two sub-coordinate sets, namely the fourth lower sub-coordinate set and the fifth lower sub-coordinate set. The maximum x and y coordinate values ​​in the fourth lower sub-coordinate set are less than the minimum x-coordinate value in the fifth lower sub-coordinate set.

[0030] S16, take the pixel coordinates corresponding to the maximum ordinate value in the fourth sub-coordinate set as the coordinates of the fifth key point, and take the pixel coordinates corresponding to the maximum ordinate value in the fifth sub-coordinate set as the coordinates of the sixth key point.

[0031] Optionally, the S300 specifically includes:

[0032] S301, select any key point in the region corresponding to the prosthesis area in the preoperative lower limb image as a reference point.

[0033] S302, set counter i = 1.

[0034] S303, if i≤n, execute S304, otherwise execute S309; ​​n is the number of key points in the non-spoof region.

[0035] S304, Obtain the relative distance d between the reference point and the i-th non-prosthetic region key point in the preoperative lower limb image. i .

[0036] S305, based on d i Obtain the initial position P0 of the i-th non-prosthetic region key point in the postoperative lower limb image. i .

[0037] S306, with P0 i Construct a defined region centered on a given region, and uniformly sample m sampling points within the defined region. Then, crop an image region with a width of w and a height of h centered on each sampling point to obtain m image regions.

[0038] S307, obtain the feature matching degree between each image region and the reference image region respectively, and obtain m feature matching degrees; wherein, the reference image region is an image region with width w and height h cropped with the i-th non-prosthetic region key point in the preoperative lower limb image as the center.

[0039] S308, take the sampling point corresponding to the maximum feature matching degree among m feature matching degrees as the final position of the i-th non-prosthetic region key point after surgery, and add the final position of the i-th non-prosthetic region key point to the current non-prosthetic region key point set; set i = i + 1, execute S303; the initial value of the current non-prosthetic region key point set is empty.

[0040] S309, obtain the key point information of the non-spoofed region based on the current set of key points in the non-spoofed region.

[0041] Optionally, the designated area is square.

[0042] According to a second aspect of the present invention, a postoperative lower limb key point detection device is provided, the device comprising:

[0043] The image acquisition module is used to acquire the postoperative lower limb image to be detected and the corresponding preoperative lower limb image; the postoperative lower limb image is an image taken after knee replacement surgery of the lower limb, and the preoperative lower limb image is associated with the corresponding preoperative lower limb key point information.

[0044] The prosthesis region key point acquisition module is used to detect key points in the prosthesis region of the postoperative lower limb image based on the postoperative lower limb image, and obtain key point information of the prosthesis region.

[0045] A non-prosthetic region key point acquisition module is used to detect key points in the non-prosthetic region of the postoperative lower limb image based on the preoperative lower limb image and the key point information of the prosthetic region, to obtain key point information of the non-prosthetic region. According to a third aspect of the present invention, an electronic device is provided, including a processor and a memory; the processor executes the steps of the method described in the first aspect of the present invention by calling a program or instruction stored in the memory.

[0046] According to a fourth aspect of the present invention, a computer-readable storage medium is provided that stores a program or instructions that cause a computer to perform the steps of the method described in the first aspect of the present invention.

[0047] The present invention has at least the following beneficial effects:

[0048] The postoperative lower limb key point detection method provided in this invention includes: acquiring a postoperative lower limb image to be detected and a corresponding preoperative lower limb image; the postoperative lower limb image is an image taken after knee replacement surgery, and the preoperative lower limb image is associated with corresponding preoperative lower limb key point information; based on the postoperative lower limb image, key points in the prosthesis region of the postoperative lower limb image are detected to obtain prosthesis region key point information; based on the preoperative lower limb image and the prosthesis region key point information, key points in the non-prosthesis region of the postoperative lower limb image are detected to obtain non-prosthesis region key point information. The method provided in this invention can achieve accurate detection of key points in postoperative lower limb X-ray images without the need for additional deep learning models, thus improving the model deployment efficiency of lower limb bone intelligent measurement systems.

[0049] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0050] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0051] Figure 1 A flowchart of a postoperative lower limb key point detection method provided in an embodiment of the present invention;

[0052] Figure 2 A schematic diagram of key point detection in the prosthesis area;

[0053] Figure 3 a is a schematic diagram of the lower limb before surgery;

[0054] Figure 3 b is a schematic diagram of the lower limb image after surgery. Detailed Implementation

[0055] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0056] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. The terminology used herein in the description of this invention is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. The term "and / or" as used herein includes any and all combinations of one or more of the associated listed items.

[0057] It should be noted that some exemplary embodiments are described as processes or methods depicted as flowcharts. Although the flowcharts describe the steps as sequential processes, many of these steps can be performed in parallel, concurrently, or simultaneously. Furthermore, the order of the steps can be rearranged. A process can be terminated when its operation is complete, but it may also have additional steps not included in the figures. A process can correspond to a method, function, procedure, subroutine, subroutine, etc.

[0058] To address the technical challenge of requiring independent modeling of preoperative physiological structures and postoperative implant interference for detecting key points in the lower limbs, this invention provides a method for accurate detection of key points in postoperative lower limb X-ray images without the need for additional deep learning models. Figure 1 As shown, the method may include the following steps:

[0059] S100, acquire the postoperative lower limb image to be detected and the corresponding preoperative lower limb image; the postoperative lower limb image is an image taken after total knee replacement surgery of the lower limb, and the preoperative lower limb image is associated with the corresponding preoperative lower limb key point information, that is, the lower limb key points in the preoperative lower limb image have been detected.

[0060] In this embodiment of the invention, the postoperative lower limb image to be detected can be an X-ray image. The preoperative lower limb image corresponding to the postoperative lower limb image is an image taken of the same subject before undergoing total knee replacement surgery. The subject is a patient who needs to undergo total knee replacement surgery.

[0061] In this embodiment of the invention, lower limb key points are locations within the lower limb that can be used to measure the force line and related angles of the lower limb. The specific location and number of key points can be set based on actual needs, i.e., based on the application purpose of the image. In one illustrative embodiment, lower limb key points may include key points in the knee joint region and key points other than those in the knee joint region. Specifically, key points in the knee joint region may include the lowest point of the lateral femoral condyle, the apex of the intercondylar fossa of the femur, the lowest point of the medial femoral condyle, the lowest point of the lateral tibial plateau, the lowest point of the medial tibial plateau, and the midpoint of the intercondylar crest of the tibia. Key points other than those in the knee joint region may include the center of the femoral head and the tip of the greater trochanter in the hip joint region, and the lateral and medial malleoli in the ankle joint region, etc.

[0062] S200, based on the postoperative lower limb image, detect the key points of the prosthesis region in the postoperative lower limb image to obtain key point information of the prosthesis region.

[0063] In this embodiment of the invention, key points in the postoperative lower limb images may include key points in the prosthesis region and key points in the non-prosthesis region. Key points in the prosthesis region correspond to key points in the knee joint region in the preoperative lower limb images, and key points in the non-prosthesis region correspond to key points in the preoperative lower limb images excluding the knee joint region. The prosthesis region is a specific space formed after the knee joint prosthesis is surgically implanted, and it completely corresponds to the original anatomical location of the knee joint.

[0064] S300: Based on the preoperative lower limb image and the key point information of the prosthesis region, the key points of the non-prosthesis region in the postoperative lower limb image are detected to obtain the key point information of the non-prosthesis region.

[0065] S400, the key point information of the prosthesis region and the key point information of the non-prosthesis region are used as the key point information of the lower limb corresponding to the postoperative lower limb image that needs to be detected.

[0066] The postoperative lower limb key point detection method provided in this invention can achieve accurate detection of key points in postoperative lower limb X-ray images without the need to build an additional deep learning model, which can improve the model deployment efficiency of the lower limb bone intelligent measurement system.

[0067] Furthermore, in this embodiment of the invention, the key point information of the prosthesis region may include the coordinates of the first key point to the sixth key point.

[0068] Furthermore, such as Figure 2 As shown, S200 may specifically include:

[0069] S201, the prosthesis region in the postoperative lower limb image is segmented using an image segmentation algorithm to obtain the prosthesis region.

[0070] In this embodiment of the invention, the image segmentation algorithm can be any existing image segmentation algorithm. In one illustrative embodiment, for example, it can be a segmentation algorithm such as adaptive thresholding, watershed algorithm, and OTSU algorithm.

[0071] As those skilled in the art will know, any method that uses an image segmentation algorithm to segment the prosthesis region in the postoperative lower limb image to obtain the prosthesis region falls within the protection scope of this invention.

[0072] In this embodiment of the invention, postoperative lower limb images can be as follows: Figure 2 As shown in M1, the segmented prosthesis region can be as follows: Figure 2 As shown in M2.

[0073] S202, the contours of the upper and lower regions of the prosthesis region are extracted using an image contour extraction method to obtain the upper region contour and the lower region contour.

[0074] In this embodiment of the invention, the upper region of the prosthesis area refers to the region connected to the thigh, and the lower region refers to the region connected to the calf.

[0075] In this embodiment of the invention, the image contour extraction method can be any existing image contour extraction method. In an illustrative embodiment, for example, it can be the Sobel operator, the Canny operator, the Laplacian operator, etc.

[0076] As those skilled in the art will know, any method that uses image contour extraction to extract the contours of the upper and lower regions of the prosthesis region to obtain the upper and lower region contours is within the protection scope of this invention.

[0077] As those skilled in the art will know, the upper and lower region outlines are represented using corresponding pixel coordinates.

[0078] In this embodiment of the invention, the upper region contour and the lower region contour can be respectively as follows: Figure 2 As shown in M3 and M4.

[0079] S203, based on the upper region contour, obtain the coordinates of the first to third key points in the key point information of the prosthesis region, and based on the lower region contour, obtain the coordinates of the fourth to sixth key points in the key point information of the prosthesis region.

[0080] In this embodiment of the invention, the coordinate system of the pixel can be a rectangular coordinate system constructed with the top left corner of the image as the origin o, the downward direction as the y-axis, and the upward direction as the x-axis.

[0081] Furthermore, such as Figure 2 As shown, in S203, the step of obtaining the coordinates of the first to third key points in the key point information of the prosthesis region based on the upper region contour specifically includes:

[0082] S2031, Sort the coordinates of all pixels corresponding to the upper region contour in ascending order of their ordinates to obtain the sorted upper coordinate set.

[0083] S2032, take the last k1 pixel coordinates of the sorted upper coordinate set as the first upper coordinate set.

[0084] In this embodiment of the invention, k1 = f1 × N1, where f1 is a preset coefficient, and N1 is the number of pixel coordinates corresponding to the upper region contour, 0 < f1 < 1. f1 can be an empirical value, or a value that makes the first upper coordinate set include the vertex of the intercondylar fossa of the femur, and can be obtained through historical data. For example, f1 can be obtained through the following steps:

[0085] (1) Obtain Q sample images; the sample images are X-ray images of the lower limbs with key points of the lower limbs marked.

[0086] (2) The ordinate values ​​of the upper region contour corresponding to each sample image are sorted in ascending order to obtain the sorted ordinate values ​​of the sample image.

[0087] (3) Obtain the sorting position value of the ordinate value of the intercondylar fossa of the femur in each sample image in the corresponding sorting ordinate value.

[0088] (4) k1 is determined based on the sorting position value of the ordinate value of the apex of the intercondylar fossa of the femur in Q sample images.

[0089] In one illustrative embodiment, f1 = (P1 / M1 + P2 / M2 + ... + PQ / MQ) / Q, where PQ is the sorting position value of the ordinate of the intercondylar fossa apex of the femur in the Qth sample image, and MQ is the number of pixel coordinates of the upper region contour of the Qth sample image. In another illustrative embodiment, f1 = min(P1 / M1 + P2 / M2 + ... + PQ / MQ) / Q. In a specific embodiment, f1 = 1 / 2.

[0090] S2033, sort the pixel coordinates in the first coordinate set in ascending order of their horizontal coordinates to obtain the second upper coordinate set.

[0091] S2034 divides the second upper coordinate set into three sub-coordinate sets: the first upper sub-coordinate set, the second upper sub-coordinate set, and the third upper sub-coordinate set.

[0092] In this embodiment of the invention, the second upper coordinate set can be divided into three sub-coordinate sets: the first third of the coordinates in the second upper coordinate set is used as the first upper sub-coordinate set, the middle third of the coordinates is used as the second upper sub-coordinate set, and the last third of the coordinates is used as the third upper sub-coordinate set.

[0093] S2035, take the pixel coordinates corresponding to the minimum ordinate value in the second sub-coordinate set as the coordinates of the first key point.

[0094] S2036, using the x-coordinate of the first key point as the dividing point, the second upper coordinate set is divided into two sub-coordinate sets, namely the fourth upper sub-coordinate set and the fifth upper sub-coordinate set, wherein the maximum x and y coordinate values ​​in the fourth upper sub-coordinate set are less than the minimum x-coordinate value in the fifth upper sub-coordinate set.

[0095] S2037, the pixel coordinates corresponding to the maximum ordinate value in the fourth sub-coordinate set are used as the coordinates of the second key point, and the pixel coordinates corresponding to the maximum ordinate value in the fifth sub-coordinate set are used as the coordinates of the third key point.

[0096] Furthermore, in S203, the step of obtaining the coordinates of the fourth to sixth key points in the key point information of the prosthesis region based on the lower region contour specifically includes:

[0097] S10, sort all pixel coordinates corresponding to the lower region contour in ascending order of ordinate to obtain the sorted lower sorted coordinate set.

[0098] S11, take the coordinates of the first k2 pixels in the sorted lower coordinate set as the first lower coordinate set.

[0099] In this embodiment of the invention, k2 = f2 × N2, where f2 is a preset coefficient and N2 is the number of pixel coordinates corresponding to the lower region contour, 0 < f2 < 1. F2 can be an empirical value, which is the value that makes the first lower coordinate set include the lowest point of the lateral femoral condyle and the lowest point of the medial femoral condyle. Specifically, it can be obtained through historical data, and the specific acquisition method can be similar to f1. In a specific embodiment, f2 = 1 / 2.

[0100] S12, sort the pixel coordinates in the first lower coordinate set in ascending order of their horizontal coordinates to obtain the second lower coordinate set.

[0101] S13, divide the second lower coordinate set into three equal sub-coordinate sets, namely the first lower coordinate set, the second lower sub-coordinate set, and the third lower sub-coordinate set;

[0102] S14, take the pixel coordinates corresponding to the maximum ordinate value in the second sub-coordinate set as the coordinates of the fourth key point.

[0103] S15, using the x-coordinate of the fourth key point as the dividing point, the second lower coordinate set is divided into two sub-coordinate sets, namely the fourth lower sub-coordinate set and the fifth lower sub-coordinate set. The maximum x and y coordinate values ​​in the fourth lower sub-coordinate set are less than the minimum x-coordinate value in the fifth lower sub-coordinate set.

[0104] S16, take the pixel coordinates corresponding to the maximum ordinate value in the fourth sub-coordinate set as the coordinates of the fifth key point, and take the pixel coordinates corresponding to the maximum ordinate value in the fifth sub-coordinate set as the coordinates of the sixth key point.

[0105] like Figure 2 As shown, the first to sixth keypoints obtained can be respectively as follows: Figure 2 As shown in the numbers 1 to 6.

[0106] Furthermore, the S300 may specifically include:

[0107] S301, select any key point in the region corresponding to the prosthesis area in the preoperative lower limb image as a reference point.

[0108] In this embodiment of the invention, the area corresponding to the prosthesis area in the preoperative lower limb image is the knee joint area. The reference point can be selected based on actual needs; for example, a first key point can be selected as the reference point.

[0109] S302, set counter i = 1.

[0110] S303, if i≤n, execute S304, otherwise execute S309; ​​n is the number of key points in the non-spoof region.

[0111] S304, Obtain the relative distance d between the reference point and the i-th non-prosthetic region key point in the preoperative lower limb image. i .

[0112] In an embodiment of the present invention, d i This includes the difference in abscissa and ordinate between the reference point and the i-th non-prosthetic region key point in the preoperative lower limb image, i.e., d. i =(△x) i , △y i ), △x i Let Δy be the difference in abscissa between the reference point and the i-th non-prosthetic key point in the preoperative lower limb image. i The difference in ordinate between the reference point and the i-th non-prosthetic key point in the preoperative lower limb image is denoted as . Figure 3 a shows the positional relationship between a reference point and a key point NKP in a non-prosthetic region in a preoperative lower limb image.

[0113] S305, based on d i Obtain the initial position P0 of the i-th non-prosthetic region key point in the postoperative lower limb image. i .

[0114] In this embodiment of the invention, P0 i Based on the coordinate information of key points corresponding to the reference point in the postoperative lower limb images and d i Determining the coordinates of key points in the postoperative lower limb images, specifically those differing from the reference point by a coordinate d. i The pixels are used as P0 i , specific, P0 i x0 i =xc + △x i P0 i The ordinate y0 i =yc+△y i xc and yc are the x and y coordinates of the key points corresponding to the reference point in the postoperative lower limb image, respectively. Figure 3 b shows the initial location of a non-prosthetic region key point NKP in a postoperative lower limb image.

[0115] S306, with P0 i Construct a defined region centered on a given region, and uniformly sample m sampling points within the defined region. Then, crop an image region with a width of w and a height of h centered on each sampling point to obtain m image regions.

[0116] In this embodiment of the invention, the designated area can be square, preferably rectangular. The size of the designated area can be set according to actual needs, as long as the constructed designated area is located on the lower limb contour line.

[0117] In this embodiment of the invention, w and h can be set based on actual needs, as long as the structural features of the corresponding region can be clearly displayed in the image area. w and h can be specifically determined based on the required analysis accuracy and computing resources. Generally, the higher the required analysis accuracy, the smaller w and h should be. If more computing resources are needed, w and h can be set to a smaller value.

[0118] S307, obtain the feature matching degree between each image region and the reference image region respectively, and obtain m feature matching degrees; wherein, the reference image region is an image region with width w and height h cropped with the i-th non-prosthetic region key point in the preoperative lower limb image as the center.

[0119] In this embodiment of the invention, the feature matching degree can be obtained based on existing feature matching methods, such as cosine similarity, structural similarity calculation methods, etc., preferably structural similarity.

[0120] S308, take the sampling point corresponding to the maximum feature matching degree among m feature matching degrees as the final position of the i-th non-prosthetic region key point after surgery, and add the final position of the i-th non-prosthetic region key point to the current non-prosthetic region key point set; set i = i + 1, execute S303; the initial value of the current non-prosthetic region key point set is empty.

[0121] S309, obtain the key point information of the non-spoofing region based on the current set of key points in the non-spoofing region, that is, obtain the coordinate information of each key point in the non-spoofing region.

[0122] Based on the same inventive concept, embodiments of the present invention also provide a postoperative lower limb key point detection device, the device comprising:

[0123] The image acquisition module is used to acquire the postoperative lower limb image to be detected and the corresponding preoperative lower limb image; the postoperative lower limb image is an image taken after knee replacement surgery of the lower limb, and the preoperative lower limb image is associated with the corresponding preoperative lower limb key point information.

[0124] The prosthesis region key point acquisition module is used to detect key points in the prosthesis region of the postoperative lower limb image based on the postoperative lower limb image, and obtain key point information of the prosthesis region.

[0125] The non-prosthesis region key point acquisition module is used to detect key points in the non-prosthesis region of the postoperative lower limb image based on the preoperative lower limb image and the key point information of the prosthesis region, and obtain the key point information of the non-prosthesis region.

[0126] Furthermore, the key point information of the prosthesis region includes the coordinates of the first key point to the sixth key point, wherein the first key point is the apex of the intercondylar fossa of the femur, the second key point is the lowest point of the lateral femoral condyle, the third key point is the lowest point of the medial femoral condyle, the fourth key point is the midpoint of the intercondylar crest of the tibia, the fifth key point is the lowest point of the lateral tibial plateau, and the sixth key point is the lowest point of the medial tibial plateau.

[0127] The key point acquisition module for the prosthesis region specifically includes:

[0128] The prosthesis region segmentation unit is used to segment the prosthesis region in the postoperative lower limb image using an image segmentation algorithm to obtain the prosthesis region.

[0129] The contour extraction unit is used to extract the contours of the upper and lower regions of the prosthesis region using an image contour extraction method, thereby obtaining the upper region contour and the lower region contour.

[0130] The prosthesis region key point acquisition unit is used to acquire the coordinates of the first to third key points in the prosthesis region key point information based on the upper region contour, and to acquire the coordinates of the fourth to sixth key points in the prosthesis region key point information based on the lower region contour. This device can be used to perform... Figure 1 The method shown in the illustrated embodiment is relevant here; therefore, the functions that each functional module of the device can achieve can be referred to. Figure 1 The embodiments shown are described in detail below.

[0131] This invention also provides an electronic device, including: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being configured to perform the method described in this invention.

[0132] This invention also provides a computer-readable storage medium storing computer-executable instructions for performing the methods described in this invention.

[0133] It should be understood that the various forms of processes shown above can be used to reorder, add, or delete steps. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this invention can be achieved, and this is not limited herein.

[0134] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A method for detecting key points in the lower limb after surgery, characterized in that, The method includes the following steps: S100, acquire the postoperative lower limb image to be detected and the corresponding preoperative lower limb image; the postoperative lower limb image is an image taken after knee replacement surgery of the lower limb, and the preoperative lower limb image is associated with the corresponding preoperative lower limb key point information. S200, based on the postoperative lower limb image, detect the key points of the prosthesis region in the postoperative lower limb image to obtain key point information of the prosthesis region; S300, based on the preoperative lower limb image and the key point information of the prosthesis region, detect the key points of the non-prosthesis region in the postoperative lower limb image to obtain the key point information of the non-prosthesis region; S400, the key point information of the prosthesis region and the key point information of the non-prosthesis region are used as the key point information of the lower limb corresponding to the postoperative lower limb image that needs to be detected; The S300 specifically includes: S301, Select any key point in the region corresponding to the prosthesis area in the preoperative lower limb image as a reference point; S302, set counter i=1; S303, if i≤n, execute S304; otherwise, execute S309; ​​n is the number of key points in the non-spoof region. S304, Obtain the relative distance d between the reference point and the i-th non-prosthetic region key point in the preoperative lower limb image. i ; S305, based on d i Obtain the initial position P0 of the i-th non-prosthetic region key point in the postoperative lower limb image. i ; S306, with P0 i Construct a defined region centered on the image, and uniformly sample m sampling points within the defined region. Then, crop an image region with a width of w and a height of h centered on each sampling point to obtain m image regions. S307, obtain the feature matching degree between each image region and the reference image region respectively, and obtain m feature matching degrees; wherein, the reference image region is an image region with width w and height h cropped with the i-th non-prosthetic region key point in the preoperative lower limb image as the center; S308, take the sampling point corresponding to the maximum feature matching degree among m feature matching degrees as the final position of the i-th non-prosthetic region key point after surgery, and add the final position of the i-th non-prosthetic region key point to the current non-prosthetic region key point set; set i=i+1, execute S303; the initial value of the current non-prosthetic region key point set is empty; S309, obtain the key point information of the non-spoofed region based on the current set of key points in the non-spoofed region.

2. The method according to claim 1, characterized in that, The key point information of the prosthesis region includes the coordinates of the first key point to the sixth key point, wherein the first key point is the apex of the intercondylar fossa of the femur, the second key point is the lowest point of the lateral femoral condyle, the third key point is the lowest point of the medial femoral condyle, the fourth key point is the midpoint of the intercondylar crest of the tibia, the fifth key point is the lowest point of the lateral tibial plateau, and the sixth key point is the lowest point of the medial tibial plateau. S200 specifically includes: S201, The prosthesis region in the postoperative lower limb image is segmented using an image segmentation algorithm to obtain the prosthesis region; S202, the contours of the upper and lower regions of the prosthesis region are extracted using an image contour extraction method to obtain the upper region contour and the lower region contour. S203, based on the upper region contour, obtain the coordinates of the first to third key points in the key point information of the prosthesis region, and based on the lower region contour, obtain the coordinates of the fourth to sixth key points in the key point information of the prosthesis region.

3. The method according to claim 2, characterized in that, In S203, the step of obtaining the coordinates of the first to third key points in the key point information of the prosthesis region based on the upper region contour specifically includes: S2031, Sort the coordinates of all pixels corresponding to the upper region contour in ascending order of their ordinates to obtain the sorted upper coordinate set. S2032, take the last k1 pixel coordinates of the sorted upper coordinate set as the first upper coordinate set; S2033, sort the pixel coordinates in the first coordinate set in ascending order of their horizontal coordinates to obtain the second upper coordinate set; S2034, the second upper coordinate set is divided into three sub-coordinate sets, namely the first upper sub-coordinate set, the second upper sub-coordinate set, and the third upper sub-coordinate set; S2035, take the pixel coordinates corresponding to the minimum ordinate value in the second upper sub-coordinate set as the coordinates of the first key point; S2036, the second upper coordinate set is divided into two sub-coordinate sets, namely the fourth upper sub-coordinate set and the fifth upper sub-coordinate set, with the x-coordinate of the first key point as the dividing point. The maximum x and y coordinate values ​​in the fourth upper sub-coordinate set are less than the minimum x-coordinate value in the fifth upper sub-coordinate set. S2037, the pixel coordinates corresponding to the maximum ordinate value in the fourth sub-coordinate set are used as the coordinates of the second key point, and the pixel coordinates corresponding to the maximum ordinate value in the fifth sub-coordinate set are used as the coordinates of the third key point.

4. The method according to claim 2, characterized in that, In S203, the step of obtaining the coordinates of the fourth to sixth key points in the key point information of the prosthesis region based on the lower region contour specifically includes: S10, sort all pixel coordinates corresponding to the lower region contour in ascending order of ordinate to obtain the sorted lower sorted coordinate set. S11, take the coordinates of the first k2 pixels in the sorted lower coordinate set as the first lower coordinate set; S12, sort the pixel coordinates in the first lower coordinate set in ascending order of their horizontal coordinates to obtain the second lower coordinate set; S13, divide the second lower coordinate set into 3 sub-coordinate sets, namely the first lower coordinate set, the second lower sub-coordinate set and the third lower sub-coordinate set; S14, take the pixel coordinates corresponding to the maximum ordinate value in the second sub-coordinate set as the coordinates of the fourth key point; S15, the second lower coordinate set is divided into two sub-coordinate sets with the x-coordinate of the fourth key point as the dividing point. These are the fourth lower sub-coordinate set and the fifth lower sub-coordinate set. The maximum x and y coordinate values ​​in the fourth lower sub-coordinate set are less than the minimum x-coordinate value in the fifth lower sub-coordinate set. S16, take the pixel coordinates corresponding to the maximum ordinate value in the fourth sub-coordinate set as the coordinates of the fifth key point, and take the pixel coordinates corresponding to the maximum ordinate value in the fifth sub-coordinate set as the coordinates of the sixth key point.

5. The method according to claim 1, characterized in that, The designated area is square.

6. A postoperative lower limb key point detection device, characterized in that, The device includes: The image acquisition module is used to acquire the postoperative lower limb image that needs to be detected and the corresponding preoperative lower limb image; the postoperative lower limb image is an image taken after knee replacement surgery on the lower limb, and the preoperative lower limb image is associated with the corresponding preoperative lower limb key point information. The prosthesis region key point acquisition module is used to detect key points in the prosthesis region in the postoperative lower limb image based on the postoperative lower limb image, and obtain key point information of the prosthesis region; The non-prosthetic region key point acquisition module is used to detect key points in the non-prosthetic region of the postoperative lower limb image based on the preoperative lower limb image and the key point information of the prosthetic region, and obtain the key point information of the non-prosthetic region. The key point acquisition module for the prosthesis region is specifically used to perform the following operations: S301, Select any key point in the region corresponding to the prosthesis area in the preoperative lower limb image as a reference point; S302, set counter i=1; S303, if i≤n, execute S304; otherwise, execute S309; ​​n is the number of key points in the non-spoof region. S304, Obtain the relative distance d between the reference point and the i-th non-prosthetic region key point in the preoperative lower limb image. i ; S305, based on d i Obtain the initial position P0 of the i-th non-prosthetic region key point in the postoperative lower limb image. i ; S306, with P0 i Construct a defined region centered on the image, and uniformly sample m sampling points within the defined region. Then, crop an image region with a width of w and a height of h centered on each sampling point to obtain m image regions. S307, obtain the feature matching degree between each image region and the reference image region respectively, and obtain m feature matching degrees; wherein, the reference image region is an image region with width w and height h cropped with the i-th non-prosthetic region key point in the preoperative lower limb image as the center; S308, take the sampling point corresponding to the maximum feature matching degree among m feature matching degrees as the final position of the i-th non-prosthetic region key point after surgery, and add the final position of the i-th non-prosthetic region key point to the current non-prosthetic region key point set; set i=i+1, execute S303; the initial value of the current non-prosthetic region key point set is empty; S309, obtain the key point information of the non-spoofed region based on the current set of key points in the non-spoofed region.

7. The apparatus according to claim 6, characterized in that, The key point information of the prosthesis region includes the coordinates of the first key point to the sixth key point, wherein the first key point is the apex of the intercondylar fossa of the femur, the second key point is the lowest point of the lateral femoral condyle, the third key point is the lowest point of the medial femoral condyle, the fourth key point is the midpoint of the intercondylar crest of the tibia, the fifth key point is the lowest point of the lateral tibial plateau, and the sixth key point is the lowest point of the medial tibial plateau. The key point acquisition module for the prosthesis region specifically includes: The prosthesis region segmentation unit is used to segment the prosthesis region in the postoperative lower limb image using an image segmentation algorithm to obtain the prosthesis region; The contour extraction unit is used to extract the contours of the upper and lower regions of the prosthesis region using an image contour extraction method, so as to obtain the upper region contour and the lower region contour. The prosthesis region key point acquisition unit is used to acquire the coordinates of the first to third key points in the key point information of the prosthesis region based on the upper region contour, and to acquire the coordinates of the fourth to sixth key points in the key point information of the prosthesis region based on the lower region contour.

8. An electronic device, characterized in that, Including processor and memory; The processor executes the steps of the method as described in any one of claims 1 to 5 by invoking programs or instructions stored in the memory.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium is used to store a program or instructions that cause a computer to perform the steps of the method as described in any one of claims 1 to 5.

Images