Postoperative lower limb key point detection method and device, electronic equipment and storage medium

Abstract

The invention relates to the field of computer technology application, and provides a postoperative lower limb key point detection method and device, electronic equipment and a storage medium, and the method comprises the steps: obtaining a postoperative lower limb image needing to be detected at present and a preoperative lower limb image corresponding to the postoperative lower limb image; the post-operation lower limb image is an image shot after a knee joint replacement operation is carried out on the lower limb, and the pre-operation lower limb image is associated with the corresponding pre-operation lower limb key point information; based on the postoperative lower limb image, key points of a prosthesis area in the postoperative lower limb image are detected, and prosthesis area key point information is obtained; based on the preoperative lower limb image and the key point information of the prosthesis area, key points of a non-prosthesis area in the postoperative lower limb image are detected, and key point information of the non-prosthesis area is obtained. According to the method provided by the embodiment of the invention, accurate detection of the key points of the post-operation lower limb X-ray image can be realized without additionally establishing a deep learning model, and the model deployment efficiency of a lower limb skeleton intelligent measurement and calculation related system can be improved.

Description

Technical Field

[0001] The present invention relates to the field of computer technology applications, and in particular to a method, device, electronic equipment and storage medium for detecting key points of lower limbs after surgery. Background Art

[0002] Lower limb alignment assessment is a core technology for orthopedic surgery planning and efficacy verification. Its measurement accuracy directly impacts clinical outcomes in procedures such as joint replacements and osteotomies. In current clinical practice, physicians must manually annotate key anatomical landmarks, such as the femoral mechanical axis and the tibial plateau geometric center, on X-ray images to calculate biomechanical parameters of the hip, knee, and ankle joints. However, widespread issues such as implant artifacts, skeletal morphological distortions, and blurred soft tissue layers in postoperative images significantly impair the reliability and consistency of manual annotation. With the increasing application of artificial intelligence in medical image analysis, existing intelligent measurement methods, while capable of automatically detecting key points in X-ray images, are forced to employ independent model development strategies due to the domain differences between preoperative physiological structures and postoperative complex features. The development processes for preoperative and postoperative models are often very similar, with significant repetitive work. This also leads to high redundancy in algorithmic modules, hindering the efficiency of clinical deployment. There is an urgent need to develop a postoperative key point detection method that avoids duplicate modeling while ensuring measurement accuracy and eliminating model redundancy throughout the preoperative and postoperative processes in intelligent assistance systems to improve development efficiency.

[0003] In recent years, deep learning-based keypoint detection technology has been gradually applied to the field of medical image analysis, and automatic detection methods for lower limb keypoints have also begun to emerge. For example, Patent Document 1 (CN115345928A) and Patent Document 2 (CN114782449A) are available. Patent Document 1 discloses a two-stage cascade keypoint detection model that uses a coarse detection, initial positioning, and fine detection for refined positioning to detect keypoints in medical images and calculate mechanical measurement parameters. Patent Document 2 mainly discloses a method and system for extracting keypoints from lower limb X-ray images. This method specifically generates directional heatmaps for edge points based on binary Gaussian algorithms and utilizes a channel feature complementation module to model inter-channel correlations, enhancing image features and thereby improving the localization accuracy of keypoints in lower limb X-ray images. However, these detection methods are all targeted at detecting keypoints in preoperative lower limb X-ray images without prostheses, and most focus on improving detection accuracy. There is a lack of a postoperative lower limb keypoint recognition method that avoids the need for repetitive model development. Summary of the Invention

[0004] In view of the above technical problems, the technical solution adopted by the present invention is:

[0005] According to a first aspect of the present invention, a method for detecting key points of lower limbs after surgery is provided, the method comprising the following steps:

[0006] S100, obtaining the postoperative lower limb image currently required to be detected and the preoperative lower limb image corresponding to the postoperative lower limb image; the postoperative lower limb image is an image taken after the knee replacement surgery on the lower limb, and the preoperative lower limb image is associated with the corresponding preoperative lower limb key point information.

[0007] S200 , based on the post-operative lower limb image, detecting key points of the prosthesis region in the post-operative lower limb image to obtain key point information of the prosthesis region.

[0008] S300 , based on the preoperative lower limb image and the prosthesis area key point information, detecting key points of the non-prosthesis area in the postoperative lower limb image to obtain the non-prosthesis area key point information.

[0009] S400: Using the prosthesis region key point information and the non-prosthesis region key point information as lower limb key point information corresponding to a post-operative lower limb image currently to be detected.

[0010] Optionally, the key point information of the prosthesis area includes the coordinates of the first key point to the sixth key point, wherein the first key point is the apex of the femoral intercondylar fossa, the second key point is the lowest point of the lateral femoral condyle, the third key point is the lowest point of the medial femoral condyle, the fourth key point is the midpoint of the tibial intercondylar eminence, the fifth key point is the lowest point of the lateral tibial plateau, and the sixth key point is the lowest point of the medial tibial plateau.

[0011] S200 specifically includes:

[0012] S201 , using an image segmentation algorithm to segment the prosthesis region in the post-operative lower limb image to obtain the prosthesis region.

[0013] S202 : Extracting the contours of the upper region and the lower region of the prosthesis region using an image contour extraction method to obtain an upper region contour and a lower region contour.

[0014] S203 , obtaining coordinates of the first to third key points in the prosthesis region key point information based on the upper region contour, and obtaining coordinates of the fourth to sixth key points in the prosthesis region key point information based on the lower region contour.

[0015] Optionally, in S203, the step of obtaining the coordinates of the first key point to the third key point in the key point information of the prosthesis region based on the upper region contour specifically includes:

[0016] S2031 , sorting the coordinates of all pixel points corresponding to the upper area outline in ascending order of the vertical coordinate to obtain a sorted upper sorted coordinate set.

[0017] S2032: Use the coordinates of the last k1 pixels in the sorted upper sorted coordinate set as the first upper coordinate set.

[0018] S2033 , sorting the pixel coordinates in the first coordinate set in ascending order of the horizontal coordinates to obtain a second upper coordinate set.

[0019] S2034: Divide the second upper coordinate set into three sub-coordinate sets, namely a first upper sub-coordinate set, a second upper sub-coordinate set, and a third upper sub-coordinate set.

[0020] S2035 , taking the pixel coordinates corresponding to the minimum vertical coordinate value in the second upper sub-coordinate set as the coordinates of the first key point.

[0021] S2036: Divide the second upper coordinate set into two sub-coordinate sets using the horizontal coordinate of the first key point as a dividing point, namely a fourth upper sub-coordinate set and a fifth upper sub-coordinate set, wherein the maximum horizontal and vertical coordinate values ​​in the fourth upper sub-coordinate set are smaller than the minimum horizontal coordinate value in the fifth upper sub-coordinate set.

[0022] S2037 , using the pixel coordinates corresponding to the maximum vertical coordinate value in the fourth upper sub-coordinate set as the coordinates of the second key point, and using the pixel coordinates corresponding to the maximum vertical coordinate value in the fifth upper sub-coordinate set as the coordinates of the third key point.

[0023] Optionally, in S203, the step of acquiring the coordinates of the fourth to sixth key points in the key point information of the prosthesis region based on the lower region contour specifically includes:

[0024] S10 , sorting the coordinates of all pixel points corresponding to the lower area outline in ascending order of the vertical coordinate to obtain a sorted lower sorted coordinate set.

[0025] S11 , taking the first k2 pixel coordinates in the sorted lower sorted coordinate set as the first lower coordinate set.

[0026] S12 , sorting the pixel coordinates in the first lower coordinate set in ascending order of the horizontal coordinates to obtain a second lower coordinate set.

[0027] S13: Divide the second lower coordinate set into three sub-coordinate sets, namely the first sub-coordinate set, the second lower sub-coordinate set and the third lower sub-coordinate set.

[0028] S14: Taking the coordinates of the pixel point corresponding to the maximum vertical coordinate value in the second lower sub-coordinate set as the coordinates of the fourth key point.

[0029] S15, dividing the second lower coordinate set into two sub-coordinate sets using the horizontal coordinate of the fourth key point as a dividing point, namely a fourth lower sub-coordinate set and a fifth lower sub-coordinate set, wherein the maximum horizontal and vertical coordinate values ​​in the fourth lower sub-coordinate set are less than the minimum horizontal coordinate value in the fifth lower sub-coordinate set.

[0030] S16, using the pixel coordinates corresponding to the maximum vertical coordinate value in the fourth sub-coordinate set as the coordinates of the fifth key point, and using the pixel coordinates corresponding to the maximum vertical coordinate value in the fifth sub-coordinate set as the coordinates of the sixth key point.

[0031] Optionally, S300 specifically includes:

[0032] S301 , selecting any key point in the area corresponding to the prosthesis area in the preoperative lower limb image as a reference point.

[0033] S302, set counter i=1.

[0034] S303, if i≤n, execute S304, otherwise, execute S309; ​​n is the number of key points in the non-prosthesis area.

[0035] S304, obtaining the relative distance d between the reference point and the key point of the i-th non-prosthesis area in the preoperative lower limb image. i .

[0036] S305, based on d i , get the initial position P0 of the key point of the i-th non-prosthesis area in the postoperative lower limb image i .

[0037] S306, with P0 i A set area is constructed as the center, and m sampling points are uniformly sampled in the set area. An image area with a width of w and a height of h is intercepted with each sampling point as the center to obtain m image areas.

[0038] S307, respectively obtain the feature matching degree between each image region and the reference image region, and obtain m feature matching degrees; wherein the reference image region is an image region with a width w and a height h cut with the i-th non-prosthesis region key point in the preoperative lower limb image as the center.

[0039] S308, take the sampling point corresponding to the maximum feature matching degree among the m feature matching degrees as the final position of the i-th non-prosthesis area key point after surgery, and add the final position of the i-th non-prosthesis area key point to the current non-prosthesis area key point set; set i=i+1, execute S303; the initial value of the current non-prosthesis area key point set is empty.

[0040] S309 : Obtaining the non-prosthesis area key point information based on the current non-prosthesis area key point set.

[0041] Optionally, the set area is square.

[0042] According to a second aspect of the present invention, a device for detecting key points of lower limbs after surgery is provided, the device comprising:

[0043] The image acquisition module is used to obtain the postoperative lower limb image that needs to be detected and the preoperative lower limb image corresponding to the postoperative lower limb image; the postoperative lower limb image is an image taken after the lower limb undergoes knee replacement surgery, and the preoperative lower limb image is associated with the corresponding preoperative lower limb key point information.

[0044] The prosthesis region key point acquisition module is used to detect the key points of the prosthesis region in the postoperative lower limb image based on the postoperative lower limb image to obtain the prosthesis region key point information.

[0045] A non-prosthetic area key point acquisition module is configured to detect key points of the non-prosthetic area in the post-operative lower limb image based on the pre-operative lower limb image and the prosthetic area key point information, thereby obtaining the non-prosthetic area key point information. According to a third aspect of the present invention, an electronic device is provided, comprising a processor and a memory; the processor is configured to execute the steps of the method described in the first aspect of the present invention by invoking a program or instruction stored in the memory.

[0046] According to a fourth aspect of the present invention, there is provided a computer-readable storage medium storing a program or instructions, wherein the program or instructions enable a computer to execute the steps of the method according to the first aspect of the present invention.

[0047] The present invention has at least the following beneficial effects:

[0048] The postoperative lower limb key point detection method provided by an embodiment of the present invention includes: obtaining a postoperative lower limb image that currently needs to be detected and a preoperative lower limb image corresponding to the postoperative lower limb image; the postoperative lower limb image is an image taken after a knee replacement surgery on the lower limb, and the preoperative lower limb image is associated with the corresponding preoperative lower limb key point information; based on the postoperative lower limb image, the key points of the prosthesis area in the postoperative lower limb image are detected to obtain the prosthesis area key point information; based on the preoperative lower limb image and the prosthesis area key point information, the key points of the non-prosthesis area in the postoperative lower limb image are detected to obtain the non-prosthesis area key point information. The method provided by the embodiment of the present invention can achieve accurate detection of postoperative lower limb X-ray image key points without the need to establish an additional deep learning model, and can improve the model deployment efficiency of the lower limb skeleton intelligent measurement related system.

[0049] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present invention, nor is it intended to limit the scope of the present invention. Other features of the present invention will become readily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0051] Figure 1 A flowchart of a method for detecting key points of lower limbs after surgery provided by an embodiment of the present invention;

[0052] Figure 2 Schematic diagram of key point detection in the prosthesis area;

[0053] Figure 3 a is a schematic diagram of the lower limb image before surgery;

[0054] Figure 3 b is a schematic diagram of the lower limb image after surgery. DETAILED DESCRIPTION

[0055] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work shall fall within the scope of protection of the present invention.

[0056] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which this invention pertains. The terms used herein in the specification of the present invention are for the purpose of describing specific embodiments only and are not intended to limit the present invention. The term "and / or" as used herein includes any and all combinations of one or more of the associated listed items.

[0057] It should be noted that some exemplary embodiments are described as processes or methods depicted as flowcharts. Although the flowcharts describe the steps as sequential processes, many of the steps can be performed in parallel, concurrently, or simultaneously. In addition, the order of the steps can be rearranged. A process can be terminated when its operation is completed, but can also have additional steps not included in the accompanying drawings. A process can correspond to a method, function, procedure, subroutine, subprogram, etc.

[0058] In view of the technical problem that the interference scenario between preoperative physiological structure and postoperative implants requires independent modeling to detect the key points of the lower limbs, the embodiment of the present invention provides a postoperative lower limb key point detection method that can accurately detect the key points of postoperative lower limb X-ray images without the need to establish an additional deep learning model. Figure 1 As shown, the method may include the following steps:

[0059] S100, obtain the postoperative lower limb image that currently needs to be detected and the preoperative lower limb image corresponding to the postoperative lower limb image; the postoperative lower limb image is an image taken after the total knee replacement surgery on the lower limb, and the preoperative lower limb image is associated with the corresponding preoperative lower limb key point information, that is, the lower limb key points in the preoperative lower limb image have been detected.

[0060] In an embodiment of the present invention, the postoperative lower limb image currently being examined may be an X-ray image. The preoperative lower limb image corresponding to the postoperative lower limb image is an image of the same subject taken before total knee replacement surgery. The subject is a patient who requires total knee replacement surgery.

[0061] In an embodiment of the present invention, the key points of the lower limbs are the parts of the lower limbs that can be used to measure the force lines and related angles of the lower limbs. The specific positions and number of the key points can be set based on actual needs, that is, they can be set based on the application purpose of the image. In an illustrative embodiment, the key points of the lower limbs may include key points in the knee joint area and key points other than the knee joint area. Among them, the key points in the knee joint area may include the lowest point of the lateral femoral condyle, the apex of the femoral intercondylar fossa, the lowest point of the medial femoral condyle, the lowest point of the lateral tibial plateau, the lowest point of the medial tibial plateau and the midpoint of the tibial intercondylar eminence. The key points other than the knee joint area may include the center of the femoral head and the tip of the greater trochanter in the hip joint area, the lateral malleolus point and the medial malleolus point in the ankle joint area, etc.

[0062] S200 , based on the post-operative lower limb image, detecting key points of the prosthesis region in the post-operative lower limb image to obtain key point information of the prosthesis region.

[0063] In this embodiment of the present invention, key points in a postoperative lower limb image may include prosthesis region key points and non-prosthesis region key points. The prosthesis region key points correspond to key points in the knee joint region in the preoperative lower limb image, while the non-prosthesis region key points correspond to key points in the preoperative lower limb image excluding the knee joint region. The prosthesis region is the specific space created after surgical implantation of a knee prosthesis and fully corresponds to the original anatomical location of the knee joint.

[0064] S300 , based on the preoperative lower limb image and the prosthesis area key point information, detecting key points of the non-prosthesis area in the postoperative lower limb image to obtain the non-prosthesis area key point information.

[0065] S400: Using the prosthesis region key point information and the non-prosthesis region key point information as lower limb key point information corresponding to a post-operative lower limb image currently to be detected.

[0066] The postoperative lower limb key point detection method provided by the embodiment of the present invention can realize accurate detection of key points in postoperative lower limb X-ray images without the need to establish an additional deep learning model, and can improve the model deployment efficiency of the lower limb bone intelligent measurement related system.

[0067] Furthermore, in an embodiment of the present invention, the key point information of the prosthesis region may include coordinates of the first key point to the sixth key point.

[0068] Further, if Figure 2 As shown, S200 may specifically include:

[0069] S201 , using an image segmentation algorithm to segment the prosthesis region in the post-operative lower limb image to obtain the prosthesis region.

[0070] In the embodiment of the present invention, the image segmentation algorithm may be any existing image segmentation algorithm. In an exemplary embodiment, for example, it may be a segmentation algorithm such as an adaptive threshold method, a watershed algorithm, and an OTSU algorithm.

[0071] It is known to those skilled in the art that any method of segmenting the prosthesis region in the post-operative lower limb image using an image segmentation algorithm to obtain the prosthesis region falls within the scope of protection of the present invention.

[0072] In the embodiment of the present invention, the postoperative lower limb image can be as follows Figure 2 As shown in M1 in the figure, the segmented prosthesis area can be Figure 2 As shown in M2.

[0073] S202 : Extracting the contours of the upper region and the lower region of the prosthesis region using an image contour extraction method to obtain an upper region contour and a lower region contour.

[0074] In the embodiment of the present invention, the upper region of the prosthesis region refers to the region connected to the thigh, and the lower region refers to the region connected to the calf.

[0075] In the embodiment of the present invention, the image contour extraction method may be any existing image contour extraction method. In an exemplary embodiment, for example, it may be a Sobel operator, a Canny operator, a Laplacian operator, and the like.

[0076] Those skilled in the art will appreciate that any method of extracting the contours of the upper and lower regions of the prosthesis region using an image contour extraction method to obtain the contours of the upper and lower regions falls within the scope of protection of the present invention.

[0077] Those skilled in the art know that the upper region outline and the lower region outline are represented by corresponding pixel coordinates.

[0078] In the embodiment of the present invention, the upper region outline and the lower region outline may be respectively as follows: Figure 2 As shown in M3 and M4.

[0079] S203 , obtaining coordinates of the first to third key points in the prosthesis region key point information based on the upper region contour, and obtaining coordinates of the fourth to sixth key points in the prosthesis region key point information based on the lower region contour.

[0080] In the embodiment of the present invention, the coordinate system where the coordinates of the pixel points are located may be a rectangular coordinate system constructed with the upper left corner of the image as the origin o, the downward direction as the y-axis, and the upward direction as the x-axis.

[0081] Further, if Figure 2 As shown, in S203, the coordinates of the first key point to the third key point in the key point information of the prosthesis region based on the upper region contour are obtained, specifically including:

[0082] S2031 , sorting the coordinates of all pixel points corresponding to the upper area outline in ascending order of the vertical coordinate to obtain a sorted upper sorted coordinate set.

[0083] S2032: Use the coordinates of the last k1 pixels in the sorted upper sorted coordinate set as the first upper coordinate set.

[0084] In this embodiment of the present invention, k1 = f1 × N1, where f1 is a preset coefficient and N1 is the number of pixel coordinates corresponding to the upper region contour, and 0 < f1 < 1. f1 can be an empirical value, such as a value that ensures that the first upper coordinate set includes the vertex of the intercondylar notch of the femoral fossa. Specifically, it can be obtained from historical data. For example, f1 can be obtained by the following steps:

[0085] (1) Obtain Q sample images; the sample images are lower limb X-ray images with key points of the lower limbs marked.

[0086] (2) The ordinate values ​​corresponding to the upper region contour of each sample image are sorted in ascending order to obtain the sorted ordinate value corresponding to the sample image.

[0087] (3) Obtain the sorted position value of the ordinate value of the vertex of the intercondylar notch of the femur in each sample image in the corresponding sorted ordinate value.

[0088] (4) k1 is determined based on the sorted position values ​​of the ordinate values ​​of the vertices of the intercondylar notch of the femoral bones of the Q sample images.

[0089] In one exemplary embodiment, f1 = (P1 / M1+P2 / M2+…+PQ / MQ) / Q, where PQ is the sorted position of the ordinate value of the apex of the intercondylar notch of the femoral fossa in the Qth sample image, and MQ is the number of pixel coordinates of the upper region contour in the Qth sample image. In another exemplary embodiment, f1 = min(P1 / M1+P2 / M2+…+PQ / MQ) / Q. In a specific embodiment, f1 = 1 / 2.

[0090] S2033 , sorting the pixel coordinates in the first coordinate set in ascending order of the horizontal coordinates to obtain a second upper coordinate set.

[0091] S2034: Divide the second upper coordinate set into three sub-coordinate sets, namely a first upper sub-coordinate set, a second upper sub-coordinate set, and a third upper sub-coordinate set.

[0092] In an embodiment of the present invention, the second upper coordinate set can be equally divided into three sub-coordinate sets, that is, the first third of the coordinates in the second upper coordinate set is used as the first upper sub-coordinate set, the middle third of the coordinates is used as the second upper sub-coordinate set, and the last third of the coordinates is used as the third upper sub-coordinate set.

[0093] S2035 , taking the pixel coordinates corresponding to the minimum vertical coordinate value in the second upper sub-coordinate set as the coordinates of the first key point.

[0094] S2036: Divide the second upper coordinate set into two sub-coordinate sets using the horizontal coordinate of the first key point as a dividing point, namely a fourth upper sub-coordinate set and a fifth upper sub-coordinate set, wherein the maximum horizontal and vertical coordinate values ​​in the fourth upper sub-coordinate set are smaller than the minimum horizontal coordinate value in the fifth upper sub-coordinate set.

[0095] S2037 , using the pixel coordinates corresponding to the maximum vertical coordinate value in the fourth upper sub-coordinate set as the coordinates of the second key point, and using the pixel coordinates corresponding to the maximum vertical coordinate value in the fifth upper sub-coordinate set as the coordinates of the third key point.

[0096] Furthermore, in S203, the step of obtaining the coordinates of the fourth to sixth key points in the key point information of the prosthesis region based on the lower region contour specifically includes:

[0097] S10 , sorting the coordinates of all pixel points corresponding to the lower area outline in ascending order of the vertical coordinate to obtain a sorted lower sorted coordinate set.

[0098] S11 , taking the first k2 pixel coordinates in the sorted lower sorted coordinate set as the first lower coordinate set.

[0099] In this embodiment of the present invention, k2 = f2 × N2, where f2 is a preset coefficient and N2 is the number of pixel coordinates corresponding to the lower region outline, with 0 < f2 < 1. F2 can be an empirical value such that the first lower coordinate set includes the lowest point of the lateral femoral condyle and the lowest point of the medial femoral condyle. This value can be obtained from historical data in a similar manner to f1. In one specific embodiment, f2 = 1 / 2.

[0100] S12 , sorting the pixel coordinates in the first lower coordinate set in ascending order of the horizontal coordinates to obtain a second lower coordinate set.

[0101] S13, dividing the second lower coordinate set into three equal sub-coordinate sets, namely the first sub-coordinate set, the second sub-coordinate set, and the third sub-coordinate set;

[0102] S14: Taking the coordinates of the pixel point corresponding to the maximum vertical coordinate value in the second lower sub-coordinate set as the coordinates of the fourth key point.

[0103] S15, dividing the second lower coordinate set into two sub-coordinate sets using the horizontal coordinate of the fourth key point as a dividing point, namely a fourth lower sub-coordinate set and a fifth lower sub-coordinate set, wherein the maximum horizontal and vertical coordinate values ​​in the fourth lower sub-coordinate set are less than the minimum horizontal coordinate value in the fifth lower sub-coordinate set.

[0104] S16, using the pixel coordinates corresponding to the maximum vertical coordinate value in the fourth sub-coordinate set as the coordinates of the fifth key point, and using the pixel coordinates corresponding to the maximum vertical coordinate value in the fifth sub-coordinate set as the coordinates of the sixth key point.

[0105] like Figure 2 As shown, the first to sixth key points obtained can be respectively as Figure 2 As shown in the numbers 1 to 6.

[0106] Furthermore, S300 may specifically include:

[0107] S301 , selecting any key point in the area corresponding to the prosthesis area in the preoperative lower limb image as a reference point.

[0108] In the embodiment of the present invention, the area corresponding to the prosthesis area in the preoperative lower limb image is the knee joint area. The reference point can be selected based on actual needs, for example, the first key point is selected as the reference point.

[0109] S302, set counter i=1.

[0110] S303, if i≤n, execute S304, otherwise, execute S309; ​​n is the number of key points in the non-prosthesis area.

[0111] S304, obtaining the relative distance d between the reference point and the key point of the i-th non-prosthesis area in the preoperative lower limb image. i .

[0112] In the embodiment of the present invention, d i The difference between the horizontal coordinate and the vertical coordinate between the reference point and the key point of the i-th non-prosthesis area in the preoperative lower limb image is d i =(△x i , △y i ), △x i is the horizontal coordinate difference between the reference point and the key point of the i-th non-prosthesis area in the preoperative lower limb image, △y i is the vertical coordinate difference between the reference point and the key point of the i-th non-prosthesis area in the preoperative lower limb image. Figure 3 a shows the positional relationship between a reference point and a key point NKP in a non-prosthesis area in a preoperative lower limb image.

[0113] S305, based on d i , get the initial position P0 of the key point of the i-th non-prosthesis area in the postoperative lower limb image i .

[0114] In the embodiment of the present invention, P0 i Based on the coordinate information of the key points corresponding to the reference points in the postoperative lower limb images and d i Determine, that is, in the postoperative lower limb image, obtain the coordinate information of the key point corresponding to the reference point in the postoperative lower limb image, which differs by d i The pixel point is P0 i , specific, P0 i The horizontal coordinate x0 i =xc+△x i , P0 i The vertical coordinate y0 i =yc+△y i , xc and yc are the horizontal and vertical coordinates of the key points corresponding to the reference points in the postoperative lower limb image, respectively. Figure 3 b shows the initial position of a key point NKP in the non-prosthesis area in the postoperative lower limb image.

[0115] S306, with P0 i A set area is constructed as the center, and m sampling points are uniformly sampled in the set area. An image area with a width of w and a height of h is intercepted with each sampling point as the center to obtain m image areas.

[0116] In the embodiment of the present invention, the set area can be square, preferably square. The size of the set area can be set based on actual needs, as long as the constructed set area is located on the contour line of the lower limb.

[0117] In the embodiments of the present invention, w and h can be set based on actual needs, as long as the structural features of the corresponding area in the image region can be clearly displayed. w and h can be specifically determined based on the analysis accuracy requirements and computing resources. Generally, the higher the analysis accuracy requirements, the smaller w and h are. As more computing resources are required, w and h can be appropriately set smaller.

[0118] S307, respectively obtain the feature matching degree between each image region and the reference image region, and obtain m feature matching degrees; wherein the reference image region is an image region with a width w and a height h cut with the i-th non-prosthesis region key point in the preoperative lower limb image as the center.

[0119] In the embodiment of the present invention, the feature matching degree can be obtained based on an existing feature matching method, such as cosine similarity, structural similarity calculation method, etc., preferably structural similarity.

[0120] S308, take the sampling point corresponding to the maximum feature matching degree among the m feature matching degrees as the final position of the i-th non-prosthesis area key point after surgery, and add the final position of the i-th non-prosthesis area key point to the current non-prosthesis area key point set; set i=i+1, execute S303; the initial value of the current non-prosthesis area key point set is empty.

[0121] S309 , obtaining the non-prosthesis area key point information based on the current non-prosthesis area key point set, that is, obtaining the coordinate information of each non-prosthesis area key point.

[0122] Based on the same inventive concept, an embodiment of the present invention further provides a postoperative lower limb key point detection device, the device comprising:

[0123] The image acquisition module is used to obtain the postoperative lower limb image that needs to be detected and the preoperative lower limb image corresponding to the postoperative lower limb image; the postoperative lower limb image is an image taken after the lower limb undergoes knee replacement surgery, and the preoperative lower limb image is associated with the corresponding preoperative lower limb key point information.

[0124] The prosthesis region key point acquisition module is used to detect the key points of the prosthesis region in the postoperative lower limb image based on the postoperative lower limb image to obtain the prosthesis region key point information.

[0125] The non-prosthesis area key point acquisition module is used to detect the key points of the non-prosthesis area in the post-operative lower limb image based on the pre-operative lower limb image and the prosthesis area key point information to obtain the non-prosthesis area key point information.

[0126] Furthermore, the key point information of the prosthesis area includes the coordinates of the first key point to the sixth key point, wherein the first key point is the apex of the femoral intercondylar fossa, the second key point is the lowest point of the lateral femoral condyle, the third key point is the lowest point of the medial femoral condyle, the fourth key point is the midpoint of the tibial intercondylar eminence, the fifth key point is the lowest point of the lateral tibial plateau, and the sixth key point is the lowest point of the medial tibial plateau.

[0127] The prosthesis area key point acquisition module specifically includes:

[0128] The prosthesis region segmentation unit is used to segment the prosthesis region in the post-operative lower limb image using an image segmentation algorithm to obtain the prosthesis region.

[0129] The contour extraction unit is used to extract the contours of the upper region and the lower region of the prosthesis region by using an image contour extraction method to obtain the upper region contour and the lower region contour.

[0130] The prosthesis region key point acquisition unit is used to acquire the coordinates of the first to third key points in the prosthesis region key point information based on the upper region outline, and to acquire the coordinates of the fourth to sixth key points in the prosthesis region key point information based on the lower region outline. Figure 1 The method shown in the embodiment shown, therefore, for the functions that can be realized by each functional module of the device, please refer to Figure 1 The description of the illustrated embodiment is omitted for brevity.

[0131] An embodiment of the present invention also provides an electronic device, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are configured to execute the method described in the embodiment of the present invention.

[0132] An embodiment of the present invention further provides a computer-readable storage medium storing computer-executable instructions, wherein the computer instructions are used to execute the method described in the embodiment of the present invention.

[0133] It should be understood that the various forms of the processes shown above can be used to reorder, add, or delete steps. For example, the steps described in the present invention can be performed in parallel, sequentially, or in a different order, as long as the desired results of the technical solutions disclosed in the present invention can be achieved. This is not limited herein.

[0134] The above specific embodiments do not limit the scope of protection of the present invention. Those skilled in the art will appreciate that various modifications, combinations, sub-combinations, and substitutions may be made based on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention are intended to be included within the scope of protection of the present invention.

Claims

1. A method for detecting key points of lower limbs after surgery, characterized in that: The method comprises the following steps: S100, obtaining a postoperative lower limb image currently to be inspected and a preoperative lower limb image corresponding to the postoperative lower limb image; the postoperative lower limb image is an image taken after a knee replacement surgery on the lower limb, and the preoperative lower limb image is associated with corresponding preoperative lower limb key point information; S200, based on the postoperative lower limb image, detecting key points of the prosthesis region in the postoperative lower limb image to obtain key point information of the prosthesis region; S300, based on the preoperative lower limb image and the prosthesis area key point information, detecting key points of the non-prosthesis area in the postoperative lower limb image to obtain the non-prosthesis area key point information; S400: Using the prosthesis region key point information and the non-prosthesis region key point information as lower limb key point information corresponding to a post-operative lower limb image currently to be detected.

2. The method according to claim 1, characterized in that The prosthesis region key point information includes the coordinates of the first key point to the sixth key point, wherein the first key point is the apex of the femoral intercondylar fossa, the second key point is the lowest point of the lateral femoral condyle, the third key point is the lowest point of the medial femoral condyle, the fourth key point is the midpoint of the tibial intercondylar eminence, the fifth key point is the lowest point of the lateral tibial plateau, and the sixth key point is the lowest point of the medial tibial plateau; S200 specifically includes: S201, using an image segmentation algorithm to segment the prosthesis region in the post-operative lower limb image to obtain a prosthesis region; S202, extracting the contours of the upper region and the lower region of the prosthesis region using an image contour extraction method to obtain an upper region contour and a lower region contour; S203 , obtaining coordinates of the first to third key points in the prosthesis region key point information based on the upper region contour, and obtaining coordinates of the fourth to sixth key points in the prosthesis region key point information based on the lower region contour.

3. The method according to claim 2, characterized in that In S203, the step of obtaining the coordinates of the first to third key points in the key point information of the prosthesis region based on the upper region contour specifically includes: S2031, sorting the coordinates of all pixels corresponding to the upper region outline in ascending order of ordinates to obtain a sorted upper sorted coordinate set; S2032, taking the coordinates of the last k1 pixels in the sorted upper sorted coordinate set as the first upper coordinate set; S2033, sorting the pixel coordinates in the first coordinate set in ascending order of horizontal coordinates to obtain a second upper coordinate set; S2034, dividing the second upper coordinate set into three sub-coordinate sets, namely a first upper sub-coordinate set, a second upper sub-coordinate set, and a third upper sub-coordinate set; S2035: Taking the coordinates of the pixel point corresponding to the minimum vertical coordinate value in the second upper sub-coordinate set as the coordinates of the first key point; S2036: Divide the second upper coordinate set into two sub-coordinate sets using the abscissa of the first key point as a dividing point, namely a fourth upper sub-coordinate set and a fifth upper sub-coordinate set, wherein the maximum abscissa value and the minimum abscissa value of the fifth upper sub-coordinate set are less than the maximum abscissa value of the fifth upper sub-coordinate set. S2037 , using the pixel coordinates corresponding to the maximum vertical coordinate value in the fourth upper sub-coordinate set as the coordinates of the second key point, and using the pixel coordinates corresponding to the maximum vertical coordinate value in the fifth upper sub-coordinate set as the coordinates of the third key point.

4. The method according to claim 2, characterized in that In S203, the step of obtaining the coordinates of the fourth to sixth key points in the key point information of the prosthesis region based on the lower region contour specifically includes: S10, sorting the coordinates of all pixels corresponding to the lower area outline in ascending order of the ordinate to obtain a sorted lower sorted coordinate set; S11, taking the coordinates of the first k2 pixels in the sorted lower sorted coordinate set as the first lower coordinate set; S12, sorting the pixel coordinates in the first lower coordinate set in ascending order of horizontal coordinates to obtain a second lower coordinate set; S13, dividing the second lower coordinate set into three sub-coordinate sets, namely the first sub-coordinate set, the second sub-coordinate set and the third sub-coordinate set; S14, taking the coordinates of the pixel point corresponding to the maximum vertical coordinate value in the second sub-coordinate set as the coordinates of the fourth key point; S15, dividing the second lower coordinate set into two sub-coordinate sets using the abscissa of the fourth key point as a dividing point, namely a fourth lower sub-coordinate set and a fifth lower sub-coordinate set, wherein the maximum abscissa value and the vertical coordinate value in the fourth lower sub-coordinate set are less than the minimum abscissa value in the fifth lower sub-coordinate set; S16, using the pixel coordinates corresponding to the maximum vertical coordinate value in the fourth sub-coordinate set as the coordinates of the fifth key point, and using the pixel coordinates corresponding to the maximum vertical coordinate value in the fifth sub-coordinate set as the coordinates of the sixth key point.

5. The method according to claim 2, characterized in that S300 specifically includes: S301, selecting any key point in the area corresponding to the prosthesis area in the preoperative lower limb image as a reference point; S302, set counter i=1; S303, if i≤n, execute S304, otherwise execute S309; ​​n is the number of key points in the non-prosthesis area; S304, obtaining the relative distance d between the reference point and the key point of the i-th non-prosthesis area in the preoperative lower limb image. i ; S305, based on d i , get the initial position P0 of the key point of the i-th non-prosthesis area in the postoperative lower limb image i ; S306, with P0 i Construct a set area as the center, and uniformly sample m sampling points in the set area, and intercept the image area with width w and height h with each sampling point as the center to obtain m image areas; S307, respectively obtaining a feature matching degree between each image region and a reference image region, obtaining m feature matching degrees; wherein the reference image region is an image region with a width w and a height h cut off centered at the key point of the i-th non-prosthesis region in the preoperative lower limb image; S308: The sampling point corresponding to the maximum feature matching degree among the m feature matching degrees is used as the final position of the i-th non-prosthesis area key point after surgery, and the final position of the i-th non-prosthesis area key point is added to the current non-prosthesis area key point set; i is set to i+1, and S303 is executed; the initial value of the current non-prosthesis area key point set is empty; S309 : Obtaining the non-prosthesis area key point information based on the current non-prosthesis area key point set.

6. The method according to claim 5, characterized in that The setting area is square.

7. A postoperative lower limb key point detection device, characterized in that: The device comprises: An image acquisition module is used to acquire a postoperative lower limb image that needs to be detected and a preoperative lower limb image corresponding to the postoperative lower limb image; the postoperative lower limb image is an image taken after a knee replacement surgery on the lower limb, and the preoperative lower limb image is associated with corresponding preoperative lower limb key point information; A prosthesis region key point acquisition module is used to detect key points of the prosthesis region in the post-operative lower limb image based on the post-operative lower limb image to obtain key point information of the prosthesis region; The non-prosthesis area key point acquisition module is used to detect the key points of the non-prosthesis area in the post-operative lower limb image based on the pre-operative lower limb image and the prosthesis area key point information to obtain the non-prosthesis area key point information.

8. The device according to claim 7, characterized in that The prosthesis region key point information includes the coordinates of the first key point to the sixth key point, wherein the first key point is the apex of the femoral intercondylar fossa, the second key point is the lowest point of the lateral femoral condyle, the third key point is the lowest point of the medial femoral condyle, the fourth key point is the midpoint of the tibial intercondylar eminence, the fifth key point is the lowest point of the lateral tibial plateau, and the sixth key point is the lowest point of the medial tibial plateau; The prosthesis area key point acquisition module specifically includes: a prosthesis region segmentation unit, configured to segment the prosthesis region in the post-operative lower limb image using an image segmentation algorithm to obtain a prosthesis region; a contour extraction unit, configured to extract the contours of the upper region and the lower region of the prosthesis region by using an image contour extraction method to obtain an upper region contour and a lower region contour; The prosthesis area key point acquisition unit is used to acquire the coordinates of the first to third key points in the prosthesis area key point information based on the upper area contour, and to acquire the coordinates of the fourth to sixth key points in the prosthesis area key point information based on the lower area contour.

9. An electronic device, characterized in that: including processor and memory; The processor is configured to execute the steps of the method according to any one of claims 1 to 8 by calling the program or instructions stored in the memory.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium is used to store a program or instruction, and the program or instruction enables a computer to execute the steps of the method according to any one of claims 1 to 8.

Images