Total lower limb image splicing method and system and electronic equipment

Abstract

The invention discloses a full-lower-limb image splicing method and system and electronic equipment. The method comprises the following steps: determining an infrared mark point on a target object; reading coordinate information of the infrared mark point through an infrared camera; based on the coordinate information, photographing the whole lower limb of the target object to obtain a photographed image containing each mark point; carrying out edge detection on the shot image of each mark point; after the edge of the mark point is detected, determining a minimum rectangular region surrounding the mark point to obtain a region of interest; and according to the region of interest, splicing the shot images to obtain a splicing result of the total lower limb image. According to the embodiment of the invention, the accuracy can be improved while the splicing error is reduced, better image quality and measurement precision are provided for clinic, and the method can be widely applied to the technical field of medical images.

Description

Technical Field

[0001] The present invention relates to the field of medical imaging technology, and in particular to a method, system and electronic equipment for splicing whole lower limb images. Background Art

[0002] Long leg splicing technology has important application value in the field of medical imaging, especially in the diagnosis, preoperative planning and postoperative evaluation of lower limb orthopedic and joint diseases. Complete and coherent full-length images of the lower limbs help evaluate bone structure, joint alignment and leg length differences, allowing doctors to more accurately diagnose diseases such as fractures, arthritis and congenital malformations. Accurate images help to develop effective surgical plans, helping surgeons to accurately plan incision locations, implant sizes and surgical methods, thereby improving surgical success rates and accelerating patient recovery. In addition, it also supports postoperative rehabilitation tracking, helping doctors monitor patients' recovery, evaluate treatment effectiveness, and adjust rehabilitation plans when necessary to ensure that patients can resume normal activities as soon as possible.

[0003] Currently, the long leg stitching image is achieved by stitching after multiple shots, but there are several challenges in this process. First, stitching error is the most common problem. Since the length of the lower limbs usually exceeds the field of view of a single image, it is necessary to take multiple images and stitch them together. However, the traditional stitching method is easily affected by factors such as image distortion, shooting angle and body movement, which leads to stitching errors. In particular, when there is no clear reference point between the upper and lower images, the error is more significant, and the left and right offset is obvious. Therefore, when there is a large error in the overlapping area of ​​the image, the system cannot measure the accurate length. Secondly, image magnification or reduction is another major problem. In X-ray photography, the target-object distance (TOD) refers to the distance between the lower limb bone and the plate. In actual operation, technicians usually set TOD by visual inspection, which is often not accurate enough and difficult to truly reflect the actual distance between the bone and the detector. Improper TOD setting may cause the image to be magnified or reduced, which in turn makes the measured length in the image inconsistent with the actual length of the lower limb. This will affect the doctor's accurate assessment of the patient's leg length, bone and joint structure. Finally, seamless stitching of long-leg images is also a technical challenge. The current stitching technology cannot ensure seamless alignment of continuous images, resulting in obvious stitching lines or even white bands in the overlapping areas of the two images, which does not meet the high clinical requirements for image quality. Summary of the invention

[0004] The main purpose of the embodiments of the present invention is to provide a method, system and electronic device for stitching images of the entire lower limb, which can reduce stitching errors while improving accuracy, and provide better image quality and measurement accuracy for clinical practice.

[0005] To achieve the above object, one aspect of an embodiment of the present invention provides a method for stitching whole lower limb images, comprising the following steps:

[0006] Determine infrared marker points on the target object;

[0007] Read the coordinate information of the infrared marking point through an infrared camera;

[0008] Based on the coordinate information, photograph the entire lower limb of the target object to obtain a photographed image containing each marked point;

[0009] Perform edge detection on the captured images of each marking point;

[0010] After detecting the edge of the marker point, determine the minimum rectangular area surrounding the marker point to obtain the region of interest;

[0011] According to the region of interest, each captured image is stitched together to obtain a stitching result of the entire lower limb image.

[0012] In some embodiments, determining an infrared marking point on the target object comprises the following steps:

[0013] Have the subject stand upright on the pedal in a fore-aft position with his back against the camera stand;

[0014] The knee joint of the target subject is straightened, and the femur and tibia and fibula are in the same vertical direction, so that the distances from the femur and tibia and fibula to the flat-panel detector are equal;

[0015] Three images were taken and spliced ​​in pairs to form a full-length image of the lower limb, and the overlapping length of each two images was calculated; then the first infrared marking point was pasted in the overlapping area between the first image and the second image, and the second infrared marking point was pasted in the overlapping area between the second image and the third image; the two infrared marking points were respectively pasted on the femur and tibia and fibula of one lower limb, and during the pasting process, the two infrared marking points were in the same vertical direction; the line connecting the two infrared marking points was parallel to the flat-panel detector.

[0016] In some embodiments, the step of reading the coordinate information of the infrared marking point by using an infrared camera includes the following steps:

[0017] Place the infrared camera on the bracket, on the same side of the two infrared marking points;

[0018] The coordinates of two infrared markers are read by an infrared camera, and the actual distance between the two infrared markers is calculated;

[0019] The calculation formula of the actual distance D is:

[0020] Among them, (x1, y1) and (x2, y2) represent the coordinates of two infrared marking points respectively.

[0021] In some embodiments, photographing the entire lower limb of the target object based on the coordinate information to obtain a photographed image containing each marked point includes the following steps:

[0022] The entire lower limbs of the target subject were photographed in three sections through X-ray exposure; the length of each section was 42 cm;

[0023] The first infrared marking point is captured by the first image and the second image at the same time, and the second infrared marking point is captured by the second image and the third image at the same time; the infrared marking point is highlighted in the image.

[0024] In some embodiments, the step of performing edge detection on the captured image of each marking point comprises the following steps:

[0025] Use Gaussian filter to smooth the captured images;

[0026] After smoothing, the Sobel operator is used to calculate the gradient strength and direction of each pixel in the captured image, and the non-maximum suppression method is used to refine the edge;

[0027] For each pixel, compare it with the adjacent pixels in the gradient direction. If the gradient value of the pixel is not the local maximum, suppress it to 0.

[0028] Two thresholds are set as a high threshold and a low threshold, respectively, to perform dual threshold detection, and classify pixels according to their gradient strength, determining pixels with gradient strength greater than the high threshold as strong edges; pixels with gradient strength between the high threshold and the low threshold as weak edges; pixels with gradient strength less than the low threshold as non-edges, and directly removing the non-edges;

[0029] Connect weak edge pixels to strong edge pixels. If a weak edge pixel is adjacent to a strong edge, the pixel is retained as an edge, otherwise it is suppressed as a non-edge pixel.

[0030] The edges of the highlighted infrared marker points in the captured image are extracted to form a contour around the infrared marker points.

[0031] In some embodiments, the expression for smoothing the captured image using a Gaussian filter is:

[0032]

[0033] Where G(x, y) represents the Gaussian function value of the point (x, y) on the two-dimensional plane, x and y represent the horizontal and vertical coordinates of the pixel in the image, respectively, and σ is the standard deviation of the Gaussian filter;

[0034] After the smoothing process is completed, the Sobel operator is used to calculate the gradient strength and direction of each pixel in the captured image, and the non-maximum suppression method is used to refine the edge. In this step, the Sobel operator detects the edge by calculating the derivatives in the horizontal direction and the vertical direction respectively, wherein the derivatives in the horizontal direction and the vertical direction are realized by convolution operation.

[0035] Horizontal direction G x The expression is:

[0036] Vertical direction G y The expression is:

[0037] The expression of gradient strength G is:

[0038] The expression of the gradient direction θ is:

[0039] In some embodiments, stitching the captured images according to the region of interest to obtain a stitching result of the whole lower limb image includes the following steps:

[0040] The coordinates of the upper left corner of the first image are (0, 0), and the second image is offset from the upper left corner of the first image by (0, Cd); wherein the coordinates (x2, y2) of the second image can be expressed in the coordinate system of the first image as follows:

[0041] x′2=x2

[0042] y2′=y2+Cd

[0043] Calculate the relative translation between markers:

[0044] Δx=x1-x′2

[0045] Δy=y1-y2′

[0046] The second image is translated so that the marked points (x1, y1) and (x′2, y2′) overlap, thereby achieving the splicing of the first image and the second image, and then completing the splicing between the second image and the third image;

[0047] In the process of image stitching, the weighted average fusion method is used to smoothly transition the color and brightness of the overlapping areas; the process specifically includes the following steps:

[0048] The weighted average fusion method performs weighted processing on the pixel values ​​of the overlapping area. The pixel values ​​of the overlapping area are I1(x, y) and I2(x, y), respectively. I1(x, y) represents the pixel value of the first image. The coordinates of the upper left corner of the first image are set to (0, 0). The value range of x is from 0 to A, and the value range of y is from to A, I2(x, y) represents the pixel value of the second image, and the upper left corner coordinates of the second image are set to (0, 0), the value range of x is from 0 to A, and the value range of y is from 0 to

[0049] The weighted calculation is performed through the weight coefficients w1 and w2 to obtain the fused pixel value I fuse :

[0050] I fuse =w1·I1(x, y)+w2·I2(x, y),

[0051] In the allocation of weighted coefficients, the weight coefficients are allocated according to the distance from the pixel to the boundary of the overlapping area; where the width of the overlapping area is d, and for each pixel in the overlapping area, its relative position in the overlapping area is d′, and the expression for allocating weights is:

[0052]

[0053] Among them, w1 and w2 are the weight coefficients of the corresponding pixels respectively.

[0054] In some embodiments, the method further includes: calculating a shooting distance between two marking points in the captured image, comparing the shooting distance with the actual distance calculated by the infrared camera, and then adjusting the TOD value to reduce the distance difference between the shooting distance and the actual distance;

[0055] Wherein, the calculation formula of the distance difference s is:

[0056] Among them, D i represents the shooting distance between the two marked points in the captured image; D represents the actual distance calculated by the infrared camera.

[0057] Another aspect of the embodiment of the present invention further provides a system for stitching images of the entire lower limb, comprising:

[0058] The first module is used to determine infrared marking points on the target object;

[0059] The second module is used to read the coordinate information of the infrared marking point through an infrared camera;

[0060] The third module is used to photograph the entire lower limb of the target object based on the coordinate information to obtain a photographed image containing each marking point;

[0061] The fourth module is used to perform edge detection on the captured images of each marking point;

[0062] The fifth module is used to determine the minimum rectangular area surrounding the marker point after detecting the edge of the marker point to obtain the region of interest;

[0063] The sixth module is used to stitch the captured images together according to the region of interest to obtain a stitching result of the entire lower limb image.

[0064] To achieve the above objective, another aspect of an embodiment of the present invention provides an electronic device, the electronic device comprising a memory and a processor, the memory storing a computer program, and the processor implementing the above-mentioned method when executing the computer program.

[0065] To achieve the above objective, another aspect of an embodiment of the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the above-mentioned method is implemented.

[0066] The embodiment of the present invention also discloses a computer program product or a computer program, which includes a computer instruction stored in a computer-readable storage medium. A processor of a computer device can read the computer instruction from the computer-readable storage medium, and the processor executes the computer instruction, so that the computer device executes the above method.

[0067] The embodiments of the present invention include at least the following beneficial effects: the present invention provides a method, system and electronic device for stitching images of the entire lower limbs, the scheme determines infrared marking points on the target object; reads the coordinate information of the infrared marking points through an infrared camera; photographs the entire lower limbs of the target object based on the coordinate information to obtain a photographed image containing each marking point; performs edge detection on the photographed image of each marking point; after detecting the edge of the marking point, determines the minimum rectangular area surrounding the marking point to obtain a region of interest; stitches each photographed image according to the region of interest to obtain a stitching result of the entire lower limb image. The embodiments of the present invention can reduce stitching errors while improving accuracy, providing better image quality and measurement accuracy for clinical use. BRIEF DESCRIPTION OF THE DRAWINGS

[0068] Figure 1 It is a schematic diagram of an implementation environment provided by an embodiment of the present invention;

[0069] Figure 2is a flow chart of the overall steps provided by an embodiment of the present invention;

[0070] Figure 3 is a schematic diagram of a system framework provided by an embodiment of the present invention;

[0071] Figure 4 is a schematic diagram of stitched images of lower limbs provided by an embodiment of the present invention;

[0072] Figure 5 It is a schematic diagram of the hardware structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0073] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the embodiments of the present invention. They are only examples of devices and methods consistent with some aspects of the embodiments of the present invention as detailed in the attached claims.

[0074] It is understood that the terms "first", "second", etc. used in the present invention may be used herein to describe various concepts, but unless otherwise specified, these concepts are not limited by these terms. These terms are only used to distinguish one concept from another. For example, without departing from the scope of the embodiments of the present invention, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the words "if" and "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".

[0075] The terms "at least one", "multiple", "each", "any", etc. used in the present invention, at least one includes one, two or more, multiple includes two or more, each refers to each of the corresponding multiple, and any refers to any one of the multiple.

[0076] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which the present invention belongs. The terms used herein are only for the purpose of describing the embodiments of the present invention and are not intended to limit the present invention.

[0077] The stitching method, system and electronic device of the whole lower limb image provided by the embodiment of the present invention relate to the field of medical imaging technology. The stitching method of the whole lower limb image provided by the embodiment of the present invention can be applied to a terminal, can also be applied to a server, and can also be software running in a terminal or a server. In some embodiments, the terminal can be a smart phone, a tablet computer, a laptop computer, a desktop computer, a smart speaker, a smart watch, and a car terminal, etc., but is not limited to this; the server side can be configured as an independent physical server, or it can be configured as a server cluster or distributed system composed of multiple physical servers, and can also be configured as a cloud server that provides cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms and other basic cloud computing services. The server can also be a node server in a blockchain network; the software can be an application that implements the stitching method of the whole lower limb image, etc., but is not limited to the above forms.

[0078] The present invention can be used in many general or special computer system environments or configurations. For example: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronic devices, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, etc. The present invention can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present invention can also be practiced in distributed computing environments, in which tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media including storage devices.

[0079] like Figure 1 FIG. 1 is a schematic diagram of an implementation environment provided by an embodiment of the present invention. Figure 1 The implementation environment includes at least one terminal 102 and a server 101. The terminal 102 and the server 101 can be connected to a network wirelessly or wired to complete data transmission and exchange.

[0080] Server 101 can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers. It can also be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN (Content Delivery Network), as well as big data and artificial intelligence platforms.

[0081] In addition, the server 101 can also be a node server in the blockchain network. Blockchain is a new application model of computer technologies such as distributed data storage, peer-to-peer transmission, consensus mechanism, and encryption algorithm.

[0082] The terminal 102 may be a smart phone, a tablet computer, a laptop computer, a desktop computer, a smart speaker, a smart watch, etc. The terminal 102 may also be a vehicle-mounted terminal of various device types described above, but is not limited thereto. The terminal 102 and the server 101 may be directly or indirectly connected via wired or wireless communication, which is not limited in this embodiment of the present invention.

[0083] Based on the example Figure 1 In the implementation environment shown, an embodiment of the present invention provides a method for stitching images of the entire lower limbs. The following is explained using the example of the method for stitching images of the entire lower limbs being applied in the server 101. It can be understood that the method can also be applied in the terminal 102.

[0084] Reference Figure 2 , Figure 2 The flowchart of the method for stitching whole lower limb images applied to a server provided in an embodiment of the present invention, the execution subject of the method can be any of the aforementioned computer devices (including a server or a terminal). Figure 2 , the method may include the following steps:

[0085] Determine infrared marker points on the target object;

[0086] Read the coordinate information of the infrared marking point through an infrared camera;

[0087] Based on the coordinate information, photograph the entire lower limb of the target object to obtain a photographed image containing each marked point;

[0088] Perform edge detection on the captured images of each marking point;

[0089] After detecting the edge of the marker point, determine the minimum rectangular area surrounding the marker point to obtain the region of interest;

[0090] According to the region of interest, each captured image is stitched together to obtain a stitching result of the entire lower limb image.

[0091] In some embodiments, determining an infrared marking point on the target object comprises the following steps:

[0092] Have the subject stand upright on the pedal in a fore-aft position with his back against the camera stand;

[0093] The knee joint of the target subject is straightened, and the femur and tibia and fibula are in the same vertical direction, so that the distances from the femur and tibia and fibula to the flat-panel detector are equal;

[0094] Three images were taken and spliced ​​in pairs to form a full-length image of the lower limb, and the overlapping length of each two images was calculated; then the first infrared marking point was pasted in the overlapping area between the first image and the second image, and the second infrared marking point was pasted in the overlapping area between the second image and the third image; the two infrared marking points were respectively pasted on the femur and tibia and fibula of one lower limb, and during the pasting process, the two infrared marking points were in the same vertical direction; the line connecting the two infrared marking points was parallel to the flat-panel detector.

[0095] In some embodiments, the step of reading the coordinate information of the infrared marking point by using an infrared camera includes the following steps:

[0096] Place the infrared camera on the bracket, on the same side of the two infrared marking points;

[0097] The coordinates of two infrared markers are read by an infrared camera, and the actual distance between the two infrared markers is calculated;

[0098] The calculation formula of the actual distance D is:

[0099] Among them, (x1, y1) and (x2, y2) represent the coordinates of two infrared marking points respectively.

[0100] In some embodiments, photographing the entire lower limb of the target object based on the coordinate information to obtain a photographed image containing each marked point includes the following steps:

[0101] The entire lower limbs of the target subject were photographed in three sections through X-ray exposure; the length of each section was 42 cm;

[0102] The first infrared marking point is captured by the first image and the second image at the same time, and the second infrared marking point is captured by the second image and the third image at the same time; the infrared marking point is highlighted in the image.

[0103] In some embodiments, the step of performing edge detection on the captured image of each marking point comprises the following steps:

[0104] Use Gaussian filter to smooth the captured images;

[0105] After smoothing, the Sobel operator is used to calculate the gradient strength and direction of each pixel in the captured image, and the non-maximum suppression method is used to refine the edge;

[0106] For each pixel, compare it with the adjacent pixels in the gradient direction. If the gradient value of the pixel is not the local maximum, suppress it to 0.

[0107] Two thresholds are set as a high threshold and a low threshold, respectively, to perform dual threshold detection, and classify pixels according to their gradient strength, determining pixels with gradient strength greater than the high threshold as strong edges; pixels with gradient strength between the high threshold and the low threshold as weak edges; pixels with gradient strength less than the low threshold as non-edges, and directly removing the non-edges;

[0108] Connect weak edge pixels to strong edge pixels. If a weak edge pixel is adjacent to a strong edge, the pixel is retained as an edge, otherwise it is suppressed as a non-edge pixel.

[0109] The edges of the highlighted infrared marker points in the captured image are extracted to form a contour around the infrared marker points.

[0110] In some embodiments, the expression for smoothing the captured image using a Gaussian filter is:

[0111]

[0112] Where G(x, y) represents the Gaussian function value of the point (x, y) on the two-dimensional plane, x and y represent the horizontal and vertical coordinates of the pixel in the image, respectively, and σ is the standard deviation of the Gaussian filter;

[0113] After the smoothing process is completed, the Sobel operator is used to calculate the gradient strength and direction of each pixel in the captured image, and the non-maximum suppression method is used to refine the edge. In this step, the Sobel operator detects the edge by calculating the derivatives in the horizontal direction and the vertical direction respectively, wherein the derivatives in the horizontal direction and the vertical direction are realized by convolution operation.

[0114] Horizontal direction G x The expression is:

[0115] Vertical direction G y The expression is:

[0116] The expression of gradient strength G is:

[0117] The expression of the gradient direction θ is:

[0118] In some embodiments, stitching the captured images according to the region of interest to obtain a stitching result of the whole lower limb image includes the following steps:

[0119] The coordinates of the upper left corner of the first image are (0, 0), and the second image is offset from the upper left corner of the first image by (0, Cd); wherein the coordinates (x2, y2) of the second image can be expressed in the coordinate system of the first image as follows:

[0120] x′2=x2

[0121] y2′=y2+Cd

[0122] Calculate the relative translation between markers:

[0123] Δx=x1-x′2

[0124] Δy=y1-y2′

[0125] The second image is translated so that the marked points (x1, y1) and (x′2, y2′) overlap, thereby achieving the splicing of the first image and the second image, and then completing the splicing between the second image and the third image;

[0126] In the process of image stitching, the weighted average fusion method is used to smoothly transition the color and brightness of the overlapping areas; the process specifically includes the following steps:

[0127] The weighted average fusion method performs weighted processing on the pixel values ​​of the overlapping area. The pixel values ​​of the overlapping area are I1(x, y) and I2(x, y), respectively. I1(x, y) represents the pixel value of the first image. The coordinates of the upper left corner of the first image are set to (0, 0). The value range of x is from 0 to A, and the value range of y is from to A, I2(x, y) represents the pixel value of the second image, and the upper left corner coordinates of the second image are set to (0, 0), the value range of x is from 0 to A, and the value range of y is from 0 to

[0128] The weighted calculation is performed through the weight coefficients w1 and w2 to obtain the fused pixel value I fuse :

[0129] I fuse =w1·I1(x, y)+w2·I2(x, y),

[0130] In the allocation of weighted coefficients, the weight coefficients are allocated according to the distance from the pixel to the boundary of the overlapping area; where the width of the overlapping area is d, and for each pixel in the overlapping area, its relative position in the overlapping area is d′, and the expression for allocating weights is:

[0131]

[0132]

[0133] Among them, w1 and w2 are the weight coefficients of the corresponding pixels respectively.

[0134] In some embodiments, the method further includes: calculating a shooting distance between two marking points in the captured image, comparing the shooting distance with the actual distance calculated by the infrared camera, and then adjusting the TOD value to reduce the distance difference between the shooting distance and the actual distance;

[0135] Wherein, the calculation formula of the distance difference s is:

[0136] Among them, D i represents the shooting distance between the two marked points in the captured image; D represents the actual distance calculated by the infrared camera.

[0137] The following describes the specific implementation process of the method of the present invention in detail by taking the implementation process in a specific scenario as an example:

[0138] ① The location of the two infrared marking points

[0139] refer to Figure 3 , the patient stands on the pedal in an upright position in the anterior-posterior position, with the back close to the photography stand. The knee joint is straightened as much as possible, and the femur and tibia and fibula are in the same vertical direction, so that the distance from the femur and tibia and fibula to the flat-panel detector is equal, that is, the TOD of all lower limb bones is consistent. Measure the distance L from the iliac spine to the lateral malleolus of the ankle joint. The size of the flat-panel detector is C*C, that is, the photography length of each image is C.

[0140] The machine takes three images in total and splices them two by two to form a full-length image of the lower limb. The length of the overlap between each two images can be calculated as Therefore, the first infrared marker is pasted in the overlapping area of ​​the first image and the second image, and the second infrared marker is pasted in the overlapping area of ​​the second image and the third image. The distance between the first marker and the iliac spine is d1 = C-0.5*d, and the distance between the second marker and the iliac spine is d2 = 2*C-1.5*d.

[0141] Two infrared markers are respectively pasted on the femur and tibia and fibula of one side (left or right) of the lower limb. During the pasting process, the two infrared markers are required to be in the same vertical direction. The line connecting the two infrared markers is parallel to the flat panel detector.

[0142] ②Infrared camera reads the marking point

[0143] The infrared camera is placed on the bracket, on the same side of the two infrared marking points, and can clearly capture the two marking points. The infrared camera reads the coordinates of the two marking points, which are (x1, y1) and (x2, y2). Calculate the actual distance between the two marking points

[0144] ③ Image stitching and seamless fusion

[0145] refer to Figure 4 , start the X-ray exposure button, and shoot the entire lower limb in three sections, with each section shooting length C. The first image and the second image simultaneously shoot the first marked point, and the second image and the third image simultaneously shoot the second marked point. The marked point is highlighted in the image for easy subsequent processing.

[0146] In order to achieve accurate stitching, the Canny algorithm is used to detect the edges of the marker points. Before performing edge detection, the noise in the image must be removed. In order to reduce the interference of noise on edge detection, the Canny algorithm usually uses a Gaussian filter to smooth the image.

[0147]

[0148] Where G(x, y) represents the Gaussian function value of the point (x, y) on the two-dimensional plane, where x and y represent the horizontal and vertical coordinates of the pixel in the image, respectively, with the coordinates of the upper left corner of the image defined as (0, 0), and σ is the standard deviation of the Gaussian filter.

[0149] After Gaussian filtering, the Sobel operator is used to calculate the gradient strength and direction of each pixel in the image. The Sobel operator detects edges by calculating the derivatives in the horizontal and vertical directions respectively. The horizontal and vertical derivatives are implemented through convolution operations:

[0150] Horizontal direction:

[0151]

[0152] Vertical direction:

[0153]

[0154] Gradient Strength:

[0155]

[0156] Gradient direction:

[0157]

[0158] The non-maximum suppression method is used to refine the edges and only retain the points with the maximum local gradient. For each pixel, it is compared with the adjacent pixels in the gradient direction. If the gradient value of the pixel is not the local maximum, it is suppressed to 0.

[0159] Set two thresholds (high threshold and low threshold) to perform dual threshold detection. Classify pixels according to their gradient strength, which are strong edge: pixels with gradient strength greater than the high threshold; weak edge: pixels with gradient strength between the high threshold and the low threshold; non-edge: pixels with gradient strength less than the low threshold, which are directly removed.

[0160] Connect weak edge pixels to strong edge pixels. If a weak edge pixel is adjacent to a strong edge, the pixel is retained as an edge, otherwise it is suppressed as a non-edge pixel. Therefore, the Canny edge detection algorithm can effectively extract the edges of the highlighted markers in the image and form a clear outline around the markers.

[0161] After detecting the edge of the marker point, determine the smallest rectangular area surrounding the marker point, that is, the region of interest. In this area, traverse each pixel, find the point with the highest intensity value, and record the coordinate position (x, y) of the pixel.

[0162] That is, in the first image, the coordinates of the point with the highest intensity value in the marked point area are (x1, y1), and in the second image, the coordinates of the point with the highest intensity value in the same marked point area are (x2, y2). Since these two coordinates are relative to the coordinate system of the images they are in, a common coordinate system needs to be established.

[0163] The coordinates of the upper left corner of the first image are (0, 0), and the second image is offset from the upper left corner of the first image by (0, Cd). The coordinates (x2, y2) of the second image can be expressed in the coordinate system of the first image as follows:

[0164] x′2=x2

[0165] y2′=y2+Cd

[0166] The relative translation between the markers can be calculated:

[0167] Δx=x1-x′2

[0168] Δy=y1-y2′

[0169] The second image is translated so that the marked points (x1, y1) and (x′2, y2′) overlap, thereby achieving the stitching of the two images.

[0170] Similarly, the second image and the third image can be stitched together using the above method. The coordinates of the upper left corner of the first image are (0, 0), then the coordinates of the first marker point are (x1, y1), and the coordinates of the second marker point are (x3, y3).

[0171] In the process of image stitching, in addition to aligning the markers by coordinate translation, it is also necessary to deal with the brightness and color inconsistencies that may occur around the stitching line to achieve seamless stitching. To this end, the weighted average fusion method is used to smoothly transition the color and brightness of the overlapping area, reduce the obviousness of the stitching line, and ensure a natural transition in the stitching area.

[0172] Specifically, the weighted average fusion method performs weighted processing on the pixel values ​​of the overlapping areas. The pixel values ​​of the overlapping areas are I1(x, y) and I2(x, y). Among them, I1(x, y) represents the pixel value of the first image, and its upper left corner coordinate is set to (0, 0). The value range of x is from 0 to A, and the value range of y is from I2(x, y) represents the pixel value of the second image, and sets its upper left corner coordinate to (0, 0). The value range of x is from 0 to A, and the value range of y is from 0 to We perform weighted calculation using weighted coefficients w1 and w2 to obtain the fused pixel value L fuse (x, y):

[0173] I fuse =w1·I1(x,y)+w2·I2(x,y)

[0174] Among them, w1 and w2 are weight coefficients of corresponding pixels, satisfying w1+w2=1. Generally, the farther away from the marker point, the smaller the weight of the pixel, and the closer to the marker point, the larger the weight of the pixel. Through this gradual and smooth transition, the brightness and color differences caused by the splicing line can be effectively eliminated.

[0175] The weight coefficient can be allocated based on the distance from the pixel to the boundary of the overlapping area. The width of the overlapping area is d, and for each pixel in the overlapping area, its relative position in the overlapping area is d'. The weight can be allocated in the following way:

[0176]

[0177] This linear weight distribution method can effectively smooth the transition between images and reduce the visibility of the stitching line. Through this weighted average fusion method, the overlapping areas of the images are smoothed, which not only ensures the geometric consistency of the images after the markers are aligned, but also greatly improves the transition effect of brightness and color, ensuring the naturalness of the stitching and the overall quality of the image.

[0178] ④ Adjust TOD

[0179] Automatically calculate the distance D between two marked points (x1, y1) and (x3, y3) in the image i , and compare it with the actual distance D calculated by the infrared camera. Due to inaccurate TOD setting, the image taken does not match the patient's leg length. Adjust the TOD value to reduce D i The difference between s and D.

[0180]

[0181] When s is less than the threshold α, it is considered that the distance measured on the image is consistent with the actual distance.

[0182] In summary, the method for stitching whole lower limb images provided by the embodiment of the present invention has the following advantages:

[0183] The invention can accurately set TOD, truly reflect the actual length, and improve accuracy while reducing splicing errors, providing better image quality and measurement accuracy for clinical practice. This innovation can not only improve the accuracy of doctors' diagnosis of lower limb bones, but also provide patients with better treatment and rehabilitation programs.

[0184] Another aspect of the embodiment of the present invention further provides a system for stitching images of the entire lower limb, comprising:

[0185] The first module is used to determine infrared marking points on the target object;

[0186] The second module is used to read the coordinate information of the infrared marking point through an infrared camera;

[0187] The third module is used to photograph the entire lower limb of the target object based on the coordinate information to obtain a photographed image containing each marking point;

[0188] The fourth module is used to perform edge detection on the captured images of each marking point;

[0189] The fifth module is used to determine the minimum rectangular area surrounding the marker point after detecting the edge of the marker point to obtain the region of interest;

[0190] The sixth module is used to stitch the captured images together according to the region of interest to obtain a stitching result of the entire lower limb image.

[0191] It can be understood that the contents of the above method embodiments are all applicable to the present system embodiments, the functions specifically implemented by the present system embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0192] The embodiment of the present invention further provides an electronic device, the electronic device comprising a memory and a processor, the memory storing a computer program, and the processor implementing the above-mentioned method for stitching all lower limb images when executing the computer program. The electronic device can be any intelligent terminal including a tablet computer, a car computer, etc.

[0193] It can be understood that the contents of the above method embodiments are all applicable to the present device embodiments, the functions specifically implemented by the present device embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0194] See also Figure 5 , Figure 5 The hardware structure of an electronic device of another embodiment is illustrated, and the electronic device includes:

[0195] The processor 501 may be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of the present invention.

[0196] The memory 502 can be implemented in the form of a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 502 can store an operating system and other application programs. When the technical solution provided in the embodiments of this specification is implemented by software or firmware, the relevant program code is stored in the memory 502, and the processor 501 calls and executes the whole lower limb image stitching method of the embodiment of the present invention;

[0197] Input / output interface 503, used to implement information input and output;

[0198] Communication interface 504, used to realize communication interaction between the device and other devices, which can be realized through wired mode (such as USB, network cable, etc.) or wireless mode (such as mobile network, WI FI, Bluetooth, etc.);

[0199] A bus 505 that transmits information between the various components of the device (e.g., the processor 501, the memory 502, the input / output interface 503, and the communication interface 504);

[0200] The processor 501 , the memory 502 , the input / output interface 503 and the communication interface 504 are connected to each other in communication within the device via the bus 505 .

[0201] An embodiment of the present invention further provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the above-mentioned method for stitching the entire lower limb image is implemented.

[0202] It can be understood that the contents of the above method embodiments are all applicable to the present storage medium embodiments, the functions specifically implemented by the present storage medium embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0203] The memory, as a non-transient computer-readable storage medium, can be used to store non-transient software programs and non-transient computer executable programs. In addition, the memory may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory may optionally include a memory remotely disposed relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0204] It should be noted that in various specific embodiments of the present invention, when it comes to the need to perform relevant processing based on data related to user identity or characteristics such as user information, user behavior data, user historical data, and user location information, the user's permission or consent will be obtained first, and the collection, use, and processing of these data will comply with relevant laws, regulations, and standards. In addition, when the embodiment of the present invention needs to obtain the user's sensitive personal information, it will obtain the user's separate permission or consent through a pop-up window or jump to a confirmation page, and after clearly obtaining the user's separate permission or consent, it will obtain the necessary user-related data for the normal operation of the embodiment of the present invention.

[0205] The embodiments described in the embodiments of the present invention are intended to more clearly illustrate the technical solutions of the embodiments of the present invention, and do not constitute a limitation on the technical solutions provided by the embodiments of the present invention. Those skilled in the art can appreciate that with the evolution of technology and the emergence of new application scenarios, the technical solutions provided by the embodiments of the present invention are also applicable to similar technical problems.

[0206] Those skilled in the art will appreciate that the technical solutions shown in the figures do not limit the embodiments of the present invention and may include more or fewer steps than shown in the figures, or a combination of certain steps, or different steps.

[0207] The device embodiments described above are merely illustrative, and the units described as separate components may or may not be physically separated, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0208] Those skilled in the art will appreciate that all or some of the steps in the methods disclosed above, and the functional modules / units in the systems and devices may be implemented as software, firmware, hardware, or a suitable combination thereof.

[0209] The terms "first", "second", "third", "fourth", etc. (if any) in the specification of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0210] It should be understood that in the present invention, "at least one (item)" refers to one or more, and "plurality" refers to two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can represent: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship. "At least one of the following items" or similar expressions refers to any combination of these items, including any combination of single items or plural items. For example, at least one of a, b or c can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0211] In the several embodiments provided by the present invention, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the above units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0212] The units described above as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0213] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0214] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, including multiple instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the methods of various embodiments of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk, etc., various media that can store programs.

[0215] The preferred embodiments of the present invention are described above with reference to the accompanying drawings, but the scope of the rights of the embodiments of the present invention is not limited thereby. Any modification, equivalent substitution and improvement made by those skilled in the art without departing from the scope and essence of the embodiments of the present invention shall be within the scope of the rights of the embodiments of the present invention.

Claims

1. A method for stitching whole lower limb images, characterized in that: The following steps are involved: Determine infrared marker points on the target object; Read the coordinate information of the infrared marking point through an infrared camera; Based on the coordinate information, photograph the entire lower limb of the target object to obtain a photographed image containing each marked point; Perform edge detection on the captured images of each marking point; After detecting the edge of the marker point, determine the minimum rectangular area surrounding the marker point to obtain the region of interest; According to the region of interest, each captured image is stitched together to obtain a stitching result of the entire lower limb image.

2. The method for stitching whole lower limb images according to claim 1, characterized in that: Determining the infrared marking point on the target object comprises the following steps: Have the subject stand upright on the pedal in a fore-aft position with his back against the camera stand; The knee joint of the target subject is straightened, and the femur and tibia and fibula are in the same vertical direction, so that the distances from the femur and tibia and fibula to the flat-panel detector are equal; Three images were taken and spliced ​​in pairs to form a full-length image of the lower limb, and the overlapping length of each two images was calculated; then the first infrared marking point was pasted in the overlapping area between the first image and the second image, and the second infrared marking point was pasted in the overlapping area between the second image and the third image; the two infrared marking points were respectively pasted on the femur and tibia and fibula of one lower limb, and during the pasting process, the two infrared marking points were in the same vertical direction; the line connecting the two infrared marking points was parallel to the flat-panel detector.

3. The method for stitching whole lower limb images according to claim 1, characterized in that: The step of reading the coordinate information of the infrared marking point by using an infrared camera comprises the following steps: Place the infrared camera on the bracket, on the same side of the two infrared marking points; The coordinates of two infrared markers are read by an infrared camera, and the actual distance between the two infrared markers is calculated; The calculation formula of the actual distance D is: Among them, (x1, y1) and (x2, y2) represent the coordinates of two infrared marking points respectively.

4. The method for stitching whole lower limb images according to claim 1, characterized in that: Based on the coordinate information, photographing the entire lower limb of the target object to obtain a photographed image containing each marking point includes the following steps: The entire lower limbs of the target subject were photographed in three sections through X-ray exposure; the length of each section was 42 cm; The first infrared marking point is captured by the first image and the second image at the same time, and the second infrared marking point is captured by the second image and the third image at the same time; the infrared marking point is highlighted in the image.

5. The method for stitching whole lower limb images according to claim 1, characterized in that: The edge detection of the captured image of each marking point includes the following steps: Use Gaussian filter to smooth the captured images; After smoothing, the Sobel operator is used to calculate the gradient strength and direction of each pixel in the captured image, and the non-maximum suppression method is used to refine the edges; For each pixel, compare it with the adjacent pixels in the gradient direction. If the gradient value of the pixel is not the local maximum, suppress it to 0. Two thresholds are set as a high threshold and a low threshold, respectively, to perform dual threshold detection, and classify pixels according to their gradient strength, determining pixels with gradient strength greater than the high threshold as strong edges; pixels with gradient strength between the high threshold and the low threshold as weak edges; pixels with gradient strength less than the low threshold as non-edges, and directly removing the non-edges; Connect weak edge pixels to strong edge pixels. If a weak edge pixel is adjacent to a strong edge, the pixel is retained as an edge, otherwise it is suppressed as a non-edge pixel. The edges of the highlighted infrared marker points in the captured image are extracted to form a contour around the infrared marker points.

6. The method for stitching whole lower limb images according to claim 5, characterized in that: The expression for smoothing the captured image using a Gaussian filter is: Where G(x, y) represents the Gaussian function value of the point (x, y) on the two-dimensional plane, x and y represent the horizontal and vertical coordinates of the pixel in the image, respectively, and σ is the standard deviation of the Gaussian filter; After the smoothing process is completed, the Sobel operator is used to calculate the gradient strength and direction of each pixel in the captured image, and the non-maximum suppression method is used to refine the edge. In this step, the Sobel operator detects the edge by calculating the derivatives in the horizontal direction and the vertical direction respectively, wherein the derivatives in the horizontal direction and the vertical direction are realized by the convolution operation. Horizontal direction G x The expression is: Vertical direction G y The expression is: The expression of gradient strength G is: The expression of the gradient direction θ is:

7. The method for stitching whole lower limb images according to claim 1, characterized in that: The step of stitching the captured images according to the region of interest to obtain a stitching result of the whole lower limb image comprises the following steps: The coordinates of the upper left corner of the first image are (0, 0), and the second image is offset from the upper left corner of the first image by (0, Cd); wherein the coordinates (x2, y2) of the second image can be expressed in the coordinate system of the first image as follows: x ′ 2=x2 y2 ′ =y2+Cd Calculate the relative translation between the markers: Δx=x1-x ′ 2 Δy=y1-y2 ′ By translating the second image, the marker points (x1, y1) and (x ′ 2,y2 ′ ) overlap to achieve the splicing of the first image and the second image, and then complete the splicing between the second image and the third image; In the process of image stitching, the weighted average fusion method is used to smoothly transition the color and brightness of the overlapping areas; the process specifically includes the following steps: The weighted average fusion method performs weighted processing on the pixel values ​​of the overlapping area. The pixel values ​​of the overlapping area are I1(x, y) and I2(x, y), respectively. I1(x, y) represents the pixel value of the first image. The coordinates of the upper left corner of the first image are set to (0, 0). The value range of x is from 0 to A, and the value range of y is from to A, I2(x, y) represents the pixel value of the second image, and the upper left corner coordinates of the second image are set to (0, 0), the value range of x is from 0 to A, and the value range of y is from 0 to The weighted calculation is performed through the weight coefficients w1 and w2 to obtain the fused pixel value I fuse : I fuse =w1·I1(x,y)+w2·I2(x,y), In the allocation of weighted coefficients, the weight coefficients are allocated according to the distance from the pixel to the boundary of the overlapping area; where the width of the overlapping area is d, and for each pixel in the overlapping area, its relative position in the overlapping area is d′, and the expression for allocating weights is: Among them, w1 and w2 are the weight coefficients of the corresponding pixels respectively.

8. The method for stitching whole lower limb images according to claim 1, characterized in that: The method further includes: calculating the shooting distance between the two marking points in the shot image, comparing it with the actual distance calculated by the infrared camera, and then adjusting the TOD value to reduce the distance difference between the shooting distance and the actual distance; Wherein, the calculation formula of the distance difference s is: Among them, D i represents the shooting distance between the two marked points in the captured image; D represents the actual distance calculated by the infrared camera.

9. A system for stitching images of the entire lower limb, characterized in that: include: The first module is used to determine infrared marking points on the target object; The second module is used to read the coordinate information of the infrared marking point through an infrared camera; The third module is used to photograph the entire lower limb of the target object based on the coordinate information to obtain a photographed image containing each marking point; The fourth module is used to perform edge detection on the captured images of each marking point; The fifth module is used to determine the minimum rectangular area surrounding the marker point after detecting the edge of the marker point to obtain the region of interest; The sixth module is used to stitch the captured images together according to the region of interest to obtain a stitching result of the entire lower limb image.

10. An electronic device, characterized in that: including a processor and a memory; The memory is used to store programs; The processor executes the program to implement the method according to any one of claims 1 to 8.

Images