Surgical position positioning method and device

Abstract

The invention provides an operation position positioning method and a positioning device, which specifically comprises the following steps of: introducing an additional reference object near an operation position, capturing and identifying the outline and characteristics of the reference object by a computer, and selecting a specific characteristic forming the outline shape of the reference object. And then moving the camera device, or moving the reference object under the camera, then obtaining a scaling ratio of the X-ray film or the video image according to a front-back change configuration algorithm of specific characteristics, and finally performing equal-ratio scaling on the X-ray image and / or the video image according to the scaling ratio, so that the reference object in the X-ray film and the reference object in the video image coincide and align. And an operation position can be found accurately and positioning is completed. During an operation, the X-ray film and the video image are overlapped and aligned through the reference object and the change of the specific characteristics, so that the operation position is accurately found according to the image, and therefore, the extraction of tiny objects and the positioning accuracy of the position of the implantation operation are remarkably improved.

Description

Technical Field

[0001] The present invention relates to the field of positioning technology in medical surgical treatment, and specifically to a method of introducing a reference object and determining the scaling ratio of an image or picture based on changes in specific characteristics of the reference object in different states, so as to make the two overlap to improve the positioning accuracy of the hand position, as well as a positioning device used in this method. Background Art

[0002] The statements in this section merely provide background information related to the present disclosure and may not constitute prior art.

[0003] In surgical operations, especially orthopedics, spinal surgery, trauma repair and other operations involving bone structures or metal implants in the body, X-ray imaging positioning is a traditional and core technical means. By taking X-rays during surgery (such as using C-arm machines, G-arm machines and other equipment), doctors can obtain real-time positional information on the patient's skeletal anatomical structure, X-ray-opaque tissues (such as calcifications, stones) and metal substances (such as internal fixation plates, screws, foreign bodies) in the body. This type of image uses the penetrating properties of X-rays to convert density differences inside the human body into grayscale images. In this way, deep structures that cannot be directly observed by the traditional naked eye or optical cameras can be clearly displayed, providing a key basis for surgical path planning, implant positioning, and lesion identification.

[0004] Although X-ray images can provide important internal structure information, they have a fundamental flaw in clinical applications: the X-ray images are separated from the actual visual scene of the surgical field (i.e., the patient's body surface and superficial tissues observed by the camera or the naked eye).

[0005] Specifically:

[0006] 1) The separation of spatial dimension and perspective: X-ray images are essentially two-dimensional projections (compressing three-dimensional structures into a plane), and the shooting angle is limited by the equipment (such as anteroposterior and lateral views). Doctors need to reconstruct three-dimensional spatial relationships in their minds through experience; moreover, the visual images in the surgical field of view (such as the patient's skin incision, muscle tissue, etc.) are intuitive three-dimensional scenes, but can only display the surface and shallow structures of the body, and cannot directly reflect the position of deep bones or foreign objects. The lack of an intuitive spatial correspondence between the two causes doctors to frequently switch between observing the X-ray screen and the surgical incision during surgery, relying on spatial imagination to forcibly associate the "abstract grayscale image" with the "actual tissue position", which is prone to positioning errors;

[0007] 2) Visual blind spots for precise positioning: During foreign body removal, X-rays can show the location of metal foreign bodies, but cannot directly indicate their depth in the subcutaneous tissue and their projection on the body surface, which may lead to deviation of the incision position or excessive tissue peeling.

[0008] 3) Limitations on surgical efficiency and safety: To compensate for positioning deviations, doctors may need to take multiple X-rays (increasing the risk of radiation exposure), expand the surgical incision (increasing trauma), or prolong the operation time. These problems are particularly prominent when dealing with complex anatomical structures (such as joints and pelvis) or tiny lesions (such as bone tumors). Summary of the Invention

[0009] In view of this, the present invention provides a method for solving at least one of the above problems.

[0010] To address the above technical issues, the first aspect of the present invention is to provide a method for locating a surgical position. This method is applied to surgeries performed using C-arm imaging. Specifically, an additional reference object is introduced near the surgical site. A computer captures and identifies the outline and features of the reference object and selects a specific feature that constitutes the reference object's outline shape. Subsequently, the camera device is moved, or the reference object is moved under the camera. An algorithm is then configured based on the before-and-after changes in the specific feature to determine the scaling ratio of the X-ray or video image. Finally, the X-ray image and / or video image is scaled proportionally based on this ratio so that the reference objects in the X-ray and video images coincide and align, facilitating accurate identification of the surgical position and completing positioning.

[0011] Regarding scaling, the first aspect of this solution states that when the camera's travel distance is short, the pixel values of a specific feature in the image before and after the camera's movement are obtained, and the scaling ratio is determined based on the ratio of the before and after pixel values. Furthermore, the first aspect of this solution states that when the camera or reference object's travel distance is long, the impact of camera focal length drift on imaging should be considered. Correction parameters are introduced when determining the scaling ratio to correct for both the field of view scaling ratio and the optical scaling ratio.

[0012] Another preferred embodiment of the first aspect of the present solution is that the space at the surgical site is limited and a carrier cannot be set to carry the reference object, and the reference object may be tilted according to the needs of the surgery. In response to this situation, the present solution points out that the inner angle of the reference object is selected as a specific feature, and the image or picture is horizontally corrected according to the front and back changes of the angle.

[0013] The second aspect of this solution is to provide the solution described in the first aspect of the present invention, and configure a positioning device, wherein a positioning block with two in-line lasers is set at the edge position of the C-arm imaging plate. The laser beams of the two lasers converge to form a cross-shaped calibration line. The cross-shaped calibration line is projected at the surgical position and aligned with the center position of the reference object to achieve surgical position positioning.

[0014] Compared with the existing technology, the beneficial technical effect achieved by implementing the technical solution of this scheme is: during the operation, the X-ray film and the video image are superimposed and aligned through the changes of reference objects and specific features to accurately locate the surgical position according to the image. In this way, the extraction and implantation of tiny objects have been significantly improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 Schematic diagram, exemplarily showing the main view structure of the camera positioning device described in the fifth embodiment of the present invention;

[0016] Figure 2 is a schematic diagram, which exemplarily shows Figure 1 The side view structure of the camera positioning device shown;

[0017] Figure 3 This is a picture showing the positioning and alignment of medial malleolus fractures of an orthopedic ankle joint performed according to a preferred embodiment of the present invention;

[0018] Figure 4 2 is a picture showing the positioning and alignment of the distal screw locking hole of the orthopedic intramedullary nail according to a preferred embodiment of the present invention. DETAILED DESCRIPTION

[0019] One of the important reasons why it is difficult to extract tiny objects from human tissue is the limited surgical space near the extraction site. This spatial limitation means that during extraction or implantation operations at locations where human tissue is dense, such as the finger or ankle joints, or where the bone structure is complex, the operating space near the joint is narrow. At the same time, the equipment used for extraction and implantation is generally long straight or extended pieces. The resulting lack of intraoperative operating space will further amplify the problem of poor surgical positioning. In addition, the limited operating space also means that when positioning deviates, the adjustable space is also limited, which in turn leads to an increase in the frequency of secondary or even multiple positioning attempts. Obviously, improving the accuracy of the first positioning and reducing the number of position adjustments are the first problems to be solved.

[0020] One possible approach is to borrow ideas from surveying and mapping schemes used in some civil engineering construction projects, which generate images of measurement and construction objects. This approach could generate markers, such as rulers and scales, at specific locations (center or edge) on the output X-ray film to aid positioning. This would allow the measurement of parameters such as the size and distance of the measured location or target object in the image, and then compare the acquired parameters to determine the precise location for the final surgery. However, these markers cannot fundamentally compensate for the shortcomings of grayscale X-ray imaging. Furthermore, the distances and coordinates generated by these markers still need to be converted based on variables such as the camera's shooting distance and the angle relative to the target. Furthermore, in civil engineering construction, the orientation and position of the target object are often fixed. However, during surgery, not only is the shooting position not always fixed, but the shooting angle may also be adjusted depending on the surgical needs. Therefore, implementing this borrowed approach would be difficult to fundamentally reduce positioning errors.

[0021] Modern precision surgery (such as minimally invasive surgery and robot-assisted surgery) places higher demands on positioning accuracy. It is necessary to superimpose "deep structure information (X-ray images)" with "visual scenes" in real time and intuitively to form a fused image where "what you see is what you need." Specific requirements include:

[0022] a) Spatial registration requirements: Image fusion technology is used to match the coordinates of bones / foreign bodies in X-ray images with the body surface coordinates in the camera visual image, allowing doctors to see the position mapping of deep structures in the actual tissue in the same image;

[0023] b) Clinical value: It reduces the reliance on the physician's spatial imagination and reduces human error, especially for surgeries requiring X-ray C-arm fluoroscopy (extraction of foreign bodies and tumor tissues from soft tissues, and placement of implants in human tissues, including internal fixation of medial and lateral ankle fractures with screws, internal fixation of femoral neck screws, distal locking screw placement of femoral intramedullary nails, internal fixation of spinal pedicle screws, and tumor puncture positioning biopsy).

[0024] With the development of video imaging technology, more surgeries can use real-time video images to replace the previous single radiography method. The human tissue structure has a higher color contrast under real-time images, which makes the boundaries between human tissues in the image more distinct. For example, a more common application is that the C-arm X-ray machine, as a workstation-type device with both image capture and image processing, can collect image data in real time during surgery. With the help of this device, the surgical position can be calibrated and positioned in real time according to the video image during the specific operation. The final effect of this calibration and positioning can be understood as aligning the position of the foreign body in the X-ray film with the actual body part of the patient in the surgical field of view. This alignment method has a certain degree of accuracy.

[0025] However, since the image acquisition direction of the C-arm is roughly top-down, and the actual surgical position is generally located below the image acquisition direction, the imaging position of the C-arm is prone to possible obstruction in the line of sight imaging. In addition, since the X-ray image shooting assembly of the C-arm and its video image camera shooting assembly are essentially independent and different units, there are inevitable errors in the fusion between the two-dimensional image and the three-dimensional human body. This error will make it difficult to visually overlap the same feature displayed in the X-ray and the video. That is, there is a misalignment between the position of the foreign body in the X-ray and the actual body part that is difficult to overlap. For this reason, it is impossible to determine whether the current position of the surgical equipment has been accurately located, especially in small joints and delicate tissues. The inability to calibrate and overlap the position will make the entire operation difficult to perform. In addition, considering that during actual surgery, the existing technology still recognizes that the reference of images and videos is still two independently executed steps, even after the introduction of C-arm imaging, it still relies on the early method of comparing two-dimensional images of X-ray films to find positioning in real-time videos. Therefore, in general, the core problems faced by intraoperative image fusion technology are roughly as follows: in terms of multimodal image registration accuracy, the differences in imaging principles between X-ray images and visual images (X-rays vs. visible light), and differences in tissue contrast (density vs. color) make spatial coordinate matching difficult; in terms of real-time performance and equipment integration, traditional X-ray equipment (such as C-arm) imaging and visual camera imaging equipment are not integrated, and there is a lack of fusion between X-ray photos and camera video images; for images and videos that are difficult to overlap, the introduction of C-arm imaging has not made intraoperative positioning more convenient.

[0026] In view of the above content, the basic idea of the preferred embodiment of the present invention to solve its technical problem is that, since it is difficult to compare the small features contained in human tissue, it is possible to try to introduce additional significant reference objects within the shooting field of view as common reference features for pictures and videos under the imaging perspective. Since the style and specifications of the reference features (such as specific length, width, and angle) can be pre-configured or known, that is, the actual size of the reference features is known and fixed, then, according to the changes in the specific features of the reference objects at different viewing distances or field of view angles, the size of the original X-ray image features and / or video stream is scaled so that the two visually overlap. This visual overlap can be understood as the specific position shown in the X-ray film being aligned with the position in the actual image. Then, using the reference object as a comparison, the specific position of the operation can be determined.

[0027] The following description of embodiments of the present invention will be made with reference to the accompanying drawings. Those skilled in the art will recognize that the described embodiments may be modified in various ways without departing from the spirit and scope of the present invention. Therefore, the drawings and description are illustrative in nature and are not intended to limit the scope of the claims. Furthermore, throughout this specification, the drawings are not drawn to scale, and like reference numerals represent like parts.

[0028] It should be noted that the expressions "first" and "second" used in the embodiments of the present invention are intended to distinguish two non-identical entities with the same name or non-identical parameters. It can be seen that "first" and "second" are only for the convenience of expression and should not be understood as limitations on the embodiments of the invention. Subsequent embodiments will not explain this one by one.

[0029] Example 1

[0030] Guided by the aforementioned technical principles of the present invention, several preferred embodiments introduce a reference object and, based on the corresponding changes in its specific features as the viewing distance changes, obtain a corresponding image scaling ratio. This ratio is then used to synchronize the scaling of the X-ray image with the change in viewing distance in the video until the two are fused. Image and video fusion can be considered to maintain the same position of the same feature at the surgical site in both the image and the video, or, more simply, to overlap the static or dynamic images of the image and video when they are overlapped. Simply put, the relationship between size change and scaling ratio is that, for example, as the C-arm camera assembly moves closer to the target location, the size of the reference object in the video gradually increases, and vice versa. This characteristic clearly indicates that when the reference object increases in size, the image scaling ratio should be greater than 1, and the X-ray film is enlarged by the corresponding ratio. Conversely, when the reference object decreases in size, the image scaling ratio should be less than 1, and the X-ray film is reduced by the corresponding ratio. However, the above-described scaling ratio determination process is only theoretical, and determining the scaling ratio still faces at least the following challenges:

[0031] 1) The scaling ratio is determined based on the change in the size of the reference object in the field of view. However, taking a specific polygon as the reference object as an example, since the reference object contains multiple identical or different edges, the scaling ratio should be determined based on the change in the same edge of the static reference object after the camera is displaced. In addition to requiring the reference object to be a regular geometric shape with a closed structure, it must also have unique and identifiable features, such as the longest side, or a specific oblique side or diagonal angle, etc.

[0032] 2) Another question based on the first point is how to enable the computer to not only identify the reference object in the picture, but also find the specific features of the reference object; and to adaptively complete the above-mentioned ratio calculation and picture scaling process.

[0033] This preferred embodiment of the present invention selects a regular equilateral triangle as the reference object. This is because an equilateral triangle has two equal sides and a long, straight hypotenuse. These characteristics make the reference object configured as an equilateral triangle easier for the computer to capture and recognize. Furthermore, the reference object should be made of metal or any other material that can block X-ray penetration, so that its structure can still be accurately displayed under X-rays.

[0034] In the first embodiment, after selecting and determining the reference object, the next step is to keep the reference object horizontal. The purpose of placing the reference object horizontally is:

[0035] 1) As the C-arm moves and adjusts its angle, a horizontal reference object can always determine the plane in the height direction of the surgical position;

[0036] 2) In some surgeries, the specific location of the extraction or implantation of a microorganism requires not only horizontal positioning but also vertical, or depth, calibration. However, the extraction or implantation direction is not always perpendicular to the horizontal position. In some surgeries, the directions of the extraction or implantation are often tilted at a certain angle to the positioning position. This requires that the reference plane should always remain horizontal when taking positive and lateral radiographs of the surgical site.

[0037] 3) Other scenarios involve multiple possible surgical locations. For example, multiple delicate objects are scattered across a specific area, or multiple implants are placed in different locations. In these cases, the reference object must be moved sequentially between these locations according to the surgical procedure. During image and video calibration, to improve positioning accuracy and facilitate calculations, it is always desirable that the selected parameters, after each movement, do not change in any direction other than the vertical within the field of view. This requires that the reference object remain horizontal after each movement.

[0038] According to the above-mentioned requirements of Example 1, the intraoperative reference object needs to maintain its horizontal position before completing each positioning. Therefore, in this embodiment, a specific carrier is configured for the reference object, and the carrier can keep the reference object near the surgical position. The preferred embodiments of the present invention do not limit the specific style and size of the carrier. Generally, when configuring the aforementioned carrier in different preferred embodiments, a transparent or non-transparent non-metallic carrier that can provide a horizontal supporting surface is prepared. Since these non-metallic carriers can be penetrated by X-rays, they will not be ultimately displayed on the X-ray film, or will only appear in the form of blurred lines. In this way, the selected metal reference object can be better highlighted in the image. For example, in an orthopedic ankle fracture positioning and implantation of a Kirschner wire, the carrier can be a medical drape covering the ankle bone; for another example, in some limb positioning surgeries, an opening piece or a channel piece made of transparent glass can be used and placed on the limb, and the opening piece or the channel piece is configured so that at least one side of the surface forms a flat horizontal surface, so that a reference object can be placed horizontally. A better reference object configuration method also includes setting the vertical height of the reference object to be flush with the height of the surgical site, or slightly higher than the height of the surgical site, so as to improve positioning accuracy.

[0039] After determining the shape of the reference object and keeping it horizontal, its accurate size parameters need to be measured in advance. For example, in the equilateral triangle reference object mentioned above, two equal edges can be used as specific features (specific edges), or the longest hypotenuse can be used as the specific feature. The selection methods of the two specific features are only slightly different in the computer algorithm recognition process, and there is no difference in the final effect. The part about computer recognition will be explained later, so it will not be expanded here. In this preferred embodiment, the longest hypotenuse of the equilateral triangle is used as the specific edge, and its accurate length value W1 needs to be measured in advance as a size parameter. After the camera moves closer to or away from the surgical position, the hypotenuse length value W2 of the reference object after the change in the shooting field of view is obtained again, and then a ratio R is obtained, and R should satisfy:

[0040] R=\frac{W1}{W2}(1)

[0041] It is understood that in a vertically captured image, the size change trend of the same feature before and after capture should be consistent with the overall image change trend. In this preferred embodiment, the ratio of the change in the length of a specific long side of an equilateral triangle before and after capture should also be consistent with the ratio of the change in the overall image captured by the camera before and after. Therefore, by scaling the overall image according to the change in the specific feature, when the video and image are superimposed, the image feature in the video is scaled to overlap with the image feature in the X-ray film. In this way, the surgical location indicated on the X-ray film is aligned with the location in the actual image.

[0042] In the first embodiment of the present invention, the ratio R is directly used as the ratio of image scaling, that is, according to the size change of the longest hypotenuse of the equilateral triangle before and after shooting, the picture presented by the camera is adjusted accordingly. For example, before and after shooting, when the camera moves close to the positioning position, the size of the longest hypotenuse in the picture should increase, that is, the value of R is greater than 1, and the picture is adjusted to zoom in with this ratio. If the camera moves to a long-distance positioning position, the size of the longest hypotenuse in the picture should decrease. At this time, the value of R is less than 1, and the picture should be zoomed out. It should be pointed out that in the preferred embodiment of the present invention, for obtaining the length values W1 and W2, there is no need to perform additional measurements on the features in the image or picture, nor is there a need to introduce a scale into the picture for conversion, because such measurement and conversion are limited by natural errors in vision or measurement and it is difficult to improve accuracy. The best way is to use the number of pixels occupied by the same feature at the same resolution (the number of pixels in the length direction) as a reference. After determining the idea of scaling, the next step is to enable the computer to identify the reference object and its specific features, and calculate the scaling value according to the above idea.

[0043] Prior art has enabled computers to capture and identify targets that meet specific requirements through machine learning. In a preferred embodiment of the present invention, the elements in an X-ray primarily include human tissue exhibiting varying light and dark contrasts, skeletal structures exhibiting the same or similar contrasts, and additional reference objects introduced and presented with complete outlines. Through machine learning, a computer can capture pre-configured reference objects and identify their specific, unique features.

[0044] Specifically. The X-ray film taken by the C-arm machine is sent to the host computer, and the host computer first performs grayscale processing on the X-ray film. In the grayscale-processed X-ray film, the reference object made of metal material is filled with a deep color with high discrimination and presents a complete shape and a clear outline. Subsequently, the grayscale-processed image is Gaussian blurred to remove unwanted noise in the grayscale-adjusted image, which not only facilitates computer capture but also reduces the subsequent calculation complexity. Then, the Canny edge detection algorithm is used to extract features from the image. Taking the equilateral triangle reference object in Example 1 as an example, when extracting features, the contour detection algorithm (cv2.findContours) is first used to obtain all closed contours in the current X-ray film. Then, the algorithm traverses each closed contour to determine whether the current contour meets the characteristics of the currently selected reference object. For example, in this preferred embodiment, the polygon approximation algorithm (cv2.approxPolyDP) is used to determine whether the contour is approximately a triangle, so as to obtain at least one contour that meets the characteristics of the selected reference object (equilateral triangle). Finally, the pixel values of each feature that makes up the current contour are obtained to infer its pixel length. In the first embodiment of the present invention, the length of the long side is captured and obtained. This is because, in an equilateral triangle reference object, there is a hypotenuse that is significantly longer than the other two sides, making it easier for the computer to capture and recognize.

[0045] Looking back at the previous section, when selecting the reference object as an equilateral triangle, Example 1 pointed out that there is a subtle difference in algorithm recognition between using two equal edges as specific features and using a single long hypotenuse as the specific feature. This subtle difference refers to the fact that when traversing the closed contour of the image and obtaining the contour feature pixel values, the former selection method requires determining the existence of two features with equal contour values, both of which have a pixel length less than the remaining third edge. The latter selection method first determines the existence of a hypotenuse with a length significantly greater than the other two edges, without determining whether the pixel lengths of the remaining two edges are equal. Compared with the two methods, when the reference object is a triangle, selecting the long hypotenuse as the specific feature can reduce the number of computer judgment steps to a certain extent. However, it should also be noted that due to the implementation of the configured recognition algorithm, the computer already knows the contour features of the captured object. Therefore, in some preferred embodiments, when the reference object is a polygon with fewer than four edges, it is possible that the algorithm must already know the existence of two features of equal pixel length in the current captured object during approximation. In view of this, when selecting specific features, those skilled in the art should prioritize their uniqueness or accuracy, and then consider the steps involved in computer execution. When the reference object has five or more edges, simplifying the algorithm execution steps should be prioritized, and then consider the uniqueness of the specific features.

[0046] Another aspect of reference object selection is that the equilateral triangle selected in Example 1 is merely an illustrative approach. Clearly, following the above approach, various other possible triangles, closed polygons thereof, or even non-closed polygons containing closed contours can theoretically be selected. Capturing contours of varying shapes requires only providing the corresponding library during the machine learning process and adaptively modifying the specific applicable methods in algorithms such as polygon approximation. However, the selection of reference objects is not arbitrary or random. For the same reasons as for selecting specific features, while complex contours improve the recognizability of reference objects, they also increase unnecessary computational complexity. Furthermore, the more complex the contour, the more difficult it is for the computer to select and determine unique specific features from it during learning. Therefore, the equilateral triangle is a preferred embodiment, balancing ease of recognition with ease of feature acquisition. Of course, in other preferred embodiments of the present invention, other preferred reference object options exist depending on the specific surgical location. For example, in some joint surgeries, there is a human skeletal structure with an approximately triangular structure at the surgical site. In this case, it is a better method to select a reference object such as a square or an isosceles body. In some embodiments, the reference object can be configured to have at least one color feature that is obviously different from the human tissue according to the color of the human tissue at the surgical site. The color feature can be represented by a hexadecimal mark or a color space function mark or other method that can be recognized by a computer. Therefore, the selected reference object should also be significantly different from other features at the positioning position. Obviously, for the specific style and specifications of the reference object, technical personnel in this field should be able to make flexible selections in accordance with certain principles, and always seek a balance between the difficulty of computer capture and recognition and the significance of the reference object in the video and image. The preferred embodiments in this field should not be limited by the specific format of the reference object.

[0047] After obtaining the scaling ratio R, the computer adjusts the current image of the C-arm according to the scaling ratio R. This step can be achieved by the host computer sending the obtained scaling ratio R value to the C-arm, and then relying on the image scaling function of the C-arm camera to scale the real-time image according to the scaling ratio. It can also be achieved by using known image processing software or an additional auxiliary computer program. For those skilled in the art, image scaling does not require additional intellectual effort once the scaling ratio is known.

[0048] After completing the above steps, the display unit of the C-arm machine will display the pre-taken X-ray film and the zoomed real-time image together, and from the presentation effect, the equilateral triangle reference in the X-ray film and the equilateral triangle reference in the real-time image completely overlap. A further significance of the reference object overlap is that the overlapping image accurately shows the exact position of the tissue and / or bone in the human body in the real-time image, and the operator can directly locate the surgical position through the overlapping image. Taking into account the possible shadows and grayscale conditions on the X-ray film, it is easy to think that the contrast of the displayed X-ray film can be adjusted by the C-arm camera unit assembly, so that according to the actual needs of the operation, the local features at the surgical position can be made more prominent, or some local features can be hidden to avoid them from blocking the calibration of the reference object or video imaging.

[0049] Example 2

[0050] In the first embodiment of the present invention, the long hypotenuse of an equilateral triangle is used as a specific feature, and the scaling ratio is determined according to the numerical changes of the specific features before and after the camera is moved, and finally the real-time image is scaled according to the scaling ratio. However, after long-term practice, it is found that the scaling ratio calculation method in the first embodiment is still an ideal situation. In this situation, it is assumed that the pixel length change of a specific feature before and after the camera is moved is equivalent to the image scaling ratio, and it is assumed that there is no error in the measurement or calculation of the specific pixel value. In fact, when the displacement value (length) before and after the camera is moved is short, the scaling ratio determined in the first embodiment can indeed ensure sufficient accuracy, and the specific features in the overlapping images can be completely overlapped, or only slight deviations appear in local positions in specific scenes. However, during surgery, affected by many factors such as changes in ambient light intensity, camera resolution, and shooting angle, as the camera assembly moves significantly, the camera autofocus will cause undesirable focal length drift (focal length error). The drifted camera focal length will cause errors in distance and length measurement. This error caused by the camera focal length offset will be further amplified because the reference object needs to be moved during some operations.

[0051] When the second embodiment of the present invention further improves the first embodiment, the primary aspect is that the best way to avoid errors is to minimize the number of specific features that determine the reference value by direct measurement, and try to indirectly obtain the scaling ratio through a specific value with a small error that can be determined. For example, according to the technical inspiration of the first embodiment, the parameter change of the specific feature of the reference object should satisfy the same proportional relationship as the change in the directly measured distance between the imaging position and the target part. The second embodiment recognizes that there are possible errors in the distance measurement from the camera to the target part. Even if some existing equipment has the function of sensing the distance in the vertical direction of the camera through existing means such as infrared, referring to the aforementioned focal length drift problem, it is difficult to ensure the accuracy of the sensing distance after moving over a long distance. More importantly, this strategy of measuring the distance through the camera requires that the camera always remain perpendicular to the positioned position, which is obviously impossible to achieve before the final positioning is completed.

[0052] Example 2 subsequently realized that the relationship between the pixel value of a specific feature in the image and the focal length after the drift should satisfy the same relationship as the relationship between the true value of the specific feature and the displacement distance of the reference object. Given this, since the specific feature value measured after the camera moves and the distance between the camera and the target have errors caused by focal length drift, one can try to keep the camera in a fixed position while horizontally moving the carrier carrying the reference object in the dynamic image, or directly move the reference object, and then calculate the focal length after the zoom drift based on the fixed focal length of the camera. Finally, based on the focal length, calculate the actual pixel length of the reference object in the image.

[0053] When solving the problem based on the above content, under the premise of knowing the actual length of the specific feature of the reference object, first move the reference object under the camera's field of view and record the change in pixel length before and after the reference object is moved. Then, based on the principle of triangulation and the lens imaging relationship, that is, the ratio of the object height (the width of the reference object) to the camera imaging height is equal to the ratio of the object distance to the image distance, then it should be:

[0054] L1×J1=H1×F(2)

[0055] Among them, L1 refers to the distance from the set reference object to the focal length of the camera lens, J1 is the pixel length value before the camera captures the reference object, H1 is the pre-measured specific characteristic parameter of the reference object (for example, the length of the hypotenuse of an equilateral triangle), and F is the initial focal length of the camera. Looking back at the above factors, in the above formula, there is a certain error in the measured length or sensed length of L1, so direct conversion with this data should be avoided as much as possible. In the state after the reference object moves, the displacement value generated before and after the reference object moves is set to D, which should also meet the following requirements:

[0056] L2×J2=H1×F,(3)

[0057] At the same time, L1+D=L2(4)

[0058] Among them, L2 refers to the distance from the reference object to the focal length of the camera lens after it moves, and J2 is the pixel length value captured by the camera after the reference object moves. Combining the above three equations and eliminating the distance from the reference object to the camera focal length with errors, the distance D of the reference object movement can be satisfied:

[0059] D=H1×F×(1 / J2 - 1 / J1)(5)

[0060] Then the real-time focal length Fx after drift satisfies:

[0061] Fx=D / 『H1×(1 / J2 - 1 / J1) (6)

[0062] It can be recognized that although formulas (2) and (3) include parameters with possible errors (the distance from the reference object to the camera), these parameters with possible errors are eliminated in formulas (5) and (6). For the drift focal length after zooming, it is only related to the movement distance, the actual value of the specific feature, and the accurate pixel length value of the specific feature before and after the movement. Comparing the above formulas, multiple sets of changing relationships are generated with the movement of the reference object within the field of view, for example, the distance from the reference object to the camera (L1 and L2), the pixel length of the reference object in the image frame (J1 and J2). These relationships can be regarded as correcting the initial value to obtain the calibration value as the reference object moves. Therefore, for the implementation of other preferred embodiments of the present invention, a correction coefficient can also be designed to correct the above relationship.

[0063] In the second embodiment, the correction coefficient is designed based on the distance from the reference object to the camera. Specifically, referring back to Formula 1, the theoretical value of the distance from the reference object to the camera can be expressed as:

[0064] L1=H1×F / J1(7)

[0065] Similarly, the theoretical value of the distance from the reference object to the camera after movement is expressed as:

[0066] L2=H1×F / J2(8)

[0067] Then, using formula (7) and formula (8) to express the theoretical value of the reference object's displacement D, we have:

[0068] D=L2-L1=H1×F / J2-H1×F / J1(9)

[0069] Measure the actual displacement distance E of the reference object, then the correction coefficient k can be designed as:

[0070] k=E / D(10)

[0071] In the second embodiment, the relationship between the actual displacement distance and the theoretical displacement value can be corrected by the coefficient k. Those skilled in the art should know that the relationship between the initial focal length and the real-time focal length can also be designed based on the coefficient k. For example, for the initial fixed focal length F of the camera, the correction amount of the focal length after the reference object moves can be obtained based on the correction coefficient, and the focal length value after drift can be obtained by combining the two. So far, the second embodiment of the present invention has obtained the real-time focal length of the camera and / or the correction coefficient k by the above-mentioned technical means. Any one of them can reduce the acquisition error of the reference object movement displacement. Then, the scaling ratio of the image is designed based on the parameters or coefficients obtained above. For example, based on the scaling coefficient R obtained in the first embodiment, the final scaling ratio S obtained after introducing the correction coefficient satisfies:

[0072] S=R×k(11)

[0073] Example 3

[0074] In order to eliminate the error in the measurement of the distance from the camera to the target caused by the drifting focal length, the second embodiment of the present invention is limited to keeping the camera position fixed, and moving the reference object horizontally in the dynamic picture, and calculating the real-time angular distance according to the change of the specific characteristics of the reference object, in order to obtain a correction coefficient, and using the correction coefficient to correct and calculate the final zoom ratio of the video image. However, it is difficult to always maintain the position of the camera at a fixed position. During actual surgery, according to the changes in the ambient light intensity, the changes in the surgical position, and the changes in the observation field angle, it is always necessary to make a certain floating adjustment to the position of the camera, or after the X-ray film and the video image are calibrated and fused for the first time, the reference object is moved again (in the vertical and horizontal directions). Therefore, when the camera can always be kept in a fixed position, the second embodiment can meet the positioning needs, while in other scenarios, further improvements to the second embodiment are needed.

[0075] Looking back at Example 1, Formula (1) has obtained a scaling factor R by comparing the pixel length of the reference object after the camera moves. In the case of a short-range camera movement, the real-time image can be scaled by this ratio R so that the image overlaps with the X-ray image. Based on the inspiration of Example 2, the scaling factor in Example 1 can be improved to:

[0076] If the camera moves towards the reference object (surgical position), the moving distance D of the reference object satisfies:

[0077] D = L / (R-1)(12);

[0078] If the camera moves away from the reference object (surgical position), the moving distance D of the reference object satisfies:

[0079] D = L × (1-1 / R) (13).

[0080] In the scenario defined in the first embodiment, the final zoom ratio S1 of the image can be expressed as:

[0081] S1=L1 / (L1+D)(14)

[0082] In the scenario defined in Example 2, considering the pixel length scaling ratio of the specific features of the reference object before and after the movement, it should be related to the reference object movement distance D, as well as the distance from the reference object to the camera before the movement and the camera focus distance F. Specifically, first define an imaging direction, which defines that the three directions from the imaging focus of the camera to the camera plane and then to the reference object are on a straight line. Then looking back at Example 1 with reference to the imaging principle (pinhole imaging), the distance from the imaging focus of the camera to the camera plane should be consistent with the relationship between the actual length value of the specific feature of the reference object and the pixel length value. Then under the initial imaging model, there are:

[0083] J1 / H=F / (FC)(15)

[0084] Where J1 is the pixel length of the specific feature of the reference object at the initial position (before the reference object moves), H is the actual measured value of the specific feature, F is the focal length of the X-ray tube, and C is the distance from the reference object to the camera surface. The distance from the reference object to the camera surface after it moves is then defined as C + D. According to formula (2), the pixel length of the specific feature before the reference object moves can be expressed as:

[0085] J1=H×『F / (FC)』(16)

[0086] After the shift, the pixel length of a specific feature of the reference object is expressed as:

[0087] J2=H×『F / (FCD)』(17)

[0088] The final scaling ratio S2 in the second embodiment can be expressed as

[0089] S2=J1 / J2=(FCD) / (FC)(18)

[0090] Since the camera is displaced in the scenario of Example 1, the final scaling ratio S1 obtained can also be understood as the scaling ratio of the camera's field of view angle. In the scenario of Example 2, the camera position is kept fixed while the reference object moves, and the influence of the camera focal length drift is given priority. In Example 3, both the camera position and the reference object position may move. Based on the same idea of Example 1 and the inspiration of Example 2, a better embodiment is to comprehensively consider the scaling ratio of the camera's field of view angle obtained in Example 1 and the correction parameters obtained in Example 2 to obtain the final X-ray scaling ratio S.

[0091] After the X-ray photo and the camera are visually aligned, if the reference object moves in the horizontal direction, it is usually no longer necessary to scale the X-ray photo, because the horizontal movement of the reference object after fusion alignment will not cause changes in its specific features in the image. If the reference object needs to move in the vertical direction, since the pixel values of the specific features in the image have changed, the X-ray photo and / or video image must be scaled to align with the specific features of the reference object. The principle is still that the closer the reference object moves toward the camera and is to the camera C-arm imaging plate assembly, the larger the specific features of the reference object in the image will be. Conversely, when the reference object moves away from the camera, the smaller it will be. When the triangular ruler moves a distance D, the scaling rate S of the X-ray photo is equal to the camera visual scaling rate S1 multiplied by the X-ray scaling rate S2, which can be expressed as:

[0092] S=S1×S2×k. (19)

[0093] Example 4

[0094] In Examples 1 to 3 of the present invention, the reference objects selected during the specific surgery are placed on a pre-configured carrier, such as the aforementioned medical drape, or an opening or channel made of a transparent non-metallic material. In cases where the surgical space is ample, an additional carrier with a flat surface can be provided to quickly level the reference objects. However, it is foreseeable that the leveling method using a carrier has obvious limitations:

[0095] 1) When the specific surgical location is further restricted to a small space, there is no space for additional carriers;

[0096] 2) If the surgical site is not horizontal, or the surgical site has a certain inclination angle, it is difficult to configure a carrier with a plane inclined at the same angle as the surgical site;

[0097] 3) In the scenario of multiple extracts or multiple implants, it is difficult to keep the reference object level at each location by relying on the same carrier.

[0098] In the case where it is difficult to configure a carrier at the above-mentioned surgical position, in extreme cases, it is even necessary to configure a reference object of a specific size to adapt to the narrow placement space. Therefore, it is obvious that the method of introducing an additional carrier for carrying to keep the reference object horizontal is not the best, or in other words, any form of carrier is difficult to meet the complex and changeable carrying requirements in actual surgery. For this problem, the easier solution is to introduce an additional level during positioning to ensure that the plane where the reference object is located is maintained in the horizontal direction, but adding a horizontal calibration device on the basis of the carrier is putting the cart before the horse, because this method not only cannot fundamentally solve the problem of difficulty in configuring carriers and reference objects at the aforementioned surgical position, but also makes the surgical steps cumbersome. Therefore, Example 4 of the present invention attempts to solve this problem in a better way.

[0099] The algorithms of Examples 1 to 3 can capture specific features of the reference objects during the capture and recognition of the reference objects. For example, looking back at Example 1, in an equilateral triangle reference object, the algorithm can identify two equal edges and a long hypotenuse of the equilateral triangle. In fact, in the process of using the edge detection algorithm for contour traversal and polygon approximation, in addition to identifying specific contours and length features such as specific edges contained in the specific contours, the algorithm can also be configured to identify the internal angles of closed contours. For example, in an embodiment where a reference object is selected as an equilateral right triangle, it is necessary to determine during the capture and recognition process that the three internal angles currently comprising the contour include two equal acute angles and one right angle. Therefore, Example 4 of the present invention realizes that if the central focus of the camera is set to be aligned with the shooting position, then for a polygon with a determined angle, when its internal angle is inconsistent with the preset value, it means that the plane where the reference object is located at this moment is not horizontal.

[0100] When the plane where the reference object is located is not horizontal, when capturing the angle feature of its inner angle from the picture, the angle of the specific inner angle should change. Generally, no matter how the plane where the reference object is located is flipped relative to the horizontal plane, the angle value of its inner angle will be proportionally reduced. Using the same idea as in Examples 1 to 3, a proportional coefficient can also be determined based on the ratio between the angle after the inner angle of the reference object is tilted and the preset value. After knowing Examples 1 to 3 of the present invention, those skilled in the art should know how to determine the picture zoom ratio based on the angle ratio. According to the proportional coefficient, the algorithm can correct the posture of the reference object in the image in the form of Example 1, so that the reference object is flipped to be flush with the image in the corrected image; or it can be based on the display of Examples 2 and 3, and according to the change of the specific inner angle of the reference object, determine the flip angle coefficient of the image, and then flip the image to finally make the image and the reference object features in the X-ray film coincide. It should be noted that when calculating reference object correction or image flipping in the manner described in Examples 2 and 3, new correction coefficients should be configured according to the angle ratio and in the manner described in Examples 2 and 3, and included in the final scaling formula.

[0101] Example 5

[0102] In the preceding embodiments one to four, the C-arm camera is set to be located directly above the surgical position, and the projection of the camera focus on the surgical position plane is kept consistent with the center position of the reference object. In specific implementation, in order to facilitate the accurate identification and fixation of the camera center position, the camera device is initially often set at the center position of the C-arm imaging plate. In this way, the camera is integrated into the C-arm imaging plate, making it easy to adjust its position as the C-arm moves. However, due to the X-ray film shooting angle and the camera configuration direction relative to the X-ray equipment, a significant disadvantage of the conventional setting method is that the image of the camera device will be left in the acquired X-ray film, which will not only affect the identification of the X-ray film, but also constitute an obstruction in the imaging picture. Therefore, a new camera configuration method is needed to eliminate the undesirable influence of the camera device on the shooting of X-ray photos.

[0103] In the fifth embodiment of the present invention, a camera positioning device is provided. The positioning device is firstly arranged at a non-center position of the imaging plate and fixed to the outside of the imaging plate by a fixing method such as mounting. Figure 1 and Figure 2 , Figure 1 and Figure 2Both are schematic diagrams, showing the front view and side view of the camera positioning device described in Example 5, respectively. The camera positioning device 100 is a block-shaped structural member with a recessed shooting port 101 formed on one side surface. The recessed shooting port 101 is provided with a camera device 102. It should be noted that the camera positioning device 100 is located on the same side surface of the camera device 102, specifically on two mutually perpendicular surfaces surrounding the recessed shooting port 101. There are two positioning holes 103 formed, each of which is fixed with a laser 104. Figure 1 or Figure 2 The two lines connecting the center points of the two lasers 104 and the center point of the recessed imaging port 101 should form a roughly 90° angle. The lasers can emit a straight line of laser light. The two straight lines of laser light converge to form a cross-shaped calibration line, which can be projected at the surgical site. The midpoint of the cross-shaped calibration line is considered the center of the camera's field of view. In actual implementation, the imaging plate can be moved to align the midpoint of the cross-shaped calibration line with the center of the surgical plane reference object.

[0104] Figure 3 and Figure 4 These are specific surgical pictures performed according to the contents of Example 1 and Example 5. Figure 3 First, an overlay image from an orthopedic ankle medial malleolus fracture localization procedure is shown. During this procedure, a metal right-angled triangle ruler is used as a reference to guide the surgical position. The ankle joint is first secured and covered with a medical drape. The right-angled triangle ruler is then placed above the drape. The C-arm camera is then turned on, and the C-arm is moved to align the camera with the triangle ruler at the center of the imaging area. An X-ray of the imaging position is captured and transmitted to a computer capable of communicating with the C-arm.

[0105] Then, according to the feature extraction method in Example 1, the image capture extraction algorithm is used to extract the photo with the right triangle ruler feature, and the pixel length of the long hypotenuse of the triangle ruler is measured and recorded as 23. Continue to shoot the lateral view of the ankle joint, determine the medial malleolus height, place the triangle ruler at the level of the medial malleolus height, and re-measure the pixel length of the long hypotenuse of the triangle ruler as 20.9. Record the distance from the camera to the triangle ruler as 50, and the distance from the C-arm focus to the triangle ruler as 80. Then, according to the method indicated in Example 3, determine the image scaling ratio at this time as 0.9372. Finally, use this scaling ratio to align and correct the image, that is, superimpose the features of the triangle ruler in the X-ray photo on the corresponding features of the triangle ruler in the camera vision, and finally present the image as shown below. Figure 3 After calibration and fusion are completed, the image fusion software of the computer device plans the path based on the image of the ankle joint in the X-ray, and the K-wire is implanted along the path.

[0106] Figure 4 The figure shows an overlapped image during the positioning process of the distal screw locking hole of an orthopedic intramedullary nail. During this implementation process, a metal right-angled triangle ruler is also selected as a reference object, and the longest side of the right-angled triangle ruler is still selected as a specific feature. In addition, a transparent glass carrier with a flat surface and an opening on one side is configured as a carrier of the triangle ruler. The camera is turned on, and the C-arm is moved so that the camera is aimed at the triangle ruler in the center of the shooting part. An X-ray photo at this time is taken and transmitted to a computer device that can communicate with the C-arm machine. Similarly, the same steps are followed to make the specific features of the reference object in the image coincide with the corresponding features in the X-ray film. Subsequently, an X-ray side image is taken to determine the height of the intramedullary nail on the triangle ruler, ensuring that they are on the same horizontal plane. The triangle ruler is placed on the same plane as the intramedullary nail, and alignment and calibration are performed again.

[0107] Measure the pixel length value of the long hypotenuse of the triangle, and record the changes in the specific feature value at different times. For example, the width value of the registration triangle under the positive film angle is 22.5, and the width value of the registration triangle under the side film angle (when the ruler is placed on the intramedullary nail plane) is 20. Determine that the height from the C-arm ruler to the imaging plate is 52, and the distance from the focus of the X-ray tube in the C-arm to the triangle ruler is 80. According to the method indicated in Example 2, determine that the image zoom ratio at this time is 0.9187, and use this zoom ratio to adjust the picture so that the image is scaled and registered. Finally, according to the navigation map, insert the orthopedic Kirschner wire into the intramedullary nail hole.

[0108] The above embodiments merely illustrate several implementations of the present invention, and while their descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that a person skilled in the art would be able to make numerous modifications and improvements without departing from the spirit of the present invention, all of which fall within the scope of protection of the present invention. Therefore, the scope of protection of the present invention shall be determined by the appended claims.

Claims

1. A method for locating a surgical position, the method being used for intraoperative position positioning using X-rays to assist imaging by a camera device, the method comprising the following steps: A closed or non-closed polygonal metal reference object having a closed contour is provided, wherein the reference object is configured to be placed near the surgical site; selecting at least one feature constituting the reference object profile as a specific feature, and obtaining a set value of the specific feature; Taking an X-ray of the surgical site and configuring preset conditions so that the computer can identify and capture the reference object and its specific features in the X-ray according to the preset conditions; Driving the camera device to move in a direction toward or away from the reference object for shooting, and displacing from a first state position to a second state position, or causing the reference object to move from the first state position to the second state position under the shooting field of view of the camera device; Recording a first reference value of the specific feature in a first state position and a second reference value of the specific feature in a second state position; The scaling ratio is determined based on the ratio of the first reference value to the second reference value and the set value, and the X-ray film and / or the camera imaging screen is reduced or enlarged according to the scaling ratio, so that after the X-ray film and the image screen are superimposed on each other, the same specific features in the X-ray film and the image are aligned and overlapped, thereby completing the positioning.

2. The surgical position positioning method according to claim 1, wherein: The reference object and the closed contour contained therein have at least one specific feature that can be recognized and captured by the computer according to preset conditions. The specific feature is a length value, an angle value, or a parameter for representing color.

3. The surgical position positioning method according to claim 2, wherein: The shape of the reference object is an equilateral triangle, and the hypotenuse of the equilateral triangle is selected as the specific feature.

4. The surgical position positioning method according to any one of claims 1 to 3, wherein: The reference object is further configured to be supported by a carrier to be positioned close to the surgical site, wherein the carrier is a non-metallic carrier having at least one horizontal supporting surface, so that the plane of the reference object remains horizontal with the surgical site.

5. The surgical position positioning method according to claim 1, wherein: The step of configuring the preset conditions so that the computer can identify and capture the reference object in the X-ray film according to the preset conditions is specifically as follows: The configuration is composed of at least one combination of a length value, an angle value, and a parameter for expressing color, and guides the computer to capture the preset conditions of the specific feature; Adjust the grayscale of the X-ray film and then perform Gaussian blur processing to obtain a denoised image; Edge detection is performed on the denoised image, the image is traversed to extract closed contours and features therein, and contour approximation is used to determine at least one reference object contour that meets the preset conditions.

6. The surgical position positioning method according to claim 5, wherein: The specific steps of determining the scaling ratio according to the ratio of the first reference value to the second reference value and the set value are: When the camera device moves close to the reference object, the first reference value of the specific feature of the reference object before the camera device moves is set to W1, and the second reference value of the specific feature after the camera device moves is set to W2, then the field of view angle scaling ratio satisfies: R = \frac {W1}{W2} (1).

7. The surgical position positioning method according to claim 5 or 6, wherein: The specific steps of determining the scaling ratio according to the ratio of the first reference value to the second reference value and the set value are: Recording the initial focal length F of the camera device; Moving the reference object under the field of view of the camera device, setting a first reference value J1 of the specific feature under the field of view before the reference object is moved, and setting a second reference value J2 of the specific feature under the field of view after the reference object is moved; The distances of the reference object from the camera device before and after the movement and the corresponding reference values are combined to obtain a displacement expression of the reference object and a theoretical value of the displacement of the reference object; The optical scaling ratio is obtained based on the ratio of the actual value of the reference object displacement to the theoretical value.

8. The surgical position positioning method according to claim 7, wherein: Also includes The correction coefficient is determined based on the ratio of the actual value of the reference displacement to the theoretical value. Correcting the field angle scaling ratio using the correction coefficient, and / or, The optical scaling is corrected using the correction factor.

9. The surgical position positioning method according to claim 5, wherein: The specific steps of determining the scaling ratio according to the ratio of the first reference value to the second reference value and the set value are: selecting at least one inner angle of the closed contour of the reference object as the specific feature; When the camera device moves close to the reference object, the first reference value of the specific feature of the reference object before the camera device moves is set to M1, and the second reference value of the specific feature after the camera device moves is set to M2, then the field of view angle scaling ratio satisfies: R = \frac {M1}{M2} (2).

10. A surgical position positioning device, comprising: A host computer, which is connected and communicates with the camera device and the X-ray camera; at least one reference object disposed near the surgical site, the reference object having at least one specific feature that can be recognized and captured by the computer according to preset conditions; Wherein, the host computer stores a computer program configured and executed according to the positioning method according to any one of claims 1 to 9 to locate the surgical position.

11. The surgical position positioning device according to claim 10, further comprising a laser camera positioning block, the positioning block being disposed at an edge of the C-arm imaging plate and moving along with the C-arm camera device, the positioning block comprising: The main body is a hollow block with a concave shooting port formed on one side of the surface. a camera device, the camera device being embedded in the photographing port and configured to limit the field of view angle of the camera device according to the photographing port; At least two positioning through holes are formed on the surface of the body around the shooting port, and a laser is set in each positioning through hole, wherein, The laser is a straight laser, and the laser beam of the laser converges to form a cross-shaped calibration line. The cross-shaped calibration line is projected at the surgical position and aligned with the center position of the reference object.

Images