A mixed reality navigation method and system for distal locking of femoral interlocking intramedullary nail

Abstract

The present invention relates to the field of augmented reality technology and discloses a mixed reality navigation method and system for distal locking of a femoral interlocking intramedullary nail. The method comprises: constructing a surgical navigation framework for a target user, collecting a femoral X-ray and camera monitoring image of the target user in real time; registering and fusing the femoral X-ray and camera monitoring image, and enhancing the registered fused image to obtain an enhanced fused image; defining a coordinate transformation algorithm for the surgical navigation framework; constructing a surgical tool for the surgical navigation framework, collecting a tool image of the surgical tool, extracting multi-marker point features of the tool image, and calculating the center point coordinates and drill bit coordinates of the surgical tool; calculating the positional deviation and angular deviation of the surgical tool, constructing a surgical locking navigation module for the target user, and determining a navigation path corresponding to the center point of the surgical tool. The present invention can improve the navigation accuracy of mixed reality navigation for distal locking of a femoral interlocking intramedullary nail.

Description

Technical Field

[0001] The present invention relates to the field of augmented reality technology, and in particular to a mixed reality navigation method and system for distal locking of a femoral interlocking intramedullary nail. Background Art

[0002] The interlocking femoral intramedullary nail is an internal fixation device used to treat femoral fractures. It works by inserting a long, hollow nail into the femoral medullary canal. Locking nails are then driven through locking holes in the nail into the distal and proximal ends of the femur to stabilize the fracture. Precise nail placement helps provide more stable fixation, promotes fracture healing, and reduces the risk of postoperative fracture displacement, thereby improving overall surgical outcomes.

[0003] Traditional intramedullary nail locking surgery relies on continuous X-ray imaging of the fracture site using a radiographic instrument to determine the distal screw hole location. This approach can cause image clarity and errors due to the potential overlap of different tissues and organs on X-rays. This results in low positioning accuracy, prolonged radiation exposure, and potential health risks for the patient. Summary of the Invention

[0004] The present invention provides a mixed reality navigation method and system for distal locking of a femoral interlocking intramedullary nail, the main purpose of which is to improve the navigation accuracy of mixed reality navigation for distal locking of a femoral interlocking intramedullary nail.

[0005] To achieve the above objectives, the present invention provides a mixed reality navigation method for distal locking of a femoral interlocking intramedullary nail, comprising:

[0006] Constructing a surgical navigation framework for the target user, configuring a C-arm X-ray machine and an industrial camera for the surgical navigation framework, using the C-arm X-ray machine to acquire a femur X-ray of the target user in real time, and using the industrial camera to acquire a camera monitoring image of the target user in real time;

[0007] Constructing a reality augmentation module of the surgical navigation framework, extracting X-ray film features and camera image features of the femoral X-ray film and the camera monitoring screen respectively based on the reality augmentation module, registering and fusing the femoral X-ray film and the camera monitoring screen using a preset image registration algorithm based on the X-ray film features and the camera image features to obtain a registered fused image, and enhancing the registered fused image using a preset image enhancement algorithm to obtain an enhanced fused image;

[0008] Analyze the X-ray imaging principles and camera imaging principles of the C-arm X-ray machine and the industrial camera, and define a coordinate transformation algorithm of the surgical navigation framework based on the X-ray imaging principles and the camera imaging principles;

[0009] Constructing a surgical tool of the surgical navigation framework, identifying multiple marker points of the surgical tool, using the industrial camera to capture a tool image of the surgical tool in real time, extracting multi-marker point features of the tool image, and calculating the center point coordinates and drill bit coordinates of the surgical tool based on the multi-marker point features;

[0010] Identify the drilling position coordinates of the enhanced fusion image, calculate the position deviation of the surgical tool based on the drilling position coordinates and the drill bit coordinates, convert the drilling position coordinates and the center point coordinates into drilling space coordinates and center point space coordinates respectively based on the coordinate conversion algorithm, calculate the angular deviation of the surgical tool based on the drilling space coordinates and the center point space coordinates, construct a surgical locking navigation module for the target user based on the angular deviation, the position deviation and the surgical navigation framework, and determine the navigation path of the surgical tool corresponding to the center point based on the surgical locking navigation module.

[0011] In order to solve the above problems, the present invention also provides a mixed reality navigation system for distal locking of a femoral interlocking intramedullary nail, the system comprising:

[0012] An image acquisition module is used to construct a surgical navigation framework for the target user, configure a C-arm X-ray machine and an industrial camera for the surgical navigation framework, use the C-arm X-ray machine to acquire femur X-rays of the target user in real time, and use the industrial camera to acquire camera monitoring images of the target user in real time;

[0013] An image fusion module is used to construct a reality enhancement module of the surgical navigation framework, and based on the reality enhancement module, the X-ray film features and camera image features of the femur X-ray film and the camera monitoring screen are respectively extracted; based on the X-ray film features and the camera image features, the femur X-ray film and the camera monitoring screen are registered and fused using a preset image registration algorithm to obtain a registered fused image; and the registered fused image is enhanced using a preset image enhancement algorithm to obtain an enhanced fused image;

[0014] A coordinate conversion algorithm module is used to analyze the X-ray imaging principle and the camera imaging principle of the C-arm X-ray machine and the industrial camera, and define a coordinate conversion algorithm of the surgical navigation framework based on the X-ray imaging principle and the camera imaging principle;

[0015] a navigation and positioning module for constructing a surgical tool in the surgical navigation framework, identifying multiple marker points of the surgical tool, acquiring a tool image of the surgical tool in real time using the industrial camera, extracting features of the multiple marker points of the tool image, and calculating the center point coordinates and drill bit coordinates of the surgical tool based on the multiple marker point features;

[0016] A navigation path determination module is used to identify the drilling position coordinates of the enhanced fusion image, calculate the position deviation of the surgical tool based on the drilling position coordinates and the drill bit coordinates, convert the drilling position coordinates and the center point coordinates into drilling space coordinates and center point space coordinates respectively based on the coordinate conversion algorithm, calculate the angular deviation of the surgical tool based on the drilling space coordinates and the center point space coordinates, construct a surgical locking navigation module for the target user based on the angular deviation, the position deviation and the surgical navigation framework, and determine the navigation path of the surgical tool corresponding to the center point based on the surgical locking navigation module.

[0017] The embodiment of the present invention can provide detailed surgical paths and real-time feedback by constructing a surgical navigation framework for the target user, thereby reducing unnecessary trauma and risks during surgery, and thus improving the accuracy of surgical operations; optionally, the embodiment of the present invention can achieve spatial alignment of the X-ray film and the camera image by using a preset image registration algorithm based on the X-ray film features and the camera image features, so that images of two different modalities can be observed and analyzed in the same coordinate system; the embodiment of the present invention can use the coordinate conversion algorithm to define the coordinate conversion algorithm of the surgical navigation framework based on the X-ray machine imaging principle and the camera imaging principle, and more accurately predict the path and position of surgical tools in the preoperative planning stage, thereby reducing surgery. Uncertainty in; the embodiment of the present invention calculates the center point coordinates and drill bit coordinates of the surgical tool based on the multi-marker point feature, so as to accurately know the position of the surgical tool, thereby achieving accurate positioning and navigation in the surgical navigation system and reducing errors during the operation; the embodiment of the present invention converts the drilling position coordinates and the center point coordinates into drilling space coordinates and center point space coordinates respectively based on the coordinate conversion algorithm, and can convert coordinates in real time, allowing the surgical navigation system to provide real-time guidance during the operation, which helps to adjust the position and direction of the surgical tool in real time. Finally, the embodiment of the present invention determines the navigation path corresponding to the center point of the surgical tool based on the surgical locking navigation module. Through precise navigation path planning, it can ensure that the surgical tool moves according to the predetermined trajectory, thereby reducing errors during the operation. Therefore, the present invention can improve the navigation accuracy of the mixed reality navigation of the distal locking of the femoral interlocking intramedullary nail. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 A schematic flow chart of a mixed reality navigation method for distal locking of a femoral interlocking intramedullary nail provided by one embodiment of the present invention;

[0019] Figure 2 This is a functional module diagram of a mixed reality navigation system for distal locking of a femoral interlocking intramedullary nail provided by one embodiment of the present invention;

[0020] The purpose, features and advantages of the present invention will be further described with reference to the accompanying drawings and in conjunction with the embodiments. DETAILED DESCRIPTION

[0021] It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0022] The embodiment of the present application provides a mixed reality navigation method for distal locking of a femoral interlocking intramedullary nail. The execution subject of the mixed reality navigation method for distal locking of a femoral interlocking intramedullary nail includes but is not limited to at least one of the electronic devices such as a server and a terminal that can be configured to execute the method provided by the embodiment of the present application. In other words, the mixed reality navigation method for distal locking of a femoral interlocking intramedullary nail can be executed by software or hardware installed on a terminal device or a server device, and the software can be a blockchain platform. The server includes but is not limited to: a single server, a server cluster, a cloud server or a cloud server cluster, etc. The server can be an independent server, or it can be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms.

[0023] Reference Figure 1 FIG. 1 is a flow chart of a mixed reality navigation method for distal locking of a femoral interlocking intramedullary nail provided by an embodiment of the present invention. In this embodiment, the mixed reality navigation method for distal locking of a femoral interlocking intramedullary nail includes:

[0024] S1. Construct a surgical navigation framework for the target user, configure a C-arm X-ray machine and an industrial camera for the surgical navigation framework, use the C-arm X-ray machine to collect femur X-rays of the target user in real time, and use the industrial camera to collect camera monitoring images of the target user in real time.

[0025] The present invention provides a detailed surgical path and real-time feedback by constructing a surgical navigation framework tailored to the target user, reducing unnecessary trauma and risk during surgery, thereby improving the accuracy of surgical operations. The surgical navigation framework is a system that integrates multiple medical imaging devices and computer-assisted technologies.

[0026] Optionally, as an embodiment of the present invention, the surgical navigation framework for the target user may be constructed using tracking and positioning technology.

[0027] By configuring a C-arm X-ray machine and an industrial camera in the surgical navigation framework, embodiments of the present invention can obtain precise internal anatomical information while also capturing real-time visual information of the surgical scene, thereby improving surgical accuracy. The C-arm X-ray machine is a mobile X-ray imaging device widely used in operating rooms and emergency rooms. The industrial camera is a specialized camera used in industrial automation for detection, measurement, identification, and monitoring.

[0028] In this embodiment of the present invention, by utilizing the C-arm X-ray machine to capture real-time femoral X-rays of the target user, radiation exposure to patients and medical staff can be reduced, thereby helping to avoid surgical errors, reduce damage to surrounding healthy tissue, and improve surgical safety. The femoral X-ray refers to an X-ray image specifically used to display and analyze the internal structure and morphology of the femur (thigh bone).

[0029] Optionally, as an embodiment of the present invention, the real-time collection of the femur X-ray of the target user by using the C-arm X-ray machine can be obtained by X-ray imaging technology.

[0030] This embodiment of the present invention utilizes the industrial camera to capture the target user's camera monitoring image in real time, providing a real-time video image of the surgical site, enabling intuitive observation of the surgical process rather than relying solely on two-dimensional images such as X-rays. The camera monitoring image refers to the video image of the surgical site captured and displayed by the industrial camera.

[0031] S2. Construct a reality enhancement module of the surgical navigation framework, and based on the reality enhancement module, extract the X-ray film features and camera image features of the femur X-ray film and the camera monitoring screen respectively; based on the X-ray film features and the camera image features, use a preset image registration algorithm to align and fuse the femur X-ray film and the camera monitoring screen to obtain a registered fused image; use a preset image enhancement algorithm to enhance the registered fused image to obtain an enhanced fused image.

[0032] The embodiments of the present invention, through the construction of an augmented reality module within the surgical navigation framework, can provide intuitive image information, reduce uncertainty during surgery, and thus lower surgical risks. The augmented reality module is a software and hardware complex integrated into the surgical navigation system. It utilizes computer vision, image processing, and augmented reality technologies to combine virtual image information with the real-world surgical scene, providing surgeons with more intuitive and richer surgical assistance information.

[0033] Optionally, as an embodiment of the present invention, the reality enhancement module for constructing the surgical navigation framework can be constructed by reality enhancement rendering technology.

[0034] The embodiment of the present invention can achieve precise registration between the X-ray film and the real-time camera monitoring screen by extracting the X-ray film features and camera image features of the femoral X-ray film and the camera monitoring screen based on the augmented reality module. This helps ensure the accurate alignment of the virtual X-ray image with the real surgical scene and improves the accuracy of surgical navigation. The X-ray film features refer to a series of information points or attributes extracted during the X-ray image processing and analysis process to characterize the content of the X-ray image. The camera image features refer to information points or attributes extracted from the real-time image captured by the camera to describe the image content, structure, and appearance.

[0035] As an embodiment of the present invention, the extracting of X-ray features and camera image features of the femur X-ray and the camera monitoring screen based on the augmented reality module includes:

[0036] Denoising the femur X-ray and the camera monitoring image using a preset filter to obtain a denoised X-ray and a denoised camera image;

[0037] performing contrast enhancement on the denoised X-ray film to obtain an enhanced X-ray film, and performing color correction on the denoised camera image to obtain a corrected camera image;

[0038] Performing contour recognition on the enhanced X-ray film to obtain a femoral contour, calculating geometric parameters of the femoral contour, and determining X-ray shape features of the femoral X-ray film based on the geometric parameters;

[0039] Performing shape detection on the correction camera image to obtain camera image shape features;

[0040] Extracting X-ray film texture features of the enhanced X-ray film and camera image texture features of the corrected camera image respectively using a preset local binary pattern;

[0041] Identifying X-ray film structural features of the enhanced X-ray film, and extracting X-ray film position features of the enhanced X-ray film based on the X-ray film structural features;

[0042] Extracting image key points of the corrected camera image, and determining camera image spatial features of the corrected camera image based on the image key points;

[0043] Determining X-ray features of the femur X-ray according to the X-ray shape feature, the X-ray texture feature, the X-ray structure feature, and the X-ray position feature;

[0044] The camera image features of the camera monitoring screen are determined according to the camera screen shape features, the camera screen texture features and the camera screen space features.

[0045] Among them, the preset filter refers to an algorithm designed or selected to achieve a specific purpose, which is used to modify or enhance image data. The denoised X-ray refers to an X-ray image that has been denoised. The denoised camera image refers to an image in which the noise in the image captured by the camera is reduced by applying specific image processing techniques. The enhanced X-ray refers to an X-ray image that has been contrast-enhanced, the purpose of which is to improve the visual readability of the image and highlight specific anatomical structures. The corrected camera image refers to the image captured by the camera after color correction and other related image processing steps. The femoral contour refers to the visible external boundary line of the femur in the image. The geometric parameters refer to quantitative measurement indicators used to describe the femoral contour or a specific part thereof. The X-ray shape features refer to quantitative or qualitative information extracted from the X-ray film to describe the contour, size, proportion and geometric shape of the femur or other anatomical structures. The camera image shape features refer to information extracted from the image captured by the camera to describe the contour, geometry and structure of objects or specific areas in the scene. The preset local binary pattern refers to a feature descriptor used for image texture analysis, which works by comparing the brightness values ​​of a pixel with its neighboring pixels, comparing each pixel with its surrounding pixels, and encoding the comparison results into a binary number. The X-ray film texture feature refers to the quantitative or qualitative attributes used to describe the internal structure and texture information of an image in X-ray imaging. The camera image texture feature refers to the information extracted from the image captured by the camera to describe the surface structural details and arrangement pattern of the image. The X-ray film structural feature refers to the information extracted from the X-ray film to describe the geometric morphology, tissue layout, and interrelationships of bones, organs, or other anatomical structures. The X-ray film position feature refers to the quantitative information extracted from the X-ray film to describe the specific spatial location of anatomical structures or lesions in the image. The image key point refers to an important location point in an image that can be used to represent image features or content. The camera image spatial feature refers to the characteristics extracted from the image captured by the camera to describe the position, layout, and interrelationships of objects and scenes in the image in three-dimensional space.

[0046] Optionally, the contrast enhancement of the denoised X-ray film to obtain the enhanced X-ray film may be enhanced by a histogram equalization method.

[0047] Optionally, the identification of the X-ray film structural features of the enhanced X-ray film can be performed by a Hough transform method.

[0048] In this embodiment of the present invention, based on the X-ray film features and the camera image features, a preset image registration algorithm is used to register and fuse the femoral X-ray film and the camera monitoring image to obtain a registered fused image. This achieves spatial alignment of the X-ray film and the camera image, enabling observation and analysis of the two different modalities within the same coordinate system. The registered fused image refers to a single image obtained after image registration and fusion techniques are used.

[0049] As an embodiment of the present invention, the femoral X-ray film and the camera monitoring screen are registered and fused using a preset image registration algorithm based on the X-ray film features and the camera image features to obtain a registered fused image, including:

[0050] Extracting X-ray film feature points for the X-ray film features and extracting camera image feature points for the camera image features;

[0051] An X-ray film descriptor analyzing the X-ray film feature points and a camera image descriptor analyzing the camera image feature points;

[0052] unifying the descriptor dimensions of the X-ray film descriptor and the camera image descriptor;

[0053] Based on the descriptor dimension, the inter-descriptor distance between the X-ray film descriptor and the camera image descriptor is calculated using the following formula:

[0054]

[0055] in, represents the distance between descriptors, represents an X-ray film descriptor, represents the camera image descriptor, Indicates the first X-ray elements, Represents the camera image descriptor image elements, represents the descriptor dimension;

[0056] Matching the X-ray film features and the camera image features according to the distance between the descriptors to obtain a feature matching relationship;

[0057] Based on the feature matching relationship, construct an image transformation model of the femoral X-ray and the camera monitoring screen;

[0058] According to the image transformation model, the femur X-ray film and the camera monitoring image are fused to obtain a registered fused image.

[0059] Among them, the X-ray feature points refer to significant points in the X-ray image that can be detected and used for subsequent image analysis and processing. The camera image feature points refer to points with specific attributes detected in the image captured by the camera. The X-ray descriptor refers to a vector used to describe the local image features around the feature points in the X-ray image. The camera image descriptor refers to a vector used to describe the local image features of the feature points extracted from the image captured by the camera. The descriptor dimension refers to the length of the feature descriptor vector, that is, the number of elements contained in the descriptor. The inter-descriptor distance refers to the degree of difference between two feature descriptor vectors. The feature matching relationship refers to the correspondence established between the feature points in the two images. The image transformation model refers to a mathematical model used to describe how one image is mapped to another image through geometric transformation or other forms of transformation.

[0060] Optionally, the X-ray film descriptor obtained by analyzing the X-ray film feature points may be obtained by analyzing using an accelerated robust feature algorithm.

[0061] Optionally, the X-ray film features and the camera image features are matched according to the distance between the descriptors, and the feature matching relationship obtained can be obtained by a nearest neighbor matching method.

[0062] Optionally, constructing an image transformation model of the femur X-ray and the camera monitoring screen based on the feature matching relationship includes:

[0063] defining X-ray feature point coordinates of the femur X-ray film, and mapping the image feature point coordinates of the camera monitoring screen based on the feature matching relationship;

[0064] Constructing an initial affine transformation model of the femur X-ray film and the camera monitoring screen, and determining transformation parameters of the initial affine transformation model;

[0065] Based on the transformation parameters, transform the femoral X-ray film using the initial affine transformation model to obtain a transformed femoral X-ray film;

[0066] Calculating the transformed X-ray feature point coordinates of the transformed femoral X-ray film based on the X-ray feature point coordinates;

[0067] Based on the transformation parameters, the X-ray feature point coordinates, and the image feature point coordinates, the objective function and transformation parameter vector of the initial affine transformation model are determined using the following formula:

[0068]

[0069]

[0070] in, represents the objective function, represents the transformation parameter vector, Indicates the total number of transformed X-ray feature point coordinates, Indicates the The horizontal coordinates of the transformed X-ray feature points are transformed. Indicates the The transformed X-ray feature point coordinates are the transformed X-ray feature point ordinates, Indicates the The horizontal coordinates of the image feature points, Indicates the The vertical coordinates of the image feature points, Indicates that the transformation parameters correspond to the horizontal axis scaling parameters, Indicates that the transformation parameters correspond to the horizontal axis rotation parameters, Indicates that the transformation parameter corresponds to the horizontal axis shear parameter, Indicates that the transformation parameters correspond to the vertical axis scaling parameters, Indicates that the transformation parameters correspond to the vertical axis rotation parameters, Indicates that the transformation parameter corresponds to the longitudinal axis shear parameter;

[0071] Calculating the transformation residual value of the X-ray feature point coordinates and the image feature point coordinates according to the objective function and the transformation parameter vector;

[0072] When the transformation residual value is less than a preset transformation residual threshold, the initial affine transformation model is used as the image transformation model of the femur X-ray and the camera monitoring screen.

[0073] The X-ray feature point coordinates refer to the locations of feature points on a femoral X-ray. The image feature point coordinates refer to the locations of feature points on the camera monitoring screen. The initial affine transformation model refers to a mathematical model that describes how to transform a set of points from one coordinate system to another in two-dimensional space. The transformation parameters refer to specific numerical values ​​used to define the image transformation model. The transformed femoral X-ray refers to the image obtained by transforming the femoral X-ray using the initial affine transformation model. The transformed X-ray feature point coordinates refer to the locations of corresponding feature points on the transformed femoral X-ray. The objective function refers to a mathematical function used to evaluate registration quality and guide the optimization process during image registration. The transformation parameter vector refers to a vector containing all parameters used to define the image transformation model. The horizontal axis scaling parameter refers to a parameter in the image transformation model specifically used to adjust the horizontal scaling ratio of the image. The horizontal axis rotation parameter refers to a parameter in the image transformation model specifically used to adjust the horizontal rotation ratio of the image. The horizontal axis shearing parameter refers to a parameter in the image transformation model specifically used to adjust the horizontal shearing ratio of the image. The vertical axis scaling parameter refers to a parameter specifically used to adjust the scaling ratio of the image in the vertical direction in the image transformation model. The vertical axis rotation parameter refers to a parameter specifically used to adjust the rotation ratio of the image in the vertical direction in the image transformation model. The vertical axis shearing parameter refers to a parameter specifically used to adjust the shearing ratio of the image in the vertical direction in the image transformation model. The transformation residual value refers to the difference between the transformed point and the actual target point during the image transformation process. The preset transformation residual threshold refers to a pre-set value indicating the highest acceptable residual value when performing image transformation.

[0074] Optionally, the calculating the transformed X-ray feature point coordinates of the transformed femoral X-ray film based on the X-ray feature point coordinates includes:

[0075] Based on the X-ray feature point coordinates, the transformed X-ray feature point coordinates of the transformed femoral X-ray film are calculated using the following formula:

[0076]

[0077] in, The transformed horizontal coordinate represents the transformed X-ray feature point coordinates. The transformed ordinate represents the transformed X-ray feature point coordinates. The horizontal coordinate represents the coordinates of the X-ray feature point. The ordinate represents the coordinates of the X-ray feature point. Indicates that the transformation parameters correspond to the horizontal axis scaling parameters, Indicates that the transformation parameters correspond to the horizontal axis rotation parameters, Indicates that the transformation parameter corresponds to the horizontal axis shear parameter, Indicates that the transformation parameters correspond to the vertical axis scaling parameters, Indicates that the transformation parameters correspond to the vertical axis rotation parameters, Indicates that the transformation parameter corresponds to the longitudinal axis shear parameter;

[0078] In the embodiments of the present invention, the registered fused image is enhanced using a preset image enhancement algorithm to obtain an enhanced fused image that can make the differences between different tissues or structures more apparent, thereby facilitating the identification of key features in the image. The enhanced fused image refers to a fused image processed using image enhancement technology.

[0079] As an embodiment of the present invention, the step of enhancing the registered fused image using a preset image enhancement algorithm to obtain an enhanced fused image includes:

[0080] Performing edge detection on the registered fused image to obtain the edge of the fused image;

[0081] Determining target pixel points that need to be enhanced in the registered fused image according to the edge of the fused image;

[0082] Determining the pixel coordinates and pixel values ​​of the target pixel, and determining the image target intensity of the registered fused image;

[0083] The image enhancement coefficient of the registered fusion image is calculated using the following formula according to the pixel coordinates, the target pixel value, and the image target intensity:

[0084]

[0085] in, represents the image enhancement coefficient, represents the image target intensity, represents the pixel value, Indicates the horizontal coordinate of the pixel point corresponding to the pixel coordinate, Indicates the vertical coordinate of the pixel point corresponding to the pixel coordinate, represents partial derivative;

[0086] Based on the image enhancement coefficient, the registered fused image is enhanced using a preset image enhancement algorithm to obtain an enhanced fused image.

[0087] The fused image edge refers to the important edge features of the registered fused image. The target pixel refers to the specific pixel positions selected as the enhancement targets during the image enhancement process. The pixel coordinates refer to the location identifiers of the target pixel. The pixel value refers to the numerical value represented by the target pixel. The image target intensity refers to the image intensity level that is expected to be achieved or optimized in image processing or analysis tasks. The image enhancement coefficient refers to the numerical value used to adjust the image pixel intensity.

[0088] Optionally, the edge detection is performed on the registered fused image to obtain the edge of the fused image by using a Canny edge detector.

[0089] S3. Analyze the X-ray imaging principles and camera imaging principles of the C-arm X-ray machine and the industrial camera, and define a coordinate transformation algorithm of the surgical navigation framework based on the X-ray imaging principles and the camera imaging principles.

[0090] By analyzing the X-ray imaging principles and camera imaging principles of the C-arm X-ray machine and the industrial camera, the present invention can more accurately locate surgical tools and target anatomical structures, improve surgical precision, and reduce surgical risks. The X-ray imaging principle refers to the physical process by which an X-ray machine uses X-rays to penetrate an object and produce an image. The camera imaging principle refers to the process by which a camera captures light in a scene through its optical system and converts it into a visible image.

[0091] Optionally, as an embodiment of the present invention, the analysis of the X-ray machine imaging principle and the camera imaging principle of the C-arm X-ray machine and the industrial camera can be analyzed by a schematic diagram analysis method.

[0092] By defining a coordinate transformation algorithm for the surgical navigation framework based on the imaging principles of X-ray machines and cameras, embodiments of the present invention can utilize the coordinate transformation algorithm to more accurately predict the path and position of surgical tools during preoperative planning, thereby reducing uncertainty during surgery. The coordinate transformation algorithm is a mathematical method used to transform the position information of a point or object in one coordinate system into another.

[0093] As an embodiment of the present invention, the coordinate conversion algorithm of the surgical navigation framework is defined based on the imaging principle of the X-ray machine and the imaging principle of the camera, including:

[0094] Based on the camera imaging principle, determining the camera intrinsic parameters and camera extrinsic parameters of the industrial camera corresponding to the camera imaging principle;

[0095] extracting target pixel coordinates of the surgical navigation frame and constructing a femoral geometric model of the surgical navigation frame;

[0096] Constructing a camera coordinate conversion algorithm of the target pixel coordinates according to the camera intrinsic parameters, the camera extrinsic parameters and the femoral geometric model;

[0097] Based on the X-ray imaging principle, determining the baseline length and X-ray focal length of the C-arm X-ray machine corresponding to the X-ray imaging principle;

[0098] Determining an imaging equation of the C-arm X-ray machine according to the baseline length and the X-ray focal length;

[0099] Determining an X-ray coordinate conversion algorithm of the target pixel coordinates according to the imaging equation and the femoral geometric model;

[0100] The X-ray coordinate conversion algorithm and the camera coordinate conversion algorithm are integrated to obtain a coordinate conversion algorithm.

[0101] The camera intrinsic parameters refer to the inherent properties of the camera, describing the relationship between the camera lens and imaging sensor. The camera extrinsic parameters describe the position and orientation of the camera relative to the observed scene. The target pixel coordinates refer to the coordinates of the target object or specific feature in the image coordinate system, expressed in pixel units. The femoral geometric model refers to a coordinate system used to describe the position of the femur in three-dimensional space. The camera coordinate conversion algorithm refers to a series of mathematical processes and methods used to convert the image coordinates captured by the camera (usually two-dimensional coordinates) into three-dimensional spatial coordinates in the real world. The baseline length refers to the distance between two imaging sensors (such as camera lenses or detectors in an X-ray machine). In the X-ray tube, it refers to the distance from the X-ray emission source (usually the anode) to the focal point where the X-rays are focused. The imaging equation refers to the mathematical equation that describes the relationship between a point in object space and its corresponding pixel on the image plane during the image formation process. The X-ray coordinate conversion algorithm refers to a series of mathematical processing steps used to convert the two-dimensional pixel coordinates in the X-ray image into three-dimensional spatial coordinates in the real world.

[0102] Optionally, the femoral geometric model for constructing the surgical navigation framework can be constructed by computer-aided design.

[0103] Optionally, the imaging equation of the C-arm X-ray machine determined according to the baseline length and the X-ray focal length can be determined by establishing a perspective projection relationship.

[0104] S4. Construct the surgical tool of the surgical navigation framework, identify the multiple marking points of the surgical tool, use the industrial camera to collect the tool image of the surgical tool in real time, extract the multi-marker point features of the tool image, and calculate the center point coordinates and drill bit coordinates of the surgical tool based on the multi-marker point features.

[0105] The embodiment of the present invention can track the position and direction of surgical tools in real time by constructing the surgical navigation framework, thereby performing surgical operations more accurately and reducing surgical errors. The surgical tools refer to various precision medical instruments used in the surgical navigation framework.

[0106] Optionally, as an embodiment of the present invention, the surgical tools for constructing the surgical navigation framework can be constructed by optical tracking technology.

[0107] The embodiments of the present invention can more accurately track the position and orientation of a surgical tool in three-dimensional space by identifying multiple markers on the surgical tool, thereby reducing tracking errors. The multiple markers refer to multiple specific identification points arranged on the surgical tool and used in a surgical navigation system to track and determine the position and orientation of the tool.

[0108] In embodiments of the present invention, by utilizing the industrial camera to capture a tool image of the surgical tool in real time, real-time visual feedback of the surgical tool during surgery can be provided, enabling intuitive determination of the tool's position and movement. The tool image refers to a visual representation of the surgical tool captured by the industrial camera.

[0109] By extracting multi-marker features from the tool image, embodiments of the present invention can help identify and eliminate abnormal data points caused by image noise, occlusion, or reflection, thereby improving the stability and accuracy of marker tracking. The multi-marker features refer to the various attributes and descriptors of the markers used for tracking and navigation in the surgical tool image.

[0110] As an embodiment of the present invention, extracting multi-marker point features of the tool image includes:

[0111] Preprocessing the tool image to obtain a preprocessed tool image, and identifying a multi-marker point image of the preprocessed tool image;

[0112] Performing contour inspection on the preprocessed multi-marker point image to obtain a multi-marker point contour;

[0113] Identify the contour area and contour perimeter of the multi-marker point contour, and calculate the circular similarity coefficient of the multi-marker point contour based on the contour area and the contour perimeter using the following formula:

[0114]

[0115] in, represents the circular similarity coefficient, represents pi, represents the contour area, Indicates the contour perimeter;

[0116] When the circular similarity coefficient is greater than a preset circular similarity coefficient threshold, the multi-marker point image is filtered to obtain a multi-marker point feature.

[0117] Among them, the preprocessed tool image is a surgical tool image that has been processed through the preprocessing step. The multi-marker point image refers to an image containing multiple marker points. The multi-marker point contour refers to the boundary line of each marker point identified by edge detection or other image segmentation techniques in image processing. The contour area refers to the total area of ​​the area surrounded by the contour of a single marker point in the image. The contour perimeter refers to the length of the boundary line of the contour of the object in the image. The circular similarity coefficient refers to a metric in image processing and computer vision, which is used to describe the degree of similarity between the contour of an object and a perfect circle. The preset circular similarity coefficient threshold refers to a critical value of a circular similarity coefficient set in advance in a specific application.

[0118] The embodiment of the present invention can accurately know the position of the surgical tool by calculating the center point coordinates and drill bit coordinates of the surgical tool based on the multi-marker point feature, thereby achieving accurate positioning and navigation in the surgical navigation system and reducing errors during the operation. The center point coordinates refer to the pixel coordinates of the center mark point among multiple mark points in the surgical tool image. The drill bit coordinates refer to the position of the drill bit tip or specific part on the surgical tool image, which is represented by pixel values.

[0119] Optionally, as an embodiment of the present invention, the calculation of the center point coordinates and the drill bit coordinates of the surgical tool based on the multi-marker point feature can be calculated by positioning the marker points.

[0120] S5. Identify the drilling position coordinates of the enhanced fusion image, calculate the position deviation of the surgical tool based on the drilling position coordinates and the drill bit coordinates, convert the drilling position coordinates and the center point coordinates into drilling space coordinates and center point space coordinates respectively based on the coordinate conversion algorithm, calculate the angular deviation of the surgical tool based on the drilling space coordinates and the center point space coordinates, construct a surgical locking navigation module for the target user based on the angular deviation, the position deviation and the surgical navigation framework, and determine the navigation path of the surgical tool corresponding to the center point based on the surgical locking navigation module.

[0121] The embodiment of the present invention can help avoid damage to surrounding healthy tissue or important structures by identifying the coordinates of the drilling location in the enhanced fusion image, thereby reducing surgical risks. The drilling location refers to the specific location of the target area where a hole needs to be precisely drilled or other operations need to be performed during surgery.

[0122] By calculating the positional deviation of the surgical tool based on the drilling location coordinates and drill bit coordinates, embodiments of the present invention can significantly improve surgical accuracy, helping to prevent the surgical tool from accidentally damaging surrounding healthy tissue or vital structures, thereby reducing surgical risk. The positional deviation refers to the difference between the actual position of the surgical tool and its predetermined target position (e.g., drilling location coordinates) in a surgical navigation system.

[0123] Optionally, as an embodiment of the present invention, the position deviation of the surgical tool calculated based on the drilling position coordinates and the drill bit coordinates can be determined by a Euclidean distance calculation method.

[0124] In embodiments of the present invention, based on the coordinate conversion algorithm, the drilling position coordinates and the center point coordinates are converted into drilling space coordinates and center point space coordinates, respectively. This allows for real-time coordinate conversion, allowing the surgical navigation system to provide real-time guidance during surgery and facilitate real-time adjustment of the position and orientation of surgical tools. The drilling space coordinates refer to the precise coordinate points of the drilling position defined in three-dimensional space. The center point space coordinates refer to the precise coordinate position of the center point of the surgical tool in three-dimensional space.

[0125] As an embodiment of the present invention, the converting of the drilling position coordinates and the center point coordinates into drilling space coordinates and center point space coordinates respectively based on the coordinate conversion algorithm includes:

[0126] Extracting the rotation matrix, translation vector, depth information and focal length parameters of the coordinate transformation algorithm;

[0127] Identify that the center point coordinates correspond to the tool image center coordinates of the tool image;

[0128] Determining the camera focal length parameters and vertical distance of the industrial camera corresponding to the tool image;

[0129] Identifying the fused image center coordinates of the enhanced fused image corresponding to the drilling position coordinates;

[0130] Based on the focal length parameter, the rotation matrix, the translation vector, the camera focal length parameter, the tool image center coordinates, and the fused image center coordinates, the drilling position coordinates and the center point coordinates are converted into drilling space coordinates and center point space coordinates, respectively, using the following formulas:

[0131]

[0132]

[0133] in, The horizontal coordinate of the drilling hole represents the spatial coordinate of the drilling hole, The drilling depth coordinates representing the drilling space coordinates, The vertical coordinate of the drilling hole represents the spatial coordinate of the drilling hole, The horizontal coordinate of the center represents the spatial coordinate of the center point. The central depth coordinate representing the spatial coordinate of the center point, The vertical coordinate of the center point represents the spatial coordinate of the center point. represents the rotation matrix, represents the translation vector, Represents depth information, , The horizontal coordinate of the fused image represents the coordinate of the center of the fused image, The ordinate of the fused image representing the center coordinate of the fused image, The horizontal coordinate of the drilling hole represents the coordinate of the drilling position. The vertical coordinate of the drilling hole represents the coordinates of the drilling position. The tool image abscissa representing the center coordinates of the tool image, The tool image ordinate representing the center coordinate of the tool image, Indicates the vertical distance, Represents the camera focal length parameter, The horizontal coordinate of the center point represents the coordinate of the center point. The vertical coordinate of the center point indicating the coordinates of the center point.

[0134] Among them, the rotation matrix refers to a 3x3 matrix used to describe the rotation of the camera coordinate system relative to the world coordinate system. The translation vector refers to a three-dimensional vector that describes the translation from the camera coordinate system to the world coordinate system. The depth information refers to the distance from the camera to the observed object. The focal length parameter is the distance from the light entering the lens from infinity to the focused imaging plane. The tool image center coordinates refer to the coordinates of the exact center of the tool image. The camera focal length parameter refers to the focal length of the camera lens, which is an important parameter that describes the optical properties of the camera lens. The vertical distance refers to the actual distance in the vertical direction between the camera imaging plane and a specific point in the photographed scene. The fused image center coordinates refer to the coordinates of the exact center of the fused image.

[0135] By calculating the angular deviation of the surgical tool based on the drilling spatial coordinates and the center point spatial coordinates, embodiments of the present invention can ensure that the surgical tool operates along a predetermined trajectory and angle, thereby improving surgical precision. The angular deviation refers to the difference between the actual direction of the surgical tool and the predetermined ideal direction in a surgical navigation system.

[0136] As an embodiment of the present invention, the calculating the angular deviation of the surgical tool based on the drilling space coordinates and the center point space coordinates includes:

[0137] determining an actual direction vector of the surgical tool based on the spatial coordinates of the center point;

[0138] determining an expected direction vector of the surgical tool based on the drilling space coordinates;

[0139] Calculating a pitch angle, a yaw angle, and a roll angle between the actual direction vector and the expected direction vector;

[0140] An angular deviation of the surgical tool is calculated according to the pitch angle, the yaw angle, and the roll angle.

[0141] The actual direction vector refers to the actual direction of movement or pointing of the surgical tool in three-dimensional space. The expected direction vector refers to the expected direction of movement of the surgical tool when performing a surgical task. The pitch angle refers to the angle of rotation of an object around its transverse axis (usually a horizontal axis perpendicular to the object's forward direction). The yaw angle refers to the angle of rotation of an object around its vertical axis (usually an axis pointing toward the center of the Earth). The roll angle refers to the angle of rotation of an object around its longitudinal axis (usually an axis parallel to the object's forward direction).

[0142] Optionally, the pitch angle, yaw angle, and roll angle calculated between the actual direction vector and the expected direction vector may be calculated by using a Rodrigues rotation formula.

[0143] By constructing a surgical lock-on navigation module for the target user based on the angle deviation, position deviation, and surgical navigation framework, embodiments of the present invention can reduce risks caused by incorrect tool position or orientation during surgery, protecting the user's vital structures and organs from accidental damage. The surgical lock-on navigation module is an integrated system that provides real-time navigation information during surgery, ensuring that surgical tools follow a predetermined path and target.

[0144] Optionally, as an embodiment of the present invention, the surgical locking navigation module for the target user constructed according to the angle deviation, the position deviation and the surgical navigation framework may be constructed by a system integration method.

[0145] In embodiments of the present invention, by determining the navigation path corresponding to the center point of the surgical tool based on the surgical lock navigation module, accurate navigation path planning can be used to ensure that the surgical tool moves along a predetermined trajectory, thereby reducing errors during surgery. The navigation path planning refers to a safe path for the surgical tool to reach the target location from the starting position.

[0146] The embodiment of the present invention can provide detailed surgical paths and real-time feedback by constructing a surgical navigation framework for the target user, thereby reducing unnecessary trauma and risks during surgery, and thus improving the accuracy of surgical operations; optionally, the embodiment of the present invention can achieve spatial alignment of the X-ray film and the camera image by using a preset image registration algorithm based on the X-ray film features and the camera image features, so that images of two different modalities can be observed and analyzed in the same coordinate system; the embodiment of the present invention can use the coordinate conversion algorithm to define the coordinate conversion algorithm of the surgical navigation framework based on the X-ray machine imaging principle and the camera imaging principle, and more accurately predict the path and position of surgical tools in the preoperative planning stage, thereby reducing surgery. Uncertainty in; the embodiment of the present invention calculates the center point coordinates and drill bit coordinates of the surgical tool based on the multi-marker point feature, so as to accurately know the position of the surgical tool, thereby achieving accurate positioning and navigation in the surgical navigation system and reducing errors during the operation; the embodiment of the present invention converts the drilling position coordinates and the center point coordinates into drilling space coordinates and center point space coordinates respectively based on the coordinate conversion algorithm, and can convert coordinates in real time, allowing the surgical navigation system to provide real-time guidance during the operation, which helps to adjust the position and direction of the surgical tool in real time. Finally, the embodiment of the present invention determines the navigation path corresponding to the center point of the surgical tool based on the surgical locking navigation module. Through precise navigation path planning, it can ensure that the surgical tool moves according to the predetermined trajectory, thereby reducing errors during the operation. Therefore, the present invention can improve the navigation accuracy of the mixed reality navigation of the distal locking of the femoral interlocking intramedullary nail.

[0147] like Figure 2 , which is a functional module diagram of a mixed reality navigation system for distal locking of a femoral interlocking intramedullary nail provided by an embodiment of the present invention.

[0148] The mixed reality navigation system 200 for distal locking of a femoral interlocking intramedullary nail described in the present invention can be installed in an electronic device. Depending on the functionality implemented, the system can include an image acquisition module 201, an image fusion module 202, a coordinate conversion algorithm module 203, a navigation positioning module 204, and a navigation path determination module 205. A module, also referred to as a unit, is a series of computer program segments that can be executed by an electronic device processor and perform a fixed function, and is stored in the electronic device's memory.

[0149] In this embodiment, the functions of each module / unit are as follows:

[0150] The image acquisition module 201 is used to build a surgical navigation framework for the target user, configure a C-arm X-ray machine and an industrial camera for the surgical navigation framework, use the C-arm X-ray machine to acquire a femur X-ray of the target user in real time, and use the industrial camera to acquire a camera monitoring image of the target user in real time;

[0151] The image fusion module 202 is used to construct a reality enhancement module of the surgical navigation framework, and based on the reality enhancement module, extracts the X-ray film features and camera image features of the femur X-ray film and the camera monitoring screen respectively, and based on the X-ray film features and the camera image features, uses a preset image registration algorithm to register and fuse the femur X-ray film and the camera monitoring screen to obtain a registered fused image, and uses a preset image enhancement algorithm to enhance the registered fused image to obtain an enhanced fused image;

[0152] The coordinate conversion algorithm module 203 is used to analyze the X-ray imaging principle and the camera imaging principle of the C-arm X-ray machine and the industrial camera, and define the coordinate conversion algorithm of the surgical navigation framework based on the X-ray imaging principle and the camera imaging principle;

[0153] The navigation and positioning module 204 is configured to construct a surgical tool in the surgical navigation framework, identify multiple marker points of the surgical tool, acquire a tool image of the surgical tool in real time using the industrial camera, extract features of the multiple marker points of the tool image, and calculate the center point coordinates and drill bit coordinates of the surgical tool based on the multiple marker point features;

[0154] The navigation path determination module 205 is used to identify the drilling position coordinates of the enhanced fusion image, calculate the position deviation of the surgical tool based on the drilling position coordinates and the drill bit coordinates, convert the drilling position coordinates and the center point coordinates into drilling space coordinates and center point space coordinates respectively based on the coordinate conversion algorithm, calculate the angular deviation of the surgical tool based on the drilling space coordinates and the center point space coordinates, construct the surgical locking navigation module for the target user based on the angular deviation, the position deviation and the surgical navigation framework, and determine the navigation path of the surgical tool corresponding to the center point based on the surgical locking navigation module.

[0155] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not limiting. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention.

Claims

1. A mixed reality navigation system for distal locking of femoral interlocking intramedullary nails, characterized in that: The system comprises: An image acquisition module is used to construct a surgical navigation framework for the target user, configure a C-arm X-ray machine and an industrial camera for the surgical navigation framework, use the C-arm X-ray machine to acquire a femur X-ray of the target user in real time, and use the industrial camera to acquire a camera monitoring image of the target user in real time; An image fusion module is used to construct a reality enhancement module of the surgical navigation framework, and based on the reality enhancement module, the X-ray film features and camera image features of the femur X-ray film and the camera monitoring screen are respectively extracted; based on the X-ray film features and the camera image features, the femur X-ray film and the camera monitoring screen are registered and fused using a preset image registration algorithm to obtain a registered fused image; and the registered fused image is enhanced using a preset image enhancement algorithm to obtain an enhanced fused image; A coordinate conversion algorithm module is used to analyze the X-ray imaging principle and the camera imaging principle of the C-arm X-ray machine and the industrial camera, and define a coordinate conversion algorithm of the surgical navigation framework based on the X-ray imaging principle and the camera imaging principle; a navigation and positioning module for constructing a surgical tool in the surgical navigation framework, identifying multiple marker points of the surgical tool, acquiring a tool image of the surgical tool in real time using the industrial camera, extracting features of the multiple marker points of the tool image, and calculating the center point coordinates and drill bit coordinates of the surgical tool based on the multiple marker point features; a navigation path determination module, configured to identify the drilling position coordinates of the enhanced fusion image, calculate the position deviation of the surgical tool based on the drilling position coordinates and the drill bit coordinates, convert the drilling position coordinates and the center point coordinates into drilling space coordinates and the center point space coordinates, respectively, based on the coordinate conversion algorithm, calculate the angular deviation of the surgical tool based on the drilling space coordinates and the center point space coordinates, construct a surgical locking navigation module for the target user based on the angular deviation, the position deviation, and the surgical navigation framework, and determine a navigation path corresponding to the center point of the surgical tool based on the surgical locking navigation module; The extracting multi-marker point features of the tool image includes: Preprocessing the tool image to obtain a preprocessed tool image, and identifying a multi-marker point image of the preprocessed tool image; Performing contour inspection on the multi-marker point image to obtain a multi-marker point contour; Identify the contour area and contour perimeter of the multi-marker point contour, and calculate the circular similarity coefficient of the multi-marker point contour based on the contour area and the contour perimeter using the following formula: ； in, represents the circular similarity coefficient, represents pi, represents the contour area, Indicates the contour perimeter; When the circular similarity coefficient is greater than a preset circular similarity coefficient threshold, filtering the multi-marker point image to obtain a multi-marker point feature; The step of converting the drilling position coordinates and the center point coordinates into drilling space coordinates and center point space coordinates based on the coordinate conversion algorithm includes: Extracting the rotation matrix, translation vector, depth information and focal length parameters of the coordinate transformation algorithm; Identify that the center point coordinates correspond to the tool image center coordinates of the tool image; Determining the camera focal length parameters and vertical distance of the industrial camera corresponding to the tool image; Identifying the fused image center coordinates of the enhanced fused image corresponding to the drilling position coordinates; Based on the focal length parameter, the rotation matrix, the translation vector, the camera focal length parameter, the tool image center coordinates, and the fused image center coordinates, the drilling position coordinates and the center point coordinates are converted into drilling space coordinates and center point space coordinates, respectively, using the following formulas: ； ； in, The horizontal coordinate of the drilling hole represents the spatial coordinate of the drilling hole, The drilling depth coordinates representing the drilling space coordinates, The vertical coordinate of the drilling hole represents the spatial coordinate of the drilling hole, The horizontal coordinate of the center represents the spatial coordinate of the center point. The central depth coordinate representing the spatial coordinate of the center point, The vertical coordinate of the center point represents the spatial coordinate of the center point. represents the rotation matrix, represents the translation vector, Represents depth information, , The horizontal coordinate of the fused image represents the coordinate of the center of the fused image, The ordinate of the fused image representing the center coordinate of the fused image, The horizontal coordinate of the drilling hole represents the coordinate of the drilling position. The vertical coordinate of the drilling hole represents the coordinates of the drilling position. The tool image abscissa representing the center coordinates of the tool image, The tool image ordinate representing the center coordinate of the tool image, Indicates the vertical distance, Represents the camera focal length parameter, The horizontal coordinate of the center point represents the coordinate of the center point. The vertical coordinate of the center point indicating the coordinates of the center point; The calculating the angular deviation of the surgical tool based on the drilling space coordinates and the center point space coordinates includes: determining an actual direction vector of the surgical tool based on the spatial coordinates of the center point; determining an expected direction vector of the surgical tool based on the drilling space coordinates; Calculating a pitch angle, a yaw angle, and a roll angle between the actual direction vector and the expected direction vector; An angular deviation of the surgical tool is calculated according to the pitch angle, the yaw angle, and the roll angle.

2. The mixed reality navigation system for distal locking of a femoral interlocking intramedullary nail according to claim 1, characterized in that: The extracting of X-ray features and camera image features of the femur X-ray and the camera monitoring screen based on the augmented reality module includes: Denoising the femur X-ray and the camera monitoring image using a preset filter to obtain a denoised X-ray and a denoised camera image; performing contrast enhancement on the denoised X-ray film to obtain an enhanced X-ray film, and performing color correction on the denoised camera image to obtain a corrected camera image; Contour recognition is performed on the enhanced X-ray film to obtain a femoral contour, geometric parameters of the femoral contour are calculated, and X-ray shape features of the femoral X-ray film are determined based on the geometric parameters.

3. The mixed reality navigation system for distal locking of a femoral interlocking intramedullary nail according to claim 2, characterized in that: The extracting of X-ray features and camera image features of the femur X-ray and the camera monitoring screen respectively based on the augmented reality module includes: performing shape detection on the calibrated camera screen to obtain camera screen shape features; Extracting X-ray film texture features of the enhanced X-ray film and camera image texture features of the corrected camera image respectively using a preset local binary pattern; Identify the X-ray film structure features of the enhanced X-ray film, and extract the X-ray film position features of the enhanced X-ray film based on the X-ray film structure features.

4. The mixed reality navigation system for distal locking of a femoral interlocking intramedullary nail according to claim 3, characterized in that: The extracting, based on the augmented reality module, X-ray features and camera image features of the femur X-ray and the camera monitoring screen respectively includes: extracting screen key points of the corrected camera screen, and determining camera screen spatial features of the corrected camera screen based on the screen key points; Determining X-ray features of the femur X-ray according to the X-ray shape feature, the X-ray texture feature, the X-ray structure feature, and the X-ray position feature; The camera image features of the camera monitoring screen are determined according to the camera screen shape features, the camera screen texture features and the camera screen space features.

5. The mixed reality navigation system for distal locking of a femoral interlocking intramedullary nail according to claim 1, characterized in that: The method of registering and fusing the femur X-ray film and the camera monitoring screen using a preset image registration algorithm based on the X-ray film features and the camera image features to obtain a registered fused image includes: Extracting X-ray film feature points for the X-ray film features and extracting camera image feature points for the camera image features; An X-ray film descriptor analyzing the X-ray film feature points and a camera image descriptor analyzing the camera image feature points; unifying the descriptor dimensions of the X-ray film descriptor and the camera image descriptor; Based on the descriptor dimension, an inter-descriptor distance between the X-ray film descriptor and the camera image descriptor is calculated.

6. The mixed reality navigation system for distal locking of a femoral interlocking intramedullary nail according to claim 5, characterized in that: The method further comprises: registering and fusing the femur X-ray film and the camera monitoring image using a preset image registration algorithm based on the X-ray film features and the camera image features to obtain a registered fused image; Matching the X-ray film features and the camera image features according to the distance between the descriptors to obtain a feature matching relationship; Based on the feature matching relationship, construct an image transformation model of the femoral X-ray and the camera monitoring screen; The femur X-ray film and the camera monitoring image are fused according to the image transformation model to obtain a registered fused image.

7. The mixed reality navigation system for distal locking of a femoral interlocking intramedullary nail according to claim 6, characterized in that: The step of constructing an image transformation model of the femur X-ray and the camera monitoring screen based on the feature matching relationship includes: defining X-ray feature point coordinates of the femur X-ray film, and mapping the image feature point coordinates of the camera monitoring screen based on the feature matching relationship; Constructing an initial affine transformation model of the femur X-ray film and the camera monitoring screen, and determining transformation parameters of the initial affine transformation model; Based on the transformation parameters, transform the femoral X-ray film using the initial affine transformation model to obtain a transformed femoral X-ray film; Calculating the transformed X-ray feature point coordinates of the transformed femoral X-ray film based on the X-ray feature point coordinates; Determining an objective function and a transformation parameter vector of the initial affine transformation model based on the transformation parameters, the X-ray feature point coordinates, and the image feature point coordinates; Calculating the transformation residual value of the X-ray feature point coordinates and the image feature point coordinates according to the objective function and the transformation parameter vector; When the transformation residual value is less than a preset transformation residual threshold, the initial affine transformation model is used as the image transformation model of the femur X-ray and the camera monitoring screen.

Images