Femoral interlocking intramedullary nail far-end locking mixed reality navigation method

Abstract

The invention relates to the technical field of augmented reality, and discloses a femoral interlocking intramedullary nail far-end locking mixed reality navigation method and system, and the method comprises the steps: constructing a surgical navigation frame of a target user, and collecting a femoral X-ray film and a camera monitoring image of the target user in real time; the thighbone X-ray film and the camera monitoring picture are subjected to registration fusion, a registration fusion image is enhanced, and an enhanced fusion image is obtained; defining a coordinate conversion algorithm of the surgical navigation framework; constructing a surgical tool of the surgical navigation framework, collecting a tool image of the surgical tool, extracting a multi-mark-point feature of the tool image, and calculating a center point coordinate and a drill bit coordinate of the surgical tool; calculating the position deviation of the surgical tool, calculating the angle deviation of the surgical tool, constructing a surgical locking navigation module of the target user, and determining a navigation path of a central point corresponding to the surgical tool. According to the method, the navigation precision of the femoral interlocking intramedullary nail far-end locking mixed reality navigation can be improved.

Description

Technical Field

[0001] The present invention relates to the field of augmented reality technology, and in particular to a mixed reality navigation method and system for distal locking of a femoral interlocking intramedullary nail. Background Art

[0002] The femoral interlocking intramedullary nail is an internal fixation device used to treat femoral fractures. It works by inserting a long, hollow nail into the femoral medullary cavity, and then driving locking nails into the distal and proximal ends of the femur through the locking holes on the nail to fix the fracture site. Accurate locking nail placement helps provide more stable fixation, promote fracture healing, and reduce the risk of postoperative fracture displacement, thereby improving the overall surgical outcome.

[0003] Traditional intramedullary nail locking surgery relies on continuous fluoroscopy of the fracture site by radiological instruments, and the position of the distal screw hole is determined based on the fluoroscopy results. This method may cause unclear images and certain errors due to the overlap of different tissues and organs on the X-ray film, and the positioning accuracy is low, which leads to long radiation exposure time and affects the health of the user. Summary of the invention

[0004] The present invention provides a mixed reality navigation method and system for distal locking of a femoral interlocking intramedullary nail, the main purpose of which is to improve the navigation accuracy of the mixed reality navigation for distal locking of a femoral interlocking intramedullary nail.

[0005] To achieve the above-mentioned purpose, the present invention provides a mixed reality navigation method for distal locking of a femoral interlocking intramedullary nail, comprising:

[0006] Constructing a surgical navigation framework for the target user, configuring a C-arm X-ray machine and an industrial camera for the surgical navigation framework, using the C-arm X-ray machine to collect femur X-rays of the target user in real time, and using the industrial camera to collect camera monitoring images of the target user in real time;

[0007] Constructing a reality enhancement module of the surgical navigation framework, based on the reality enhancement module, respectively extracting the X-ray film features and camera image features of the femur X-ray film and the camera monitoring screen, based on the X-ray film features and the camera image features, using a preset image registration algorithm to register and fuse the femur X-ray film and the camera monitoring screen to obtain a registered fused image, and using a preset image enhancement algorithm to enhance the registered fused image to obtain an enhanced fused image;

[0008] Analyze the X-ray machine imaging principle and the camera imaging principle of the C-arm X-ray machine and the industrial camera, and define the coordinate transformation algorithm of the surgical navigation framework based on the X-ray machine imaging principle and the camera imaging principle;

[0009] Constructing a surgical tool of the surgical navigation framework, identifying multiple marking points of the surgical tool, using the industrial camera to collect a tool image of the surgical tool in real time, extracting multi-marking point features of the tool image, and calculating the center point coordinates and drill bit coordinates of the surgical tool based on the multi-marking point features;

[0010] Identify the drilling position coordinates of the enhanced fusion image, calculate the position deviation of the surgical tool based on the drilling position coordinates and the drill bit coordinates, convert the drilling position coordinates and the center point coordinates into drilling space coordinates and center point space coordinates respectively based on the coordinate conversion algorithm, calculate the angular deviation of the surgical tool based on the drilling space coordinates and the center point space coordinates, construct a surgical locking navigation module for the target user based on the angular deviation, the position deviation and the surgical navigation framework, and determine the navigation path of the surgical tool corresponding to the center point based on the surgical locking navigation module.

[0011] In order to solve the above problems, the present invention also provides a mixed reality navigation system for distal locking of a femoral interlocking intramedullary nail, the system comprising:

[0012] An image acquisition module is used to construct a surgical navigation framework for a target user, configure a C-arm X-ray machine and an industrial camera for the surgical navigation framework, use the C-arm X-ray machine to acquire a femur X-ray of the target user in real time, and use the industrial camera to acquire a camera monitoring screen of the target user in real time;

[0013] An image fusion module is used to construct a reality enhancement module of the surgical navigation framework, based on which the X-ray film features and camera image features of the femur X-ray film and the camera monitoring screen are respectively extracted, based on the X-ray film features and the camera image features, the femur X-ray film and the camera monitoring screen are registered and fused using a preset image registration algorithm to obtain a registered fused image, and the registered fused image is enhanced using a preset image enhancement algorithm to obtain an enhanced fused image;

[0014] A coordinate conversion algorithm module, used for analyzing the X-ray machine imaging principle and the camera imaging principle of the C-arm X-ray machine and the industrial camera, and defining the coordinate conversion algorithm of the surgical navigation framework based on the X-ray machine imaging principle and the camera imaging principle;

[0015] A navigation and positioning module, used for constructing a surgical tool of the surgical navigation framework, identifying multiple marking points of the surgical tool, using the industrial camera to collect a tool image of the surgical tool in real time, extracting features of the multiple marking points of the tool image, and calculating the center point coordinates and drill bit coordinates of the surgical tool based on the features of the multiple marking points;

[0016] A navigation path determination module is used to identify the drilling position coordinates of the enhanced fusion image, calculate the position deviation of the surgical tool based on the drilling position coordinates and the drill bit coordinates, convert the drilling position coordinates and the center point coordinates into drilling space coordinates and center point space coordinates respectively based on the coordinate conversion algorithm, calculate the angular deviation of the surgical tool based on the drilling space coordinates and the center point space coordinates, construct a surgical locking navigation module for the target user based on the angular deviation, the position deviation and the surgical navigation framework, and determine the navigation path of the surgical tool corresponding to the center point based on the surgical locking navigation module.

[0017] The embodiment of the present invention can provide detailed surgical paths and real-time feedback by constructing a surgical navigation framework for the target user, thereby reducing unnecessary trauma and risks during surgery, thereby improving the accuracy of surgical operations; optionally, the embodiment of the present invention can achieve spatial alignment of the X-ray film and the camera image by using a preset image registration algorithm based on the X-ray film features and the camera image features to obtain a registered fused image, so that images of two different modalities can be observed and analyzed in the same coordinate system; the embodiment of the present invention can use a coordinate conversion algorithm to define the coordinate conversion algorithm of the surgical navigation framework based on the X-ray machine imaging principle and the camera imaging principle, so as to more accurately predict the path and position of surgical tools in the preoperative planning stage, thereby reducing surgery. uncertainty in; the embodiment of the present invention can accurately know the position of the surgical tool by calculating the center point coordinates and drill bit coordinates of the surgical tool based on the multi-marker point features, thereby achieving accurate positioning and navigation in the surgical navigation system and reducing errors during the operation; the embodiment of the present invention can convert the drilling position coordinates and the center point coordinates into drilling space coordinates and center point space coordinates respectively based on the coordinate conversion algorithm, and can convert coordinates in real time, allowing the surgical navigation system to provide real-time guidance during the operation, which helps to adjust the position and direction of the surgical tool in real time. Finally, the embodiment of the present invention determines the navigation path corresponding to the center point of the surgical tool based on the surgical locking navigation module. Through accurate navigation path planning, it can ensure that the surgical tool moves according to the predetermined trajectory and reduce errors in the operation. Therefore, the present invention can improve the navigation accuracy of the mixed reality navigation of the distal locking of the femoral interlocking intramedullary nail. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 A schematic flow chart of a mixed reality navigation method for distal locking of a femoral interlocking intramedullary nail provided by an embodiment of the present invention;

[0019] Figure 2 A functional module diagram of a mixed reality navigation system for distal locking of a femoral interlocking intramedullary nail provided by an embodiment of the present invention;

[0020] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0021] It should be understood that the specific embodiments described herein are only used to explain the present invention, and are not used to limit the present invention.

[0022] The embodiment of the present application provides a mixed reality navigation method for distal locking of a femoral interlocking intramedullary nail. The execution subject of the mixed reality navigation method for distal locking of a femoral interlocking intramedullary nail includes but is not limited to at least one of the electronic devices such as a server and a terminal that can be configured to execute the method provided in the embodiment of the present application. In other words, the mixed reality navigation method for distal locking of a femoral interlocking intramedullary nail can be executed by software or hardware installed on a terminal device or a server device, and the software can be a blockchain platform. The server includes but is not limited to: a single server, a server cluster, a cloud server or a cloud server cluster, etc. The server can be an independent server, or it can be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (Content Delivery Network, CDN), and big data and artificial intelligence platforms.

[0023] Reference Figure 1 FIG. 1 is a flow chart of a mixed reality navigation method for distal locking of a femoral interlocking intramedullary nail provided by an embodiment of the present invention. In this embodiment, the mixed reality navigation method for distal locking of a femoral interlocking intramedullary nail includes:

[0024] S1. Construct a surgical navigation framework for the target user, configure a C-arm X-ray machine and an industrial camera for the surgical navigation framework, use the C-arm X-ray machine to collect femur X-rays of the target user in real time, and use the industrial camera to collect camera monitoring images of the target user in real time.

[0025] The embodiment of the present invention can provide detailed surgical paths and real-time feedback by constructing a surgical navigation framework for target users, thereby reducing unnecessary trauma and risks during surgery and improving the accuracy of surgical operations. The surgical navigation framework refers to a system that integrates multiple medical imaging devices and computer-aided technologies.

[0026] Optionally, as an embodiment of the present invention, the surgical navigation framework for constructing the target user may be constructed by tracking and positioning technology.

[0027] The embodiment of the present invention can obtain accurate internal anatomical structure information by configuring the C-arm X-ray machine and industrial camera of the surgical navigation framework, and can also capture the visual information of the surgical site in real time, thereby improving the accuracy of surgical operations. The C-arm X-ray machine refers to a C-arm X-ray machine, which is a mobile X-ray imaging device widely used in operating rooms and emergency rooms. The industrial camera refers to a special camera used in the field of industrial automation for detection, measurement, identification and monitoring.

[0028] The embodiment of the present invention can reduce the radiation exposure of patients and medical staff by using the C-arm X-ray machine to collect the femur X-ray of the target user in real time, which helps to avoid misoperation during surgery, reduce damage to surrounding healthy tissues, and improve surgical safety. The femur X-ray refers to an X-ray image specifically used to display and analyze the internal structure and morphology of the femur (thigh bone).

[0029] Optionally, as an embodiment of the present invention, the real-time collection of the femur X-ray of the target user by using the C-arm X-ray machine can be obtained by X-ray imaging technology.

[0030] The embodiment of the present invention can provide a real-time video image of the surgical site by using the industrial camera to collect the camera monitoring image of the target user in real time, so that the surgical process can be observed intuitively instead of relying solely on two-dimensional images such as X-rays. The camera monitoring image refers to the video image of the surgical site captured and displayed by the industrial camera.

[0031] S2. Construct a reality enhancement module of the surgical navigation framework, and based on the reality enhancement module, respectively extract the X-ray film features and camera image features of the femur X-ray film and the camera monitoring screen, and based on the X-ray film features and the camera image features, use a preset image registration algorithm to register and fuse the femur X-ray film and the camera monitoring screen to obtain a registered fused image, and use a preset image enhancement algorithm to enhance the registered fused image to obtain an enhanced fused image.

[0032] The embodiment of the present invention can provide intuitive image information by constructing the reality enhancement module of the surgical navigation framework, reduce uncertainty in surgery, and thus reduce surgical risks. The reality enhancement module refers to a software and hardware complex integrated in the surgical navigation system, which uses computer vision, image processing and reality enhancement technology to combine virtual image information with the surgical scene in the real world, providing surgeons with more intuitive and rich surgical auxiliary information.

[0033] Optionally, as an embodiment of the present invention, the reality enhancement module for constructing the surgical navigation framework may be constructed by using reality enhancement rendering technology.

[0034] The embodiment of the present invention can achieve accurate registration between the X-ray film and the real-time camera monitoring screen by respectively extracting the X-ray film features and camera image features of the femur X-ray film and the camera monitoring screen based on the reality augmented module. This helps to ensure the accurate alignment of the virtual X-ray image with the real surgical scene and improve the accuracy of surgical navigation. Among them, the X-ray film features refer to a series of information points or attributes extracted during the X-ray film image processing and analysis process for characterizing the content of the X-ray image. The camera image features refer to information points or attributes extracted from the real-time image captured by the camera for describing the image content, structure and appearance.

[0035] As an embodiment of the present invention, the extracting of X-ray film features and camera image features of the femur X-ray film and the camera monitoring screen respectively based on the augmented reality module includes:

[0036] De-noising the femur X-ray film and the camera monitoring picture by using a preset filter to obtain a de-noised X-ray film and a de-noised camera picture;

[0037] Performing contrast enhancement on the denoised X-ray film to obtain an enhanced X-ray film, and performing color correction on the denoised camera image to obtain a corrected camera image;

[0038] Performing contour recognition on the enhanced X-ray film to obtain a femoral contour, calculating geometric parameters of the femoral contour, and determining X-ray shape features of the femoral X-ray film based on the geometric parameters;

[0039] Performing shape detection on the correction camera image to obtain camera image shape features;

[0040] Using a preset local binary pattern, respectively extracting X-ray film texture features of the enhanced X-ray film and camera image texture features of the corrected camera image;

[0041] Identifying X-ray film structural features of the enhanced X-ray film, and extracting X-ray film position features of the enhanced X-ray film according to the X-ray film structural features;

[0042] Extracting picture key points of the corrected camera picture, and determining camera picture spatial features of the corrected camera picture based on the picture key points;

[0043] Determine the X-ray film features of the femur X-ray film according to the X-ray film shape features, the X-ray film texture features, the X-ray film structure features and the X-ray film position features;

[0044] The camera image features of the camera monitoring screen are determined according to the camera screen shape features, the camera screen texture features and the camera screen space features.

[0045] Among them, the preset filter refers to an algorithm designed or selected for a specific purpose, which is used to modify or enhance image data. The denoised X-ray refers to an X-ray image after denoising. The denoised camera image refers to an image in which the noise in the image captured by the camera is reduced by applying a specific image processing technique. The enhanced X-ray refers to an X-ray image that has been subjected to contrast enhancement processing, the purpose of which is to improve the visual readability of the image and highlight specific anatomical structures. The corrected camera image refers to the image captured by the camera after color correction and other related image processing steps. The femoral contour refers to the visible external boundary line of the femur in the image. The geometric parameter refers to a quantitative measurement indicator used to describe the femoral contour or a specific part thereof. The X-ray shape feature refers to quantitative or qualitative information extracted from the X-ray to describe the contour, size, proportion and geometry of the femur or other anatomical structure. The camera image shape feature refers to information extracted from the image captured by the camera to describe the contour, geometry and structure of an object or a specific area in the scene. The preset local binary pattern refers to a feature descriptor used for image texture analysis, which works by comparing the brightness value of a pixel with its neighboring pixels, comparing each pixel with its surrounding pixels, and encoding the comparison result into a binary number. The X-ray film texture feature refers to a quantitative or qualitative attribute used to describe the internal structure and texture information of an image in X-ray imaging. The camera screen texture feature refers to information extracted from an image captured by a camera to describe the surface structural details and arrangement pattern of an image. The X-ray film structural feature refers to information extracted from an X-ray film to describe the geometric morphology, tissue layout and mutual relationship of bones, organs or other anatomical structures. The X-ray film position feature refers to quantitative information extracted from an X-ray film to describe the specific spatial position of an anatomical structure or lesion in an image. The screen key point refers to an important position point in an image that can be used to represent image features or content. The camera screen spatial feature refers to a feature extracted from an image captured by a camera to describe the position, layout and mutual relationship of objects and scenes in an image in three-dimensional space.

[0046] Optionally, the contrast of the denoised X-ray film is enhanced, and the enhanced X-ray film obtained can be enhanced by a histogram equalization method.

[0047] Optionally, the identification of the X-ray film structural features of the enhanced X-ray film can be performed by a Hough transform method.

[0048] The embodiment of the present invention can achieve spatial alignment of the X-ray film and the camera image by using a preset image registration algorithm to register and fuse the femur X-ray film and the camera monitoring screen based on the X-ray film features and the camera image features, so that the images of two different modalities can be observed and analyzed in the same coordinate system. The registered fused image refers to a single image obtained after image registration and fusion technology processing.

[0049] As an embodiment of the present invention, based on the X-ray film features and the camera image features, the femur X-ray film and the camera monitoring screen are registered and fused using a preset image registration algorithm to obtain a registered fused image, including:

[0050] Extracting X-ray film feature points for the X-ray film features and extracting camera image feature points for the camera image features;

[0051] An X-ray film descriptor analyzing the X-ray film feature points and a camera image descriptor analyzing the camera image feature points;

[0052] unifying the descriptor dimensions of the X-ray film descriptor and the camera image descriptor;

[0053] Based on the descriptor dimension, the inter-descriptor distance between the X-ray film descriptor and the camera image descriptor is calculated using the following formula:

[0054]

[0055] Among them, L(M X ,M J ) represents the distance between descriptors, M X Represents the X-ray film descriptor, M J represents the camera image descriptor, X i represents the i-th X-ray element in the X-ray descriptor, J i represents the i-th image element of the camera image descriptor, and n represents the descriptor dimension;

[0056] According to the distance between the descriptors, matching the X-ray film features with the camera image features to obtain a feature matching relationship;

[0057] Based on the feature matching relationship, construct an image transformation model of the femur X-ray and the camera monitoring screen;

[0058] According to the image transformation model, the femur X-ray film and the camera monitoring screen are fused to obtain a registered fused image.

[0059] Among them, the X-ray feature points refer to significant points in the X-ray image that can be detected and used for subsequent image analysis and processing. The camera image feature points refer to points with specific attributes detected in the image captured by the camera. The X-ray descriptor refers to a vector used to describe the local image features around the feature points in the X-ray image. The camera image descriptor refers to a vector used to describe the local image features of the feature points extracted from the image captured by the camera. The descriptor dimension refers to the length of the feature descriptor vector, that is, the number of elements contained in the descriptor. The inter-descriptor distance refers to the degree of difference between two feature descriptor vectors. The feature matching relationship refers to the corresponding relationship established between the feature points in the two images. The image transformation model refers to a mathematical model used to describe how one image is mapped to another image through geometric transformation or other forms of transformation.

[0060] Optionally, the X-ray film descriptor for analyzing the X-ray film feature points may be obtained by analyzing using an accelerated robust feature algorithm.

[0061] Optionally, the X-ray film features and the camera image features are matched according to the distance between the descriptors, and the feature matching relationship can be obtained by a nearest neighbor matching method.

[0062] Optionally, constructing an image transformation model of the femur X-ray and the camera monitoring screen based on the feature matching relationship includes:

[0063] Defining the X-ray feature point coordinates of the femur X-ray film, and mapping the image feature point coordinates of the camera monitoring screen based on the feature matching relationship;

[0064] Constructing an initial affine transformation model of the femur X-ray film and the camera monitoring screen, and determining transformation parameters of the initial affine transformation model;

[0065] Based on the transformation parameters, transform the femoral X-ray film using the initial affine transformation model to obtain a transformed femoral X-ray film;

[0066] Based on the X-ray feature point coordinates, calculating the transformed X-ray feature point coordinates of the transformed femoral X-ray film;

[0067] Based on the transformation parameters, the X-ray feature point coordinates and the image feature point coordinates, the objective function and transformation parameter vector of the initial affine transformation model are determined using the following formula:

[0068]

[0069] u=(a,b,c,d,h r ,h t ) T

[0070] Where F(u) represents the objective function, u represents the transformation parameter vector, N represents the total number of transformed X-ray feature point coordinates, and r bx,q The horizontal coordinate of the transformed X-ray feature point representing the coordinate of the qth transformed X-ray feature point, t bx,q The ordinate of the transformed X-ray feature point, r, represents the qth transformed X-ray feature point coordinate. J,q The horizontal coordinate of the image feature point representing the coordinate of the qth image feature point, t J,q The ordinate of the image feature point represents the coordinate of the qth image feature point, a represents the horizontal axis scaling parameter corresponding to the transformation parameter, b represents the horizontal axis rotation parameter corresponding to the transformation parameter, and h represents the horizontal axis rotation parameter corresponding to the transformation parameter. r indicates that the transformation parameter corresponds to the horizontal axis shear parameter, c indicates that the transformation parameter corresponds to the vertical axis scaling parameter, d indicates that the transformation parameter corresponds to the vertical axis rotation parameter, h t Indicates that the transformation parameter corresponds to the longitudinal shear parameter;

[0071] Calculating the transformation residual value of the X-ray feature point coordinates and the image feature point coordinates according to the objective function and the transformation parameter vector;

[0072] When the transformation residual value is less than a preset transformation residual threshold, the initial affine transformation model is used as the image transformation model of the femur X-ray and the camera monitoring screen.

[0073] Among them, the X-ray feature point coordinates refer to the positions of feature points on the femur X-ray film. The image feature point coordinates refer to the positions of feature points on the camera monitoring screen. The initial affine transformation model refers to a mathematical model that describes how to transform a set of points from one coordinate system to another coordinate system in a two-dimensional space. The transformation parameters refer to specific values ​​used to define the image transformation model. The transformed femur X-ray film refers to the image obtained by transforming the femur X-ray film through the initial affine transformation model. The transformed X-ray feature point coordinates refer to the positions of the corresponding feature points on the transformed femur X-ray film. The objective function refers to a mathematical function used to evaluate the registration quality and guide the optimization process during the image registration process. The transformation parameter vector refers to a vector containing all parameters used to define the image transformation model. The horizontal axis scaling parameter refers to a parameter specifically used to adjust the image scaling ratio in the horizontal direction in the image transformation model. The horizontal axis rotation parameter refers to a parameter specifically used to adjust the image rotation ratio in the horizontal direction in the image transformation model, and the horizontal axis shearing parameter refers to a parameter specifically used to adjust the image shearing ratio in the horizontal direction in the image transformation model. The longitudinal axis scaling parameter refers to a parameter specifically used to adjust the scaling ratio of the image in the vertical direction in the image transformation model. The longitudinal axis rotation parameter refers to a parameter specifically used to adjust the rotation ratio of the image in the vertical direction in the image transformation model. The longitudinal axis shearing parameter refers to a parameter specifically used to adjust the shearing ratio of the image in the vertical direction in the image transformation model. The transformation residual value refers to the difference between the transformed point and the actual target point during the image transformation process. The preset transformation residual threshold refers to a pre-set value, which indicates the highest acceptable residual value when performing image transformation.

[0074] Optionally, the calculating the transformed X-ray feature point coordinates of the transformed femoral X-ray film based on the X-ray feature point coordinates includes:

[0075] Based on the X-ray feature point coordinates, the transformed X-ray feature point coordinates of the transformed femoral X-ray film are calculated using the following formula:

[0076]

[0077] Among them, r bx The transformed horizontal coordinates of the transformed X-ray feature point coordinates, t bx Represents the transformed ordinate of the X-ray feature point coordinates, r x The horizontal coordinate of the X-ray feature point coordinate, t x The ordinate represents the coordinates of the X-ray feature point. a represents the transformation parameter corresponding to the horizontal axis scaling parameter. b represents the transformation parameter corresponding to the horizontal axis rotation parameter. h represents the transformation parameter corresponding to the horizontal axis rotation parameter. rindicates that the transformation parameter corresponds to the horizontal axis shear parameter, c indicates that the transformation parameter corresponds to the vertical axis scaling parameter, d indicates that the transformation parameter corresponds to the vertical axis rotation parameter, h t Indicates that the transformation parameter corresponds to the longitudinal shear parameter;

[0078] In the embodiment of the present invention, the registered fused image is enhanced by using a preset image enhancement algorithm to obtain an enhanced fused image, which can make the differences between different tissues or structures more obvious and help distinguish key features in the image. The enhanced fused image refers to a fused image processed by image enhancement technology.

[0079] As an embodiment of the present invention, the step of enhancing the registered fused image by using a preset image enhancement algorithm to obtain an enhanced fused image includes:

[0080] Performing edge detection on the registered fused image to obtain the edge of the fused image;

[0081] Determining target pixel points that need to be enhanced in the registered fused image according to the edge of the fused image;

[0082] Determine the pixel point coordinates and pixel values ​​of the target pixel point, and determine the image target intensity of the registered fused image;

[0083] According to the pixel point coordinates, the target pixel value and the image target intensity, the image enhancement coefficient of the registered fused image is calculated using the following formula:

[0084]

[0085] Among them, α represents the image enhancement coefficient, Q m represents the image target intensity, S represents the pixel value, e represents the horizontal coordinate of the pixel point corresponding to the pixel point coordinate, and f represents the vertical coordinate of the pixel point corresponding to the pixel point coordinate. represents partial derivative;

[0086] Based on the image enhancement coefficient, the registered fused image is enhanced using a preset image enhancement algorithm to obtain an enhanced fused image.

[0087] Among them, the fused image edge refers to the important edge features of the registered fused image. The target pixel point refers to the specific pixel position selected as the enhancement object during the image enhancement process. The pixel point coordinates refer to the position identifier of the target pixel point. The pixel value refers to the numerical value represented by the target pixel point. The image target intensity refers to the image intensity level that is expected to be achieved or optimized in the image processing or analysis task. The image enhancement coefficient refers to the numerical value used to adjust the image pixel intensity.

[0088] Optionally, the edge detection is performed on the registered fused image to obtain the edge of the fused image by using a Canny edge detector.

[0089] S3. Analyze the X-ray machine imaging principle and the camera imaging principle of the C-arm X-ray machine and the industrial camera, and define the coordinate transformation algorithm of the surgical navigation framework based on the X-ray machine imaging principle and the camera imaging principle.

[0090] The embodiment of the present invention can more accurately locate surgical tools and target anatomical structures, improve surgical accuracy, and reduce surgical risks by analyzing the X-ray machine imaging principle and camera imaging principle of the C-arm X-ray machine and the industrial camera. The X-ray machine imaging principle refers to the physical process of the X-ray machine using X-rays to penetrate objects and generate images. The camera imaging principle refers to the process of the camera capturing light in the scene through an optical system and converting it into a visible image.

[0091] Optionally, as an embodiment of the present invention, the analysis of the X-ray machine imaging principle and the camera imaging principle of the C-arm X-ray machine and the industrial camera can be analyzed by a schematic diagram analysis method.

[0092] The embodiment of the present invention defines the coordinate transformation algorithm of the surgical navigation framework based on the imaging principle of the X-ray machine and the imaging principle of the camera, and can use the coordinate transformation algorithm to more accurately predict the path and position of the surgical tool in the preoperative planning stage, thereby reducing the uncertainty in the operation. The coordinate transformation algorithm refers to a mathematical method used to transform the position information of a point or object in one coordinate system into another coordinate system.

[0093] As an embodiment of the present invention, the coordinate conversion algorithm of the surgical navigation framework is defined based on the imaging principle of the X-ray machine and the imaging principle of the camera, including:

[0094] Based on the camera imaging principle, determining the camera intrinsic parameters and camera extrinsic parameters of the industrial camera corresponding to the camera imaging principle;

[0095] Extracting target pixel coordinates of the surgical navigation frame and constructing a femoral geometric model of the surgical navigation frame;

[0096] Constructing a camera coordinate conversion algorithm of the target pixel coordinates according to the camera intrinsic parameters, the camera extrinsic parameters and the femoral geometric model;

[0097] Based on the imaging principle of the X-ray machine, determining the baseline length and X-ray focal length of the C-arm X-ray machine corresponding to the imaging principle of the X-ray machine;

[0098] Determining an imaging equation of the C-arm X-ray machine according to the baseline length and the X-ray focal length;

[0099] Determining an X-ray coordinate conversion algorithm of the target pixel coordinates according to the imaging equation and the femoral geometric model;

[0100] The X-ray coordinate conversion algorithm and the camera coordinate conversion algorithm are integrated to obtain a coordinate conversion algorithm.

[0101] Among them, the camera intrinsic parameters refer to the inherent properties of the camera, which describe the relationship between the camera lens and the imaging sensor. The camera extrinsic parameters refer to the position and orientation of the camera relative to the observed scene. The target pixel coordinates refer to the coordinates of the position of the target object or specific feature in the image coordinate system, expressed in pixel units. The femur geometric model refers to a coordinate system used to describe the position of the femur in three-dimensional space. The camera coordinate conversion algorithm refers to a series of mathematical processes and methods for converting the image coordinates (usually two-dimensional coordinates) captured by the camera into three-dimensional space coordinates in the real world. The baseline length refers to the distance between two imaging sensors (such as camera lenses or detectors of X-ray machines). In the X-ray tube, the distance from the X-ray emission source (usually the anode) to the focus of the X-ray focus. The imaging equation refers to the mathematical equation that describes the relationship between a point in the object space and its corresponding pixel on the image plane during the image formation process. The X-ray coordinate conversion algorithm refers to a series of mathematical processing steps for converting the two-dimensional pixel coordinates in the X-ray image into three-dimensional space coordinates in the real world.

[0102] Optionally, the femoral geometry model for constructing the surgical navigation framework may be constructed by computer-aided design.

[0103] Optionally, the imaging equation of the C-arm X-ray machine can be determined by establishing a perspective projection relationship based on the baseline length and the X-ray focal length.

[0104] S4. Construct the surgical tools of the surgical navigation framework, identify the multiple marking points of the surgical tools, use the industrial camera to collect the tool image of the surgical tools in real time, extract the multi-marking point features of the tool image, and calculate the center point coordinates and drill bit coordinates of the surgical tools based on the multi-marking point features.

[0105] The embodiment of the present invention can track the position and direction of the surgical tools in real time by constructing the surgical navigation framework, thereby performing surgical operations more accurately and reducing surgical errors. The surgical tools refer to various precision medical instruments used in the surgical navigation framework.

[0106] Optionally, as an embodiment of the present invention, the surgical tools for constructing the surgical navigation framework may be constructed by optical tracking technology.

[0107] The embodiment of the present invention can more accurately track the position and direction of the surgical tool in three-dimensional space by identifying multiple marking points of the surgical tool, thereby reducing tracking errors. The multiple marking points refer to multiple specific identification points arranged on the surgical tool, which are used in the surgical navigation system to track and determine the position and direction of the tool.

[0108] The embodiment of the present invention can provide real-time visual feedback of the surgical tool during surgery by using the industrial camera to collect the tool image of the surgical tool in real time, and can intuitively determine the position and movement of the tool. The tool image refers to the visual representation of the surgical tool captured by the industrial camera.

[0109] The embodiment of the present invention can help identify and exclude abnormal data points caused by image noise, occlusion or reflection by extracting multi-marker point features of the tool image, thereby improving the stability and accuracy of marker point tracking. The multi-marker point features refer to various attributes and descriptors of marker points used for tracking and navigation in the surgical tool image.

[0110] As an embodiment of the present invention, the step of extracting multi-marker point features of the tool image includes:

[0111] Preprocessing the tool image to obtain a preprocessed tool image, and identifying a multi-marker point image of the preprocessed tool image;

[0112] Performing contour inspection on the preprocessed multi-marker point image to obtain a multi-marker point contour;

[0113] Identify the contour area and contour perimeter of the multi-marker point contour, and calculate the circular similarity coefficient of the multi-marker point contour based on the contour area and the contour perimeter using the following formula:

[0114]

[0115] Among them, θ represents the circular similarity coefficient, π represents the circumference, B represents the contour area, and Z represents the contour circumference;

[0116] When the circular similarity coefficient is greater than a preset circular similarity coefficient threshold, the multi-marker point image is filtered to obtain a multi-marker point feature.

[0117] Among them, the preprocessed tool image is a surgical tool image that has been processed by the preprocessing step. The multi-marker point image refers to an image containing multiple marker points. The multi-marker point contour refers to the boundary line of each marker point identified by edge detection or other image segmentation techniques in image processing. The contour area refers to the total area of ​​the area surrounded by the contour of a single marker point in the image. The contour perimeter refers to the length of the boundary line of the contour of an object in the image. The circular similarity coefficient refers to a metric in image processing and computer vision, which is used to describe the degree of similarity between an object contour and a perfect circle. The preset circular similarity coefficient threshold refers to a critical value of a circular similarity coefficient set in advance in a specific application.

[0118] The embodiment of the present invention can accurately know the position of the surgical tool by calculating the center point coordinates and drill coordinates of the surgical tool based on the multi-marker point features, thereby achieving accurate positioning and navigation in the surgical navigation system and reducing errors during the operation. The center point coordinates refer to the pixel coordinates of the center mark point among multiple mark points in the surgical tool image. The drill coordinates refer to the position of the drill tip or specific part on the surgical tool image, expressed in pixel values.

[0119] Optionally, as an embodiment of the present invention, the calculation of the center point coordinates and the drill bit coordinates of the surgical tool based on the multi-marker point feature can be calculated by marking point positioning.

[0120] S5. Identify the drilling position coordinates of the enhanced fusion image, calculate the position deviation of the surgical tool based on the drilling position coordinates and the drill bit coordinates, convert the drilling position coordinates and the center point coordinates into drilling space coordinates and center point space coordinates respectively based on the coordinate conversion algorithm, calculate the angle deviation of the surgical tool based on the drilling space coordinates and the center point space coordinates, construct a surgical locking navigation module for the target user based on the angle deviation, the position deviation and the surgical navigation framework, and determine the navigation path of the surgical tool corresponding to the center point based on the surgical locking navigation module.

[0121] The embodiment of the present invention can help avoid damage to surrounding healthy tissues or important structures by identifying the coordinates of the drilling position of the enhanced fusion image, thereby reducing surgical risks. The drilling position refers to the specific position of the target area that needs to be accurately drilled or other operations performed during surgery.

[0122] The embodiment of the present invention can significantly improve the accuracy of surgery by calculating the position deviation of the surgical tool based on the drilling position coordinates and the drill bit coordinates, which helps to avoid the surgical tool from accidentally damaging surrounding healthy tissues or important structures, thereby reducing surgical risks. The position deviation refers to the difference between the actual position of the surgical tool and its predetermined target position (such as the drilling position coordinates) in the surgical navigation system.

[0123] Optionally, as an embodiment of the present invention, the position deviation of the surgical tool calculated based on the drilling position coordinates and the drill bit coordinates can be determined by a Euclidean distance calculation method.

[0124] The embodiment of the present invention can convert the drilling position coordinates and the center point coordinates into drilling space coordinates and center point space coordinates respectively based on the coordinate conversion algorithm, and can convert the coordinates in real time, allowing the surgical navigation system to provide real-time guidance during the operation, which helps to adjust the position and direction of the surgical tool in real time. Among them, the drilling space coordinates refer to the precise coordinate points of the drilling position defined in three-dimensional space. The center point space coordinates refer to the precise coordinate position of the center point of the surgical tool in three-dimensional space.

[0125] As an embodiment of the present invention, the converting the drilling position coordinates and the center point coordinates into drilling space coordinates and center point space coordinates respectively based on the coordinate conversion algorithm includes:

[0126] Extracting the rotation matrix, translation vector, depth information and focal length parameters of the coordinate transformation algorithm;

[0127] Identify the tool image center coordinates of the tool image corresponding to the center point coordinates;

[0128] Determine the camera focal length parameters and vertical distance of the industrial camera corresponding to the tool image;

[0129] Identify the fused image center coordinates of the enhanced fused image corresponding to the drilling position coordinates;

[0130] Based on the focal length parameter, the rotation matrix, the translation vector, the camera focal length parameter, the tool image center coordinates and the fused image center coordinates, the drilling position coordinates and the center point coordinates are converted into drilling space coordinates and center point space coordinates respectively using the following formulas:

[0131]

[0132] Among them, α k The horizontal coordinate of the drilling hole represents the spatial coordinate of the drilling hole, β k The drilling depth coordinate representing the drilling space coordinate, γ kThe vertical coordinate of the drilling hole represents the spatial coordinate of the drilling hole, α m represents the horizontal coordinate of the center point in space, β m The central depth coordinate representing the spatial coordinate of the center point, γ m represents the vertical coordinate of the center point space coordinate, W represents the rotation matrix, Y represents the translation vector, D k Indicates depth information, F k Represents the focal length parameter, C kx The horizontal coordinate of the fused image representing the center coordinate of the fused image, c ky The fused image ordinate, G, represents the fused image center coordinate. k The horizontal coordinate of the drilling hole represents the coordinate of the drilling position, H k The vertical coordinate of the drilling hole, c mx The tool image abscissa representing the center coordinate of the tool image, c my The tool image ordinate, D, represents the center coordinate of the tool image m Indicates the vertical distance, F m Represents the focal length parameter of the camera, G m The horizontal coordinate of the center point, H represents the coordinate of the center point m The ordinate of the center point representing the coordinates of the center point.

[0133] Among them, the rotation matrix refers to a 3x3 matrix used to describe the rotation of the camera coordinate system relative to the world coordinate system. The translation vector refers to a three-dimensional vector that describes the translation from the camera coordinate system to the world coordinate system. The depth information refers to the distance from the camera to the observed object. The focal length parameter is the distance from the light entering the lens from infinity to the focused imaging plane. The tool image center coordinates refer to the coordinates of the exact center of the tool image. The camera focal length parameter refers to the focal length of the camera lens, which is an important parameter that describes the optical properties of the camera lens. The vertical distance refers to the actual distance in the vertical direction between the camera imaging plane and a specific point in the photographed scene. The fused image center coordinates refer to the coordinates of the exact center of the fused image.

[0134] The embodiment of the present invention can ensure that the surgical tool is operated according to a predetermined trajectory and angle by calculating the angular deviation of the surgical tool based on the drilling spatial coordinates and the center point spatial coordinates, thereby improving the accuracy of the surgery. The angular deviation refers to the difference between the actual direction of the surgical tool and the predetermined ideal direction in the surgical navigation system.

[0135] As an embodiment of the present invention, the calculating the angle deviation of the surgical tool based on the drilling space coordinates and the center point space coordinates includes:

[0136] Determining the actual direction vector of the surgical tool based on the spatial coordinates of the center point;

[0137] Determining an expected direction vector of the surgical tool based on the drilling space coordinates;

[0138] Calculating a pitch angle, a yaw angle, and a roll angle between the actual direction vector and the expected direction vector;

[0139] The angular deviation of the surgical tool is calculated according to the pitch angle, the yaw angle, and the roll angle.

[0140] Among them, the actual direction vector refers to the actual movement or pointing direction of the surgical tool in three-dimensional space. The expected direction vector refers to the expected movement direction of the surgical tool when performing a surgical task. The pitch angle refers to the angle of rotation of an object around its transverse axis (usually a horizontal axis perpendicular to the direction of the object's advance). The yaw angle refers to the angle of rotation of an object around its vertical axis (usually an axis pointing to the center of the earth). The roll angle refers to the angle of rotation of an object around its longitudinal axis (usually an axis parallel to the direction of the object's advance).

[0141] Optionally, the pitch angle, yaw angle and roll angle calculated between the actual direction vector and the expected direction vector may be calculated by a Rodriguez rotation formula.

[0142] The embodiment of the present invention can reduce the risk caused by the wrong position or direction of the tool during the operation and protect the user's important structures and organs from accidental damage by constructing the surgical locking navigation module of the target user according to the angle deviation, the position deviation and the surgical navigation framework. The surgical locking navigation module refers to an integrated system for providing real-time navigation information during the operation to ensure that the surgical tools are operated according to the predetermined path and target.

[0143] Optionally, as an embodiment of the present invention, the surgical locking navigation module for the target user constructed according to the angle deviation, the position deviation and the surgical navigation framework may be constructed by a system integration method.

[0144] The embodiment of the present invention determines the navigation path corresponding to the center point of the surgical tool based on the surgical locking navigation module, and can ensure that the surgical tool moves according to the predetermined trajectory through accurate navigation path planning, thereby reducing errors in surgery. The navigation path planning refers to a safe path for the surgical tool to reach the target position from the starting position.

[0145] The embodiment of the present invention can provide detailed surgical paths and real-time feedback by constructing a surgical navigation framework for the target user, thereby reducing unnecessary trauma and risks during surgery, thereby improving the accuracy of surgical operations; optionally, the embodiment of the present invention can achieve spatial alignment of the X-ray film and the camera image by using a preset image registration algorithm based on the X-ray film features and the camera image features to obtain a registered fused image, so that images of two different modalities can be observed and analyzed in the same coordinate system; the embodiment of the present invention can use a coordinate conversion algorithm to define the coordinate conversion algorithm of the surgical navigation framework based on the X-ray machine imaging principle and the camera imaging principle, so as to more accurately predict the path and position of surgical tools in the preoperative planning stage, thereby reducing surgery. uncertainty in; the embodiment of the present invention can accurately know the position of the surgical tool by calculating the center point coordinates and drill bit coordinates of the surgical tool based on the multi-marker point features, thereby achieving accurate positioning and navigation in the surgical navigation system and reducing errors during the operation; the embodiment of the present invention can convert the drilling position coordinates and the center point coordinates into drilling space coordinates and center point space coordinates respectively based on the coordinate conversion algorithm, and can convert coordinates in real time, allowing the surgical navigation system to provide real-time guidance during the operation, which helps to adjust the position and direction of the surgical tool in real time. Finally, the embodiment of the present invention determines the navigation path corresponding to the center point of the surgical tool based on the surgical locking navigation module. Through accurate navigation path planning, it can ensure that the surgical tool moves according to the predetermined trajectory and reduce errors in the operation. Therefore, the present invention can improve the navigation accuracy of the mixed reality navigation of the distal locking of the femoral interlocking intramedullary nail.

[0146] like Figure 2 , which is a functional module diagram of a mixed reality navigation system for distal locking of a femoral interlocking intramedullary nail provided by an embodiment of the present invention.

[0147] The femoral interlocking intramedullary nail distal locking mixed reality navigation system 200 of the present invention can be installed in an electronic device. According to the functions to be implemented, the femoral interlocking intramedullary nail distal locking mixed reality navigation system 200 may include an image acquisition module 201, an image fusion module 202, a coordinate conversion algorithm module 203, a navigation positioning module 204, and a navigation path determination module 205. The module of the present invention can also be referred to as a unit, which refers to a series of computer program segments that can be executed by an electronic device processor and can complete fixed functions, which are stored in the memory of the electronic device.

[0148] In this embodiment, the functions of each module / unit are as follows:

[0149] The image acquisition module 201 is used to construct a surgical navigation framework for the target user, configure a C-arm X-ray machine and an industrial camera for the surgical navigation framework, use the C-arm X-ray machine to collect femur X-rays of the target user in real time, and use the industrial camera to collect camera monitoring images of the target user in real time;

[0150] The image fusion module 202 is used to construct a reality enhancement module of the surgical navigation framework, based on which the X-ray film features and camera image features of the femur X-ray film and the camera monitoring screen are respectively extracted, based on the X-ray film features and the camera image features, the femur X-ray film and the camera monitoring screen are registered and fused using a preset image registration algorithm to obtain a registered fused image, and the registered fused image is enhanced using a preset image enhancement algorithm to obtain an enhanced fused image;

[0151] The coordinate conversion algorithm module 203 is used to analyze the X-ray machine imaging principle and the camera imaging principle of the C-arm X-ray machine and the industrial camera, and define the coordinate conversion algorithm of the surgical navigation framework based on the X-ray machine imaging principle and the camera imaging principle;

[0152] The navigation and positioning module 204 is used to construct the surgical tool of the surgical navigation framework, identify the multiple marking points of the surgical tool, use the industrial camera to collect the tool image of the surgical tool in real time, extract the multi-marking point features of the tool image, and calculate the center point coordinates and drill bit coordinates of the surgical tool based on the multi-marking point features;

[0153] The navigation path determination module 205 is used to identify the drilling position coordinates of the enhanced fusion image, calculate the position deviation of the surgical tool based on the drilling position coordinates and the drill bit coordinates, convert the drilling position coordinates and the center point coordinates into drilling space coordinates and center point space coordinates respectively based on the coordinate conversion algorithm, calculate the angle deviation of the surgical tool based on the drilling space coordinates and the center point space coordinates, construct the surgical locking navigation module of the target user according to the angle deviation, the position deviation and the surgical navigation framework, and determine the navigation path of the surgical tool corresponding to the center point based on the surgical locking navigation module.

[0154] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solution of the present invention can be modified or replaced by equivalents without departing from the spirit and scope of the technical solution of the present invention.

Claims

1. A mixed reality navigation method for distal locking of a femoral interlocking intramedullary nail, characterized in that: The method comprises: Constructing a surgical navigation framework for the target user, configuring a C-arm X-ray machine and an industrial camera for the surgical navigation framework, using the C-arm X-ray machine to collect femur X-rays of the target user in real time, and using the industrial camera to collect camera monitoring images of the target user in real time; Construct a reality enhancement module of the surgical navigation framework, and based on the reality enhancement module, respectively extract the X-ray film features and camera image features of the femur X-ray film and the camera monitoring screen, and based on the X-ray film features and the camera image features, use a preset image registration algorithm to register and fuse the femur X-ray film and the camera monitoring screen to obtain a registered fused image, and use a preset image enhancement algorithm to enhance the registered fused image to obtain an enhanced fused image.

2. The mixed reality navigation method for distal locking of a femoral interlocking intramedullary nail according to claim 1, characterized in that: Analyze the X-ray machine imaging principle and the camera imaging principle of the C-arm X-ray machine and the industrial camera, and define the coordinate transformation algorithm of the surgical navigation framework based on the X-ray machine imaging principle and the camera imaging principle; Construct a surgical tool of the surgical navigation framework, identify multiple marking points of the surgical tool, use the industrial camera to collect a tool image of the surgical tool in real time, extract the multi-marker point features of the tool image, and calculate the center point coordinates and drill bit coordinates of the surgical tool based on the multi-marker point features.

3. The mixed reality navigation method for distal locking of a femoral interlocking intramedullary nail as claimed in claim 2, characterized in that: Identify the drilling position coordinates of the enhanced fusion image, calculate the position deviation of the surgical tool based on the drilling position coordinates and the drill bit coordinates, convert the drilling position coordinates and the center point coordinates into drilling space coordinates and center point space coordinates respectively based on the coordinate conversion algorithm, calculate the angular deviation of the surgical tool based on the drilling space coordinates and the center point space coordinates, construct a surgical locking navigation module for the target user based on the angular deviation, the position deviation and the surgical navigation framework, and determine the navigation path of the surgical tool corresponding to the center point based on the surgical locking navigation module.

4. The mixed reality navigation method for distal locking of a femoral interlocking intramedullary nail as claimed in claim 3, characterized in that: The extracting of X-ray film features and camera image features of the femur X-ray film and the camera monitoring screen based on the augmented reality module respectively includes: De-noising the femur X-ray film and the camera monitoring picture by using a preset filter to obtain a de-noised X-ray film and a de-noised camera picture; Performing contrast enhancement on the denoised X-ray film to obtain an enhanced X-ray film, and performing color correction on the denoised camera image to obtain a corrected camera image; Perform contour recognition on the enhanced X-ray film to obtain a femoral contour, calculate geometric parameters of the femoral contour, and determine X-ray shape features of the femoral X-ray film based on the geometric parameters.

5. The mixed reality navigation method for distal locking of a femoral interlocking intramedullary nail according to claim 4, characterized in that: The extracting of X-ray film features and camera image features of the femur X-ray film and the camera monitoring screen respectively based on the augmented reality module includes: performing shape detection on the correction camera screen to obtain camera screen shape features; Using a preset local binary pattern, respectively extracting X-ray film texture features of the enhanced X-ray film and camera image texture features of the corrected camera image; Identify the X-ray film structural features of the enhanced X-ray film, and extract the X-ray film position features of the enhanced X-ray film based on the X-ray film structural features.

6. The mixed reality navigation method for distal locking of a femoral interlocking intramedullary nail according to claim 5, characterized in that: The extracting of X-ray film features and camera image features of the femur X-ray film and the camera monitoring screen respectively based on the augmented reality module includes: extracting screen key points of the correction camera screen, and determining camera screen spatial features of the correction camera screen based on the screen key points; Determine the X-ray film features of the femur X-ray film according to the X-ray film shape features, the X-ray film texture features, the X-ray film structure features and the X-ray film position features; The camera image features of the camera monitoring screen are determined according to the camera screen shape features, the camera screen texture features and the camera screen space features.

7. The mixed reality navigation method for distal locking of a femoral interlocking intramedullary nail as claimed in claim 3, characterized in that: The method of registering and fusing the femur X-ray film and the camera monitoring screen based on the X-ray film features and the camera image features using a preset image registration algorithm to obtain a registered fused image includes: Extracting X-ray film feature points for the X-ray film features and extracting camera image feature points for the camera image features; An X-ray film descriptor analyzing the X-ray film feature points and a camera image descriptor analyzing the camera image feature points; unifying the descriptor dimensions of the X-ray film descriptor and the camera image descriptor; Based on the descriptor dimension, an inter-descriptor distance between the X-ray film descriptor and the camera image descriptor is calculated.

8. The mixed reality navigation method for distal locking of a femoral interlocking intramedullary nail as claimed in claim 7, characterized in that: Based on the X-ray film features and the camera image features, the femur X-ray film and the camera monitoring screen are registered and fused using a preset image registration algorithm to obtain a registered fused image, and further comprising: According to the distance between the descriptors, matching the X-ray film features with the camera image features to obtain a feature matching relationship; Based on the feature matching relationship, construct an image transformation model of the femur X-ray and the camera monitoring screen; According to the image transformation model, the femur X-ray film and the camera monitoring screen are fused to obtain a registered fused image.

9. The mixed reality navigation method for distal locking of a femoral interlocking intramedullary nail according to claim 8, characterized in that: The step of constructing an image transformation model of the femur X-ray and the camera monitoring screen based on the feature matching relationship includes: Defining the X-ray feature point coordinates of the femur X-ray film, and mapping the image feature point coordinates of the camera monitoring screen based on the feature matching relationship; Constructing an initial affine transformation model of the femur X-ray film and the camera monitoring screen, and determining transformation parameters of the initial affine transformation model; Based on the transformation parameters, transform the femoral X-ray film using the initial affine transformation model to obtain a transformed femoral X-ray film; Based on the X-ray feature point coordinates, calculating the transformed X-ray feature point coordinates of the transformed femoral X-ray film; Determining the objective function and transformation parameter vector of the initial affine transformation model based on the transformation parameters, the X-ray feature point coordinates, and the image feature point coordinates; Calculating the transformation residual value of the X-ray feature point coordinates and the image feature point coordinates according to the objective function and the transformation parameter vector; When the transformation residual value is less than a preset transformation residual threshold, the initial affine transformation model is used as the image transformation model of the femur X-ray and the camera monitoring screen.

10. A mixed reality navigation system for distal locking of femoral interlocking intramedullary nail, characterized in that: A system for executing a mixed reality navigation method for distal locking of a femoral interlocking intramedullary nail as claimed in any one of claims 1 to 9, the system comprising: An image acquisition module is used to construct a surgical navigation framework for a target user, configure a C-arm X-ray machine and an industrial camera for the surgical navigation framework, use the C-arm X-ray machine to acquire a femur X-ray of the target user in real time, and use the industrial camera to acquire a camera monitoring screen of the target user in real time; An image fusion module is used to construct a reality enhancement module of the surgical navigation framework, based on which the X-ray film features and camera image features of the femur X-ray film and the camera monitoring screen are respectively extracted, based on the X-ray film features and the camera image features, the femur X-ray film and the camera monitoring screen are registered and fused using a preset image registration algorithm to obtain a registered fused image, and the registered fused image is enhanced using a preset image enhancement algorithm to obtain an enhanced fused image; A coordinate conversion algorithm module, used for analyzing the X-ray machine imaging principle and the camera imaging principle of the C-arm X-ray machine and the industrial camera, and defining the coordinate conversion algorithm of the surgical navigation framework based on the X-ray machine imaging principle and the camera imaging principle; A navigation and positioning module, used for constructing a surgical tool of the surgical navigation framework, identifying multiple marking points of the surgical tool, using the industrial camera to collect a tool image of the surgical tool in real time, extracting features of the multiple marking points of the tool image, and calculating the center point coordinates and drill bit coordinates of the surgical tool based on the features of the multiple marking points; A navigation path determination module is used to identify the drilling position coordinates of the enhanced fusion image, calculate the position deviation of the surgical tool based on the drilling position coordinates and the drill bit coordinates, convert the drilling position coordinates and the center point coordinates into drilling space coordinates and center point space coordinates respectively based on the coordinate conversion algorithm, calculate the angular deviation of the surgical tool based on the drilling space coordinates and the center point space coordinates, construct a surgical locking navigation module for the target user based on the angular deviation, the position deviation and the surgical navigation framework, and determine the navigation path of the surgical tool corresponding to the center point based on the surgical locking navigation module.

Images