Maxillofacial surgery planning navigation method and system

Abstract

The invention provides a maxillofacial operation planning navigation method and system, relates to the technical field of operation planning navigation, and solves the technical problems that the existing maxillofacial operation planning navigation method cannot adapt to the change of an intraoperative anatomical structure due to a preoperative planning static state, and the intraoperative navigation efficiency and precision are insufficient. The method specifically comprises the following steps: acquiring first image data of a to-be-operated region; preprocessing the first image data based on an image processing algorithm to determine second image data; performing three-dimensional reconstruction and segmentation on the second image data to obtain a three-dimensional data model; analyzing the three-dimensional data model through an AI intelligent auxiliary diagnosis model, and generating a diagnosis report; inputting the three-dimensional data model and the diagnosis report into a two-channel model, and outputting an operation path scheme and a bone fracture plate scheme; in the operation, non-rigid registration is adopted to compensate real-time deformation delay for first image data before the operation and in the operation, and an AR real-time navigation technology is combined to realize maxillofacial operation planning navigation. The method is used for maxillofacial surgery planning navigation.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of surgical planning navigation, in particular to a maxillofacial surgery planning navigation method and system. BACKGROUND

[0002] Maxillofacial surgery is a high-difficulty and high-risk discipline recognized in the medical field, and its operation design involves complex anatomical structures and precise functional preservation requirements. Although the application of existing digital technologies such as preoperative planning and intraoperative navigation in maxillofacial surgery has significantly improved the predictability and accuracy of surgery, there are still defects. Preoperative planning can reconstruct a three-dimensional model based on X-ray / CT data and perform virtual osteotomy, bone grafting, and implant design operations, but the planned data is presented in a static manner and cannot adapt to changes in intraoperative anatomical structures in real time, which increases the risk of surgery. Therefore, the existing maxillofacial surgery planning navigation method has the technical problems of static preoperative planning that cannot adapt to changes in intraoperative anatomical structures, and insufficient efficiency and accuracy of intraoperative navigation. SUMMARY

[0003] The present application provides a maxillofacial surgery planning navigation method and system, which solves the technical problems of static preoperative planning that cannot adapt to changes in intraoperative anatomical structures, and insufficient efficiency and accuracy of intraoperative navigation in the existing maxillofacial surgery planning navigation method.

[0004] To achieve the above-mentioned purpose, the technical scheme adopted by the present application is as follows: In a first aspect, a maxillofacial surgery planning navigation method is provided, comprising: acquiring first image data of a region to be operated on, the first image data being X-ray projection sequence data of the maxillofacial region; pre-processing the first image data based on an image processing algorithm to determine second image data, the image processing algorithm including a 2D Transformer denoising model, CLAHE, and an improved Laplace pyramid algorithm; performing three-dimensional reconstruction and segmentation on the second image data to obtain a three-dimensional data model; analyzing the three-dimensional data model in combination with an AI intelligent auxiliary diagnosis model and a physician's artificial diagnosis to generate a diagnosis report; inputting the three-dimensional data model and the diagnosis report into a dual-channel model to output a surgical path plan and a bone plate plan; the dual-channel model includes a personalized path planning branch and an intelligent bone plate recommendation branch, the personalized path planning branch is used to generate a surgical path plan by constructing a three-dimensional coordinate system and using a gradually optimal fast expanding random tree path planning algorithm, and the intelligent bone plate recommendation branch is used to recommend a bone plate plan based on a knowledge graph recommendation model according to the three-dimensional data model and the diagnosis report, the bone plate plan including the most suitable bone plate and the corresponding combination mode, and the knowledge graph recommendation model being a model trained by a large amount of maxillofacial bone fracture bone plate selection report data; during the operation, non-rigid registration is used to compensate for real-time deformation delay of the preoperative and intraoperative first image data, and AR real-time navigation technology is used to realize maxillofacial surgery planning navigation.

[0005] In a possible implementation manner of the first aspect, the second image data is determined by pre-processing the first image data based on an image processing algorithm, including: performing denoising processing on the first image data based on a 2D Transformer denoising model, the 2D Transformer denoising model being a model trained based on a large amount of X-ray data; enhancing the local region contrast of the denoised image data by using CLAHE; and compensating the outline, edge and line of the image data after the contrast enhancement by using an improved Laplace pyramid algorithm to obtain the second image data.

[0006] In a possible implementation manner of the first aspect, the guide filter pyramid is constructed according to the following formula:

[0007] wherein, I is an original image, that is, the image data after the contrast enhancement, k is an index of a layer of the guide filter pyramid, is an initial layer of the guide filter pyramid, is a down-sampling operation, is a guide filter pyramid image of the kth layer, is a guide filter pyramid image of the k+1th layer, is a guide filter process, is a guide image of the kth layer guide filter pyramid image, is a filter kernel radius, is a regularization parameter. In a possible implementation manner of the first aspect, the Laplace pyramid is constructed according to the following formula:

[0008] wherein, I is an original image, that is, the image data after the contrast enhancement, k is an index of a layer of the guide filter pyramid,

[0009] is a Laplace layer of the kth layer in the Laplace pyramid, is a total number of layers of the Laplace pyramid, is an expansion operation on the k+1th layer guide filter pyramid image, is an up-sampling operation, is a top layer image of the guide filter pyramid.

[0010] ​​​In conjunction with the first aspect mentioned above, one possible implementation uses an asymptotically optimal fast expanding random tree path planning algorithm to generate surgical path schemes, including: S1. The initial exploration tree satisfies the following formula:

[0011] in, To explore the tree, To explore the node set of the tree, To explore the edge set of the tree (i.e., the search path), This is the initial position of the surgical instruments. The path cost is the starting point.

[0012] S2. Find the nearest node that satisfies the following formula:

[0013] in, For random sampling points, To explore tree node sets Zhongyu The nearest node, q is any node in the set of nodes in the exploration tree, and argmin(·) is the variable that makes the function take the minimum value.

[0014] S3. The generation of new nodes satisfies the following formula:

[0015] in, For the newly generated node, This is the step size parameter.

[0016] S4. Calculate the new node Path cost In the node set Search in Domain nodes, by adjusting edge sets Reconnect parent nodes to optimize the exploration tree The path cost; when the target node is included in the node set V or the preset number of iterations is reached, the algorithm terminates and backtracks from the edge set E of the exploration tree T. The path to the target node is used to generate a surgical path plan.

[0017] In a possible implementation manner of the first aspect, the method further includes: constructing a diagnosis-plate interaction matrix, the interaction matrix being a matrix representing historical selection interaction between a fracture type and a plate, wherein a matrix element represents selection association of a specific fracture type and a specific plate; extracting a fracture type entity, a plate entity, and an association relationship between the fracture type entity and the plate entity from the knowledge graph according to bone block data in the three-dimensional data model and fracture type information in the diagnosis report; learning feature representation and a scoring function of the fracture type and the plate through the knowledge graph recommendation model, and predicting the association relationship in the knowledge graph and the interaction matrix through the knowledge graph recommendation model to determine an interaction probability of the current fracture type and each plate; and recommending a most suitable plate and a corresponding combination mode from high to low according to the interaction probability to form a plate scheme.

[0018] In a possible implementation manner of the first aspect, the scoring function satisfies the following formula:

[0019] wherein u is a fracture type entity, i is a plate entity, and r is an association relationship between u and i; is an embedding vector of the fracture type entity u, is an embedding vector of the plate entity i, is an embedding vector of the relationship r; represents a vector inner product operation, is an interaction probability score.

[0020] In a possible implementation manner of the first aspect, the non-rigid registration is used to compensate for real-time deformation delay of the preoperative and intraoperative first image data, including: performing pixel-level correspondence analysis on the preoperative and intraoperative first image data to construct a deformation field representing a non-linear deformation relationship between the preoperative and intraoperative first image data; converting the deformation field into a bias parameter through a Kalman filtering algorithm, the bias parameter including a displacement vector and a rotation angle of a local region; and adjusting the deformation field based on the bias parameter to compensate for the real-time deformation delay.

[0021] In a possible implementation manner of the first aspect, the construction of the deformation field representing the non-linear deformation relationship between the preoperative and intraoperative first image data satisfies the following formula:

[0022] wherein is a deformation field, and wherein x represents each point in an image, represents a displacement vector of each point, is the preoperative first image data, the first image data in surgery, the intraoperative image after deformation field transformation, (·) is a measure with a similarity measure function, is a bending energy penalty function, is a regularization weight.

[0023] In a second aspect, a maxillofacial surgery planning and navigation system is provided, comprising: an image acquisition module, an image preprocessing module, an image reconstruction and segmentation module, an AI intelligent auxiliary diagnosis module, a personalized path planning module, an intelligent bone plate recommendation module, an offset analysis correction module, and an AR navigation module; wherein the image acquisition module is configured to acquire first image data of a region to be operated; the image preprocessing module is configured to preprocess the first image data based on an image processing algorithm to determine second image data; the image reconstruction and segmentation module is configured to perform three-dimensional reconstruction and segmentation on the second image data to obtain a three-dimensional data model; the AI intelligent auxiliary diagnosis module is configured to analyze the three-dimensional data model in combination with an AI intelligent auxiliary diagnosis model and a physician's artificial diagnosis to generate a diagnosis report; the personalized path planning module is configured to input the three-dimensional data model and the diagnosis report into a double-channel model to output a surgical path scheme; the intelligent bone plate recommendation module is configured to input the three-dimensional data model and the diagnosis report into a double-channel model to output a bone plate scheme; the offset analysis correction module is configured to compensate for real-time deformation delay of the preoperative and intraoperative first image data using non-rigid registration during surgery; and the AR navigation module is configured to realize maxillofacial surgery planning and navigation in combination with AR real-time navigation technology.

[0024] In a third aspect, the present application provides a jaw and face surgery planning and navigation device, comprising: a communication unit and a processing unit; the communication unit is configured to obtain first image data of a region to be operated, the first image data being X-ray projection sequence data of a jaw and face region; the processing unit is configured to preprocess the first image data to determine second image data based on an image processing algorithm, the image processing algorithm comprising a 2D Transformer denoising model, CLAHE, and an improved Laplace pyramid algorithm; perform three-dimensional reconstruction and segmentation on the second image data to obtain a three-dimensional data model; analyze the three-dimensional data model in combination with an AI intelligent auxiliary diagnosis model and a physician artificial diagnosis to generate a diagnosis report; input the three-dimensional data model and the diagnosis report into a double-channel model to output a surgery path scheme and a bone plate scheme; the double-channel model comprises a personalized path planning branch and an intelligent bone plate recommendation branch, the personalized path planning branch is configured to generate a surgery path scheme by constructing a three-dimensional coordinate system and using a progressive optimal rapid expansion random tree path planning algorithm, and the intelligent bone plate recommendation branch is configured to recommend a bone plate scheme based on a knowledge graph recommendation model according to the three-dimensional data model and the diagnosis report, the bone plate scheme comprising a recommended most suitable bone plate and a corresponding combination mode, and the knowledge graph recommendation model being a model trained by a large amount of jaw and face bone fracture bone plate selection report data; in the operation, the preoperative and intraoperative first image data is compensated for real-time deformation delay using non-rigid registration, and jaw and face surgery planning and navigation is realized in combination with AR real-time navigation technology.

[0025] In a fourth aspect, the present application provides a jaw and face surgery planning and navigation device, comprising: a processor and a storage medium; the storage medium comprises instructions, and the processor is configured to run the instructions to implement the method described in the first aspect and any possible implementation manner of the first aspect. The jaw and face surgery planning and navigation device can be an electronic device or a chip in an electronic device.

[0026] In a fifth aspect, the present application provides a computer readable storage medium, which stores instructions, and when the instructions run on a jaw and face surgery planning and navigation device, the jaw and face surgery planning and navigation device executes the method described in the first aspect and any possible implementation manner of the first aspect.

[0027] In a sixth aspect, the present application provides a computer program product comprising instructions, and when the computer program product runs on a jaw and face surgery planning and navigation device, the jaw and face surgery planning and navigation device executes the method described in the first aspect and any possible implementation manner of the first aspect.

[0028] The application provides a maxillofacial surgery planning navigation method and system, which obtains a maxillofacial X-ray projection sequence data, pre-processes by using a 2D Transformer denoising model, CLAHE and an improved Laplace pyramid algorithm, effectively improves image quality to retain key anatomical features, obtains accurate three-dimensional data models and bone block data through three-dimensional reconstruction and segmentation, generates a reliable diagnosis report in combination with an AI intelligent auxiliary diagnosis model and manual diagnosis of a doctor, provides accurate basis for subsequent planning, generates a suitable surgical path through a personalized path planning branch in a double-channel model, and a smart bone plate recommendation branch recommends an optimal bone plate and a combination mode, so that personalized and accurate planning schemes are realized; in-surgery non-rigid registration compensation is adopted to compensate for real-time deformation delay, and AR technology is combined to realize efficient navigation, so that preoperative planning dynamically adapts to anatomical structure changes in surgery, and navigation efficiency and accuracy are improved, and therefore, the technical problems of preoperative planning staticity that cannot adapt to anatomical structure changes in surgery and insufficient in-surgery navigation efficiency and accuracy of the existing maxillofacial surgery planning navigation method are solved.

[0029] It should be understood that the description of technical features, technical solutions, advantages or similar language in the present application does not imply that all features and advantages can be realized in any single embodiment. On the contrary, it can be understood that the description of a feature or advantage means that the specific technical feature, technical solution or advantage is included in at least one embodiment. Therefore, the description of technical features, technical solutions or advantages in the specification does not necessarily refer to the same embodiment. Further, the technical features, technical solutions and advantages described in the embodiments can be combined in any appropriate manner. Those skilled in the art will understand that the embodiments can be implemented without one or more specific technical features, technical solutions or advantages of a specific embodiment. In other embodiments, additional technical features and advantages can be identified in specific embodiments that do not embody all embodiments. BRIEF DESCRIPTION OF DRAWINGS

[0030] Figure 1 A system architecture diagram of a maxillofacial surgery planning navigation system provided by an embodiment of the present application; Figure 2 A flowchart of a maxillofacial surgery planning navigation method provided by an embodiment of the present application; Figure 3 A flowchart of another maxillofacial surgery planning navigation method provided by an embodiment of the present application; Figure 4 A flowchart of another maxillofacial surgery planning navigation method provided by an embodiment of the present application; Figure 5 A flowchart of another maxillofacial surgery planning navigation method provided by an embodiment of the present application; Figure 6A structural schematic diagram of a jaw and face surgery planning navigation device provided by an embodiment of the present application is shown in the figure. Figure 7 A hardware structural schematic diagram of a jaw and face surgery planning navigation device provided by an embodiment of the present application is shown in the figure. DETAILED DESCRIPTION

[0031] In the description of the present application, unless otherwise specified, " / " means "or", for example, A / B can mean A or B. "And / or" in this document is only a description of the relationship between the associated objects, which means that there can be three relationships, for example, A and / or B, which can mean that A exists alone, A and B exist together, and B exists alone. In addition, "at least one" means one or more, and "multiple" means two or more. "First", "second", and the like do not limit the quantity and execution order, and "first", "second", and the like do not necessarily mean different.

[0032] It should be noted that in the present application, "exemplary" or "for example" means to serve as an example, illustration or description. Any embodiment or design scheme described as "exemplary" or "for example" in the present application should not be interpreted as more preferred or more advantageous than other embodiments or design schemes. Rather, the use of "exemplary" or "for example" is intended to present the relevant concept in a specific manner.

[0033] The jaw and face surgery planning navigation method provided by the embodiment of the present application can be applied to a jaw and face surgery planning navigation system as shown in the figure. Figure 1 The system includes an image acquisition module 101, an image preprocessing module 102, an image reconstruction and segmentation module 103, an AI intelligent auxiliary diagnosis module 104, a personalized path planning module 105, an intelligent bone plate recommendation module 106, an offset analysis correction module 107, an AR navigation module 108, and a display interaction module 109.

[0034] The image acquisition module 101 is used to acquire first image data of a region to be operated, the image preprocessing module 102 is used to preprocess the first image data based on an image processing algorithm to determine second image data, the image reconstruction and segmentation module 103 is used to perform three-dimensional reconstruction and segmentation on the second image data to obtain a three-dimensional data model, the AI intelligent auxiliary diagnosis module 104 is used to analyze the three-dimensional data model in combination with an AI intelligent auxiliary diagnosis model and a doctor's artificial diagnosis to generate a diagnosis report, the personalized path planning module 105 is used to input the three-dimensional data model and the diagnosis report into a double-channel model to output a surgical path scheme, the intelligent bone plate recommendation module 106 is used to input the three-dimensional data model and the diagnosis report into the double-channel model to output a bone plate scheme, the offset analysis and correction module 107 is used to compensate for real-time deformation delay of the preoperative and intraoperative first image data by using non-rigid registration during the operation, the AR navigation module 108 is used to realize jaw and face surgery planning navigation in combination with AR real-time navigation technology, and the display interaction module 109 is used to convert the three-dimensional data model, the diagnosis report, the surgical path scheme and the bone plate scheme into instructions and transmit the instructions to the AR navigation module, so that the doctor can visually preview and interactively adjust the preoperative planning data (such as the surgical path and the bone plate position), and the anatomical structure matching result after the intraoperative offset analysis and correction and the AR navigation guide information are presented in real time, so that the information interaction and operation feedback between the doctor and the system are realized, and intuitive visual support is provided for the operation process.

[0035] To solve the technical problems of the prior art that the preoperative planning is static and cannot adapt to the changes of the intraoperative anatomical structure, and the intraoperative navigation efficiency and accuracy are insufficient, the embodiment of the present application provides a jaw and face surgery planning navigation method, which can obtain jaw and face X-ray projection sequence data, and improve the image quality through preprocessing such as denoising, contrast enhancement and edge compensation, perform three-dimensional reconstruction and segmentation based on the preprocessed image to obtain accurate three-dimensional data model and bone block data, generate a reliable report in combination with AI intelligent auxiliary diagnosis and doctor's diagnosis, and then generate personalized surgical path and bone plate scheme through a double-channel model. Meanwhile, non-rigid registration is used to compensate for real-time deformation delay during the operation, and dynamic navigation is realized in combination with AR technology. Based on this, the dynamic adaptation of the preoperative planning to the changes of the intraoperative anatomical structure and the improvement of the navigation efficiency and accuracy are realized. The technical problems of the prior art that the preoperative planning is static and cannot adapt to the changes of the intraoperative anatomical structure, and the intraoperative navigation efficiency and accuracy are insufficient are solved.

[0036] Figure 2 A flowchart of the jaw and face surgery planning navigation method provided by the embodiment of the present application is shown in FIG. 1. Figure 2 As shown in FIG. 1, the method comprises the following steps. Step 201: A jaw and face surgery planning navigation system acquires first image data of a region to be operated.

[0037] The first image data is a sequence of X-ray projection data of the maxillofacial region, and contains original projection information of anatomical structures such as maxillofacial bones and soft tissues.

[0038] In the embodiment of the present application, the image acquisition module adjusts the patient's position according to the lesion site, sets the exposure parameters of the X-ray equipment, and rotates the exposure around the maxillofacial region by using a mobile O-shaped arm X-ray machine to obtain a sequence of X-ray projections at consecutive angles as the original data for subsequent processing.

[0039] As an example, a Geelin880 mobile O-shaped arm X-ray machine can be used to calibrate the O-shaped arm center axis to coincide with the operating table coordinate system by laser positioning, and collect a sequence of X-ray projections with a resolution of 1280x1024 to provide high-quality original data for subsequent processing.

[0040] Based on the above steps, the original image basis is provided for the entire surgical planning and navigation process, and the acquisition quality directly affects the accuracy of subsequent preprocessing, reconstruction and planning. Through a standardized acquisition process, the error sources in subsequent processing can be reduced.

[0041] Step 202, the maxillofacial surgery planning and navigation system determines second image data by preprocessing the first image data based on an image processing algorithm.

[0042] The image processing algorithm includes a 2DTransformer-based denoising model, CLAHE (Contrast Limited Adaptive Histogram Equalization), and an improved Laplace pyramid algorithm; and the second image data is optimized image data after denoising, contrast enhancement and edge compensation.

[0043] In the embodiment of the present application, the image preprocessing module uses the 2DTransformer denoising model to filter quantum noise, salt and pepper noise and other interference in the first image data; secondly, it uses CLAHE to enhance the contrast of local regions, while avoiding excessive noise enhancement, highlighting key structures such as bone edges and sutures; finally, the image contour and edge are compensated by using the improved Laplace pyramid algorithm, and the traditional Gaussian filter is replaced by guided filtering to preserve edge features and enrich detail information.

[0044] As an example, the lower jaw bone edge in the original X-ray image is blurred and there is speckle noise. After preprocessing, the noise is significantly reduced, the contrast between the lower jaw bone and the surrounding tissue is improved, and the bone suture details are clear and identifiable, providing high-quality input for three-dimensional reconstruction.

[0045] Based on the above steps, the quality of the image data is effectively improved, the noise interference is reduced, and the recognizability of key anatomical structures is enhanced.

[0046] Step 203, the maxillofacial surgery planning and navigation system performs three-dimensional reconstruction and segmentation on the second image data to obtain a three-dimensional data model.

[0047] The three-dimensional data model is a three-dimensional anatomical structure model of the maxillofacial region, including the spatial positional relationship of tissues such as bones and teeth. The segmented bone block data refers to independent bone blocks classified according to anatomy (such as the maxilla, mandible, zygomatic bone, etc.).

[0048] In the embodiments of the present application, the image reconstruction and segmentation module performs three-dimensional reconstruction on the second image data based on the FDK three-dimensional reconstruction algorithm of the CUDA parallel architecture, and performs volume rendering rendering in combination with the VTK open source library to generate a three-dimensional model that can be observed from multiple angles. The nnUNet neural network trained by a large amount of labeled data is used to automatically segment the three-dimensional model, separate each bone block according to the interosseous suture, and repair the segmentation error at the junction of the teeth and the jaw bone through the graph cut algorithm to ensure that the segmentation result conforms to the anatomical features, thereby obtaining a three-dimensional data model containing bone block data.

[0049] It should be noted that the real-time performance of reconstruction and segmentation is a key indicator, and the system needs to complete the processing in a very short time (three-dimensional reconstruction ≤ 60s, segmentation ≤ 30s) to meet the time efficiency requirements of clinical surgery. At the same time, the segmentation result needs to be verified by anatomy to ensure the accuracy of the bone block boundary.

[0050] As an example, the reconstructed three-dimensional model can clearly display the fine structure of the orbital rim and zygomatic arch, and after segmentation, the mandible can be individually selected and rotated and scaled to observe its joint relationship with the temporal bone, providing intuitive reference.

[0051] Based on the above steps, the two-dimensional image is converted into a three-dimensional visual model, which realizes the three-dimensional presentation and accurate segmentation of the structure, and provides structured anatomical data support for subsequent intelligent auxiliary diagnosis, path planning, etc.

[0052] Step 204, the maxillofacial surgery planning and navigation system analyzes the three-dimensional data model in combination with an AI intelligent auxiliary diagnosis model and a physician's manual diagnosis to generate a diagnosis report.

[0053] The AI intelligent auxiliary diagnosis model is a deep learning model based on a 3DTransformer architecture, and the training data includes three-dimensional models and labeled information of normal and pathological changes. The diagnosis report includes key information such as the location, type, and degree of the lesion.

[0054] In the embodiments of the present application, the AI intelligent auxiliary diagnosis module first extracts features from the three-dimensional data model, identifies the lesion area, and outputs a preliminary diagnosis result. Secondly, the physician reviews and corrects the AI result in combination with clinical experience, supplements information such as medical history and physical signs, and finally forms a complete diagnosis report.

[0055] It is worth noting that the combination of AI and manual diagnosis is the core: AI objectively identifies by learning a large number of cases, reducing individual subjective differences; doctors deal with complex or rare cases through clinical experience, and the combination of the two improves the accuracy and reliability of diagnosis.

[0056] As an example, the AI model identifies "right zygomatic fracture" and labels the fracture line position, and the doctor confirms the fracture type as "comminuted fracture" by combining the patient's trauma history and palpation results, and supplements the soft tissue swelling in the report, providing clear basis for subsequent surgical planning.

[0057] Based on the above steps, the efficiency and accuracy of diagnosis are improved through human-machine cooperation, avoiding the limitations of single reliance on artificial or AI, and providing a reliable basis for individualized surgical planning.

[0058] Step 205, the maxillofacial surgery planning and navigation system inputs the three-dimensional data model and the diagnosis report into the dual-channel model, and outputs the surgical path plan and the bone plate plan.

[0059] Among them, the dual-channel model includes a personalized path planning branch and an intelligent bone plate recommendation branch; the surgical path plan is the optimal operation route to avoid key structures such as nerves and blood vessels, and the bone plate plan includes the recommended most suitable bone plate and its combination mode.

[0060] In the embodiments of the present application, the personalized path planning module combines the anatomical structure in the three-dimensional data model and the lesion information in the diagnosis report to construct a three-dimensional state space, generates an exploration tree through random sampling, and optimizes to obtain a surgical path plan that avoids dangerous areas; based on the knowledge graph recommendation model, the patient's fracture site and shape in the three-dimensional data model and the diagnosis report are combined to recommend suitable bone plates and combination methods, supporting 3D printing customization.

[0061] Optionally, the gradual optimal rapid expansion random tree path planning algorithm is used to generate the surgical path plan, including: S1, initialize the exploration tree to satisfy the following formula:

[0062] Among them, is the exploration tree, is the node set of the exploration tree, is the edge set (i.e. search path) of the exploration tree, is the initial position of the surgical instrument, is the path cost of the starting point.

[0063] S2, find the nearest node to satisfy the following formula:

[0064] wherein, is a random sampling point, is a set of exploration tree nodes is the nearest node to , q is any node in the set of exploration tree nodes, and argmin(·) is a variable that minimizes a function.

[0065] S3, generating a new node satisfies the following formula:

[0066] wherein, is a generated new node, is a step parameter.

[0067] S4, calculating the path cost of the new node , finding the field node of in the node set , reconnecting the parent node by adjusting the edge set to optimize the path cost of the exploration tree ; when the target node is included in the node set V or the preset iteration number is reached, the algorithm is terminated, the path from to the target node is backtracked based on the edge set E of the exploration tree T, and a surgical path scheme is generated.

[0068] It should be noted that when generating a surgical path scheme using the rapidly expanding random tree path planning algorithm with progressive optimality, the following constraints and optimization objectives need to be focused on: first, the instrument operation boundary needs to be defined in combination with the maxillofacial anatomical structure, such as the instrument swing amplitude needs to match the surgical area space (to avoid touching sensitive areas such as the eyeball and skull base), and the minimum safety distance between the instrument and the nerve (such as the inferior alveolar nerve and the facial nerve branch) needs to be preset (usually ≥3mm) to ensure that the path avoids dangerous structures; second, the path cost calculation needs to use a composite function to comprehensively weigh the straight-line distance from the node to the target lesion, the safety redundancy with important anatomical structures, and the path smoothness (to reduce unnecessary turns of the instrument), to avoid insufficient practicality of the path due to a single objective; third, the algorithm termination condition needs to be reasonably set, if the target node is included in the exploration tree, the iteration is terminated, if it is not included, the maximum iteration number (such as 500 times) is set, to balance the path planning accuracy and real-time performance (to ensure that the calculation is completed within 30s), to avoid affecting the surgical process due to too long iteration.

[0069] As an example, taking the "mandibular body fracture repair surgery" as an example, first, a three-dimensional state space containing the mandible, the inferior alveolar nerve, and the masticatory muscle group is constructed, and the initial position of the surgical instrument is set as ​The entry point of the oral vestibule is set at the three-dimensional coordinates: x=25mm, y=18mm, z=42mm. The target node of the lesion is set at the midpoint of the fracture line (x=48mm, y=22mm, z=45mm). The instrument swing amplitude is set to ≤15°, the minimum safe distance from the inferior alveolar nerve is ≥5mm, and the step size parameter η=2mm. The algorithm initializes the exploration tree. ,in , Starting point path cost ; then random sampling points (e.g., x=38mm, y=20mm, z=43mm), find the nearest node in the tree using three-dimensional Euclidean distance calculation. (distance Recent ),along arrive Direction generates new nodes (x=32mm, y=19mm, z=42.5mm), check Once the distance to the inferior alveolar nerve is 6.2 mm (meeting safety requirements), it is included in V; repeated sampling and expansion are performed (e.g., in the 120th iteration, the sampling point is close to the fracture line), the path cost of the new node's neighborhood (within a radius of 5 mm) is calculated, and the parent node is reconnected to optimize the path (adjusting the original path around the outer side of the mandible to a safe path through the inner side); when the target node (midpoint of the fracture line) is included in V in the 186th iteration, the algorithm terminates and backtracks to explore the tree edge set E, generating a sequence from... The path to the fracture line (avoiding the inferior alveolar nerve throughout, with a turning angle ≤10°) is superimposed on the 3D model for the physician to confirm.

[0070] Based on the above steps, the generated personalized surgical plan and bone plate plan can accurately adapt to the patient's anatomical structure and lesion characteristics, reduce damage to normal tissues, and provide planning assurance for the safety and effectiveness of the surgery.

[0071] Step 206: During the operation, the maxillofacial surgery planning and navigation system uses non-rigid registration to compensate for real-time deformation delays in the first image data before and during the operation, and combines AR real-time navigation technology to realize maxillofacial surgery planning and navigation.

[0072] Among them, non-rigid registration is a technique that achieves pixel-level nonlinear correspondence between preoperative and intraoperative images by constructing a deformation field; AR real-time navigation is a visualization technique that overlays virtual planning information with the real surgical field using AR devices.

[0073] In the embodiment of the present application, the offset analysis correction module uses non-rigid registration to perform pixel-level analysis on the preoperative and intraoperative first image data, constructs a deformation field, optimizes the severe deformation through a bending energy penalty function, converts the deformation field into a deviation parameter, and adjusts the preoperative planning to adapt to the intraoperative anatomical structure; the AR navigation module superimposes the corrected planning information and the surgical field in real time, tracks the position of the surgical instrument through the sensor, and alarms in the form of light flickering when the instrument deviates from the path or exceeds the safety threshold, guiding the physician to operate.

[0074] It should be noted that real-time and spatial alignment are core challenges: non-rigid registration needs to respond quickly to intraoperative dynamic changes (such as processing time ≤50ms per frame), and AR navigation needs to ensure the spatial consistency of virtual information and real surgical field through infrared positioning and surgical area markers (error ≤1mm).

[0075] As an example, a slight shift of the patient's head during the operation causes the preoperative planning to be misaligned with the surgical field, and the non-rigid registration calculates the offset through the deformation field and adjusts the path; the AR head-mounted display superimposes the adjusted path on the surgical field, and the physician operates according to the superimposed path, and when the instrument deviates from the path, the path flashes in red to prompt the adjustment direction.

[0076] Based on the above steps, the dynamic adaptation of preoperative planning and intraoperative anatomical structure changes is realized, and the AR visualization guidance reduces the physician's line-of-sight switching when observing the display to obtain intraoperative information, improves the accuracy and efficiency of the operation, and reduces the risk caused by anatomical structure changes or operation deviation.

[0077] Based on the above technical solution, the data foundation is laid through high-quality image acquisition, the image accuracy is improved through preprocessing, the anatomical structure is presented in three dimensions through three-dimensional reconstruction and segmentation, the accuracy of lesion analysis is ensured through AI and manual collaborative diagnosis, the personalized surgical path and bone plate scheme are generated through the double-channel model, the anatomical changes are dynamically adapted and the operation guidance is optimized through intraoperative non-rigid registration and AR navigation, the accuracy of surgical planning and the safety of operation are improved from data acquisition to intraoperative navigation, effectively solving the problems of static preoperative planning and inefficient intraoperative navigation.

[0078] In a possible implementation manner, in combination with the above Figure 2 As Figure 3 shown, the process of pre-processing the first image data to determine the second image data based on the image processing algorithm in step 202 can be implemented through steps 301-303 as follows: Step 301, the maxillofacial surgery planning and navigation system performs denoising processing on the first image data based on a 2D Transformer denoising model.

[0079] The 2DTransformer denoising model is a two-dimensional image denoising model based on a Transformer architecture, which captures the global dependency between image pixels through an attention mechanism, and can accurately filter noise while preserving anatomical structure details after being trained on a large number of maxillofacial X-ray image data.

[0080] In the embodiments of the present application, the training data set of the 2DTransformer denoising model covers 5000+ clinical maxillofacial X-ray projection sequences, covering images of patients of different ages, genders and various types of lesions, ensuring the generalization and adaptability of the model to various types of noise and anatomical structures. During the training process, the MSE (Mean Squared Error) loss function is used to measure the difference between the model output and the clean image, and the Adam optimizer is used to iteratively adjust the model parameters until the denoising accuracy of the validation set meets the standard. When processing, the system inputs the collected first image data into the pre-trained 2DTransformer denoising model in batches, and the model captures the spatial correlation between pixels through the self-attention layer of the encoder, identifies and separates noise features and anatomical structure features, and then outputs the denoised image frame through the decoder.

[0081] It should be noted that the advantage of the 2DTransformer denoising model is that it can more accurately distinguish noise and anatomical details compared to traditional Gaussian filtering and median filtering. Traditional filtering can blur the bone edges, while the model can selectively filter random noise and retain more than 95% of the edge information of complex structures such as the mandible and zygomatic bone. In addition, the specific principle of model denoising is not limited in the present application.

[0082] As an example, using a Geelin880 mobile O-shaped arm X-ray machine, an X-ray projection sequence (total of 360 frames, resolution 1280x1024) of a patient with a jaw fracture was collected under the parameters of tube voltage 80kV and tube current 10mA. There are obvious salt and pepper noise (spot-like interference) and quantum noise (uneven light and dark) in the original sequence, resulting in blurred fracture lines. After inputting the sequence into the 2DTransformer denoising model, the noise spots in the output denoised sequence disappear, the light and dark contrast tends to be uniform, and the continuity and clarity of the fracture lines are significantly improved.

[0083] Based on the above steps, various noise interference in the first image data is effectively filtered, laying a foundation for obtaining high-quality anatomical structure information, while avoiding subsequent segmentation errors caused by noise.

[0084] Step 302, the maxillofacial surgery planning and navigation system enhances the local area contrast of the denoised image data using CLAHE.

[0085] CLAHE is an improved local contrast enhancement algorithm. By dividing the image into multiple sub-blocks, calculating the histogram of each sub-block independently, limiting the upper limit of contrast, and splicing the sub-blocks through bilinear interpolation, the local overexposure or loss of details caused by global histogram equalization is avoided, which is suitable for the enhancement of local structures in medical images.

[0086] In the embodiments of the present application, the system first preprocesses the denoised image sequence, divides each image into a plurality of non-overlapping sub-blocks according to the size of 16x16 pixels, calculates the gray histogram for each sub-block, sets the contrast limit threshold, and if the number of pixels of a certain gray level exceeds the threshold, the excess part is evenly distributed to other gray levels to avoid local overexposure caused by excessive single gray level; histogram equalization is performed on each sub-block to enhance the local contrast within the sub-block; finally, the edges of adjacent sub-blocks are smoothly spliced using the bilinear interpolation algorithm to eliminate the transition marks between sub-blocks, and the image data with uniform overall contrast and prominent local details is obtained.

[0087] It should be noted that the core reason for choosing CLAHE instead of traditional global histogram equalization is that the maxillofacial image has the characteristic of uneven gray distribution (e.g., the gray difference between the cheek soft tissue and the mandible is large), and global equalization will cause overexposure in the soft tissue area and loss of bone details; while the local processing feature of CLAHE can enhance the contrast of the local bone area and protect the soft tissue information. In addition, the size of the sub-block and the setting of the contrast threshold need to be adjusted according to the clinical needs, for example, for the high-fineness requirement of the orbital surgery, the sub-block can be reduced to 8x8 pixels, and the threshold can be reduced to 25, to more accurately capture the subtle structures of the orbital wall.

[0088] As an example, the denoised maxillofacial fracture image in step 301 is taken, and the gray difference between the fracture line and the surrounding bone in this image is only 15-20 (gray range 0-255), which is difficult to distinguish with the naked eye. After CLAHE processing (sub-block size 16x16, threshold for bone area 50, threshold for soft tissue area 30), the gray difference in the fracture line area is increased to 40-50, and the trend of the fracture line is clearly visible, while the soft tissue in the cheek does not appear overexposed, and the texture levels of the muscle and skin can still be distinguished, providing a clear basis for the positioning of the fracture line in the subsequent three-dimensional reconstruction.

[0089] Based on the above steps, the gray difference between the key anatomical structures and the surrounding tissues in the denoised image is significantly improved, the problem of low local contrast and hidden details in maxillofacial images is solved, and the subsequent three-dimensional reconstruction can more accurately restore the spatial structure of the bone, and the AI intelligent auxiliary diagnosis module can provide clearer feature input for identifying the lesion site.

[0090] Step 303, the maxillofacial surgery planning navigation system compensates the contours, edges and lines of the image data after the contrast enhancement by the improved Laplace pyramid algorithm to obtain second image data.

[0091] In the improved Laplace pyramid algorithm, the Gaussian filter of each layer in the traditional algorithm is replaced by a guided filter, and by constructing a guided filter pyramid and a Laplace pyramid, combining down-sampling, up-sampling and difference calculation operations, the edge and contour features of the image are retained while the details are compensated, and the edge blur caused by the traditional Gaussian filter is avoided.

[0092] Optionally, the guided filter pyramid is constructed to satisfy the following formula:

[0093] wherein, is the original image, i.e. the image data after the contrast enhancement, k is the layer index of the guided filter pyramid, is the initial layer of the guided filter pyramid, is the down-sampling operation, is the guided filter pyramid image of the kth layer, is the guided filter pyramid image of the k+1th layer, is the guided filter process, is the guided image of the guided filter pyramid image of the kth layer, is the filter kernel radius, is the regularization parameter.

[0094] Optionally, the Laplace pyramid is constructed to satisfy the following formula:

[0095] wherein, is the Laplace layer of the kth layer in the Laplace pyramid, is the total number of layers of the Laplace pyramid, is the expansion operation on the guided filter pyramid image of the k+1th layer, , is the up-sampling, is the top image of the guided filter pyramid.

[0096] In the embodiments of the present application, this step is divided into three sub-steps of constructing a guided filter pyramid, constructing a Laplace pyramid, detail compensation and image reconstruction to obtain second image data. The guided filter pyramid is constructed by taking the image after the contrast enhancement in step 302 as the initial layer, i.e. ​; For the k-th layer guided filtering pyramid image Guided filtering is used for processing (the guide image is...) (Set the filter kernel radius and regularization parameters), and then obtain the (k+1)th layer image through downsampling. Repeat this process until the number of pyramid layers reaches the preset value (usually 4-6 layers, determined according to image resolution); Construct the Laplace Pyramid: For the k-th layer (k < n, n is the total number of pyramid layers), calculate... and The difference between the upsampling and guided filtering results is used to obtain the Laplace layer. ; for the top level Laplace layer = Detail compensation and image reconstruction: from the top level Initially, the Laplace layer is upsampled layer by layer and superimposed on the next Laplace layer; that is, first... Upsampling to Same size, and then with The images are then added together to obtain the edge-compensated image, which is the second image data.

[0097] It should be noted that, in the downsampling step, when constructing the guided filter pyramid, the size of the k-th layer image after guided filtering is halved using bilinear interpolation to obtain the (k+1)-th layer. Repeating this operation reduces computational complexity and simultaneously constructs multi-scale images to cover different levels of anatomical information of the maxillofacial region. In the upsampling step, when constructing the Laplacian pyramid, the guided filter image of the (k+1)-th layer is upsampled and then guided filtered again. During image reconstruction, the top Laplacian layer is upsampled to the same size as the next layer, which enables size matching of images at different scales and ensures the accuracy of subsequent calculations and the final image size restoration.

[0098] As an example, taking the enhanced image in step 302, although the contrast is improved, the edge at the junction of the zygomatic arch and temporal bone still has slight blurring (the edge transition width is about 3 pixels). An improved Laplace pyramid algorithm is used for processing (pyramid layer number 5, r=5). After (=1e-3), in the reconstructed second image, the transition width of the zygomatic arch edge is reduced to 1 pixel, and the arc-shaped outline of the zygomatic arch and the connection seam with the temporal bone are clearly distinguishable. Even the fine bone texture on the surface of the zygomatic arch can be identified. At the same time, the image as a whole does not show artifacts caused by over-sharpening of the edges, which provides a guarantee for the accurate restoration of the spatial morphology of the zygomatic arch in subsequent 3D reconstruction.

[0099] Based on the above steps, the precise compensation of the image contour, edge and line is completed, the problems of edge blur and detail loss in the preprocessing are solved, and the second image data can completely retain the anatomical features of the maxillofacial skeleton.

[0100] Based on the above technical solution, the 2DTransformer is used to filter the noise of the maxillofacial X-ray image and retain the anatomical details, the CLAHE is used to enhance the local contrast to highlight the key structures such as the fracture line, and the Laplace pyramid is used to compensate the contour edge. The three-step collaborative optimization obtains high-quality second image data, effectively solves the problems of noise interference and detail concealment of the original image.

[0101] In a possible implementation, in combination with the above Figure 2 As shown in Figure 4 The process that the above step 205 recommends the bone plate scheme according to the three-dimensional data model and the diagnosis report based on the knowledge graph recommendation model can be implemented through the following steps 401-404: Step 401, the maxillofacial surgery planning and navigation system constructs a diagnosis-bone plate interaction matrix, and the interaction matrix is a matrix representing the historical selection interaction between the fracture type and the bone plate.

[0102] The diagnosis-bone plate interaction matrix is a structured data matrix used to quantitatively record the historical clinical selection correlation between different fracture types of the maxillofacial and various bone plates. The row dimension of the matrix corresponds to the common fracture types of the maxillofacial, the column dimension corresponds to the bone plates of different specifications, shapes or materials, and the matrix element value represents the selection correlation degree of a specific fracture type and a specific bone plate in historical cases.

[0103] In the embodiments of the present application, first, 8000+ maxillofacial bone fracture surgery case reports are collected as the matrix data source; an initial empty matrix is established with the fracture type as the row index and the bone plate type as the column index, wherein the row index covers 20+ common maxillofacial fracture types, and the column index covers 15+ clinically common bone plates; then, the number of historical selections of each bone plate under each fracture type is counted, and the number is standardized to a correlation degree value between 0 and 1; the unpopular correlation with a selection number less than 10 is removed, and the matrix is regularly updated to ensure the timeliness and effectiveness of the matrix.

[0104] As an example, in the constructed diagnostic-plate interaction matrix, the column index and element value corresponding to the row index "right mandibular angle comminuted fracture" are: "titanium alloy arc plate (45° arc)" 0.89 (historical selection 178 times, postoperative fixation effect good rate 96%), "titanium alloy special-shaped plate" 0.75 (selected 150 times, good effect rate 92%), "straight titanium alloy plate" 0.12 (only selected 24 times, good effect rate 75%), and the element values of the remaining plates are all less than 0.5 (considered as insufficient adaptation degree).

[0105] Based on the above steps, by constructing the diagnostic-plate interaction matrix, the scattered clinical selection experience is converted into structured and quantifiable data, which provides an objective historical association basis for the knowledge graph recommendation model.

[0106] In step 402, the maxillofacial surgery planning navigation system extracts the fracture type entity, the plate entity and the association relationship between the two from the knowledge graph based on the bone block data in the three-dimensional data model and the fracture type information in the diagnosis report.

[0107] In the embodiments of the present application, first, the fracture is located in the bone block from the three-dimensional data model output by the image reconstruction and segmentation module through the stereoscopic view rendered by VTK, and the bone block morphological characteristics are recorded. From the preliminary diagnosis information generated by the AI intelligent auxiliary diagnosis module and the diagnosis report after the artificial review by the physician, the explicit fracture type is extracted, and the ambiguous expressions are excluded. The pre-constructed maxillofacial surgery knowledge graph database is called to match the corresponding fracture type entity with the fracture bone block and fracture type as the retrieval condition. Further, all plate entities historically adapted to the entity are associated, and the clinically verified association relationship between the two is extracted. Finally, the extraction result is verified to ensure that the bone block data and the fracture type are not contradictory, and the association relationship is from the postoperative good effect case.

[0108] Based on the above steps, by combining the bone block positioning of the three-dimensional data model and the fracture type of the diagnosis report, the corresponding entity and association relationship in the knowledge graph are accurately extracted, which avoids the ambiguity of entity matching and selects the effective association relationship verified by the clinic.

[0109] In step 403, the maxillofacial surgery planning navigation system learns the feature representation and scoring function of the fracture type and the plate through the knowledge graph recommendation model, and then predicts the interaction probability of the current fracture type and each plate through the knowledge graph recommendation model based on the interaction matrix and the association relationship in the knowledge graph.

[0110] Optionally, the scoring function satisfies the following formula:

[0111] Wherein, u is a fracture type entity, i is a bone plate entity, and r is an association relationship between u and i. is an embedding vector of the fracture type entity u, is an embedding vector of the bone plate entity i, is an embedding vector of the relationship r. represents a vector inner product operation, is an interaction probability score.

[0112] In the embodiments of the present application, the knowledge graph recommendation model loads the embedding vector parameters completed by pre-training, which are obtained by iteratively training the diagnosis-bone plate interaction matrix and the knowledge graph triple data . During training, the (u, i) pairs with high correlation degree (element value ≥ 0.7) in the interaction matrix are taken as positive samples, and the (u, i) pairs with low correlation degree (element value < 0.3) are taken as negative samples. By minimizing the difference of the score functions of the positive and negative samples, the vector values of , , are adjusted, so that the Score(u, i, r) value of the positive sample is significantly higher than that of the negative sample. The current fracture type entity u, the association relationship r, and all candidate bone plate entities i in the knowledge graph extracted in step 402 are input into the model. The model calls the embedding vectors corresponding to u, r, and i through the feature representation layer. Then, the Score(u, i, r) value is calculated, and the Score values of all candidate bone plates are normalized through the softmax function to convert them into interaction probabilities (probability value range 0-1).

[0113] Based on the above steps, the knowledge graph recommendation model learns feature representation, converts clinical experience and data logic into quantitative probability values, and avoids the subjectivity of traditional bone plate selection.

[0114] Step 404, the maxillofacial surgery planning and navigation system recommends the most suitable bone plate and the corresponding combination mode from high to low according to the interaction probability, forming a bone plate scheme.

[0115] In the embodiments of the present application, the interaction probability list output by step 403 is sorted from high to low according to the probability value, and usually the top 3 are selected as the candidate recommendation. Secondly, the bone block data of the three-dimensional data model and the fracture severity in the diagnosis report are combined to match the combination mode for each high-probability bone plate.

[0116] Based on the above steps, by sorting according to the interaction probability and matching the personalized combination mode, the generated bone plate scheme has precision and practicality.

[0117] Based on the above technical scheme, the scattered clinical selection experience is structured by constructing a diagnosis-plate interaction matrix, the bone fracture type, plate entity and associated relationship in the knowledge graph are accurately extracted by combining the three-dimensional data model of the bone block data and the diagnosis report, the interaction probability is quantified by the knowledge graph recommendation model, and finally the adaptive plate and combination mode are recommended according to the probability to form a scheme, which not only avoids the subjectivity of traditional plate selection depending on personal experience, but also ensures the recommendation accuracy through data logic and clinical verification, reduces the burden of physician decision-making, and at the same time, the combination mode of the bone block morphology and the severity of the bone fracture is adapted, which provides scientific and efficient scheme support for the jaw and face bone fracture surgery plate fixation, and guarantees the operation fixation effect and postoperative recovery quality.

[0118] In a possible implementation manner, in combination with the above Figure 2 As shown in Figure 5 The process of compensating for the real-time deformation delay of the preoperative and intraoperative first image data by non-rigid registration in the above step 206 can be implemented by the following steps 501-503: Step 501, the jaw and face surgery planning navigation system performs pixel-level corresponding analysis on the preoperative and intraoperative first image data, and constructs a deformation field representing the nonlinear deformation relationship between the two.

[0119] Optionally, the deformation field representing the nonlinear deformation relationship between the two satisfies the following formula:

[0120] Wherein, is the deformation field, and , wherein x represents each point in the image, represents the displacement vector of each point, is the preoperative first image data, is the intraoperative first image data, is the intraoperative image after deformation field transformation, (·) is a measure function for measuring and similarity, is a bending energy penalty function, is a regularization weight.

[0121] In the embodiment of the present application, the preoperative first image data after preprocessing and the real-time collected first image data during operation are synchronously called, and the image registration algorithm is used for initial alignment to eliminate the global offset caused by slight change of device position; the aligned image is subjected to pixel-level corresponding analysis, and the preoperative image is taken as the reference, the gray value difference of the corresponding position of the intraoperative image is calculated pixel by pixel, and the pixel area with the gray value difference exceeding the threshold is marked; then, the initial deformation field is constructed based on B-spline interpolation​​ The displacement vector d(x) of each pixel point is obtained by displacement interpolation of its surrounding control points; the target function The initial deformation field is optimized, and the balance between Sim( ) and Reg(Φ) is achieved. Under the constraint of λ, the control point displacement is iteratively adjusted by the gradient descent algorithm until the target function E(Φ) converges. Finally, the optimized deformation field Φ is output.

[0122] Based on the above steps, the non-linear deformation difference between preoperative and intraoperative images is accurately quantified by the deformation field constructed by pixel-level correspondence analysis and the target function, avoiding the misalignment of preoperative planning and actual surgical field caused by ignoring the intraoperative anatomical changes.

[0123] Step 502, the maxillofacial surgery planning navigation system converts the deformation field into deviation parameters by Kalman filtering algorithm.

[0124] The Kalman filtering algorithm is a recursive filtering algorithm based on linear system state equation, which can dynamically estimate the system state through the "prediction-update" iterative process to suppress noise interference and is suitable for dynamic tracking of intraoperative deformation. The deviation parameter is a quantitative index extracted from the deformation field that can be directly used to adjust the preoperative planning, including the displacement vector and rotation angle of the local region.

[0125] In the embodiments of the present application, first, the key anatomical regions of the maxillofacial region are selected from the deformation field constructed in step 501; the arithmetic mean of the displacement vectors d(x) of all pixels in each region is taken to obtain the initial displacement parameters of the region; at the same time, the rotation angle of the region is calculated through the directional distribution of the pixel displacement in the region; these initial displacement parameters and rotation angles are taken as observation data and input into the Kalman filtering model with preset parameters; the filtering model performs smoothing processing on the observation data to eliminate fluctuations caused by single-pixel noise; finally, the deviation parameters of each key region, including the displacement vector and the rotation angle, are output.

[0126] Based on the above steps, the abstract deformation field is converted into quantitative and stable deviation parameters by Kalman filtering algorithm, realizing the conversion from deformation description to adjustment basis and avoiding the lag problem of intraoperative planning adaptation caused by the difficulty of directly applying the deformation field.

[0127] Step 503, the maxillofacial surgery planning navigation system adjusts the deformation field based on the deviation parameters by non-rigid registration to compensate for the real-time deformation delay.

[0128] Non-rigid registration refers to a technique that dynamically adjusts the correspondence between image pixels to achieve elastic adaptation of preoperative and intraoperative anatomical structures; real-time deformation delay refers to the temporal and spatial deviation between preoperative planning and actual intraoperative scene caused by dynamic changes of intraoperative anatomical structures, which needs to be compensated by deformation field adjustment.

[0129] In the embodiment of the present application, the deviation parameter output in step 502 is mapped to the corresponding anatomical region of the preoperative three-dimensional data model; based on the non-rigid registration algorithm, the deformation field constructed in step 501 is optimized with the deviation parameter as a constraint, for the region with a large displacement amount, the weight of the B-spline control point in the region is adjusted to increase the displacement compensation amount of the corresponding pixel; for the region where the rotation angle exists, the deformation direction is corrected by the local pixel rotation transformation, to ensure that the adjusted deformation field Φ' can accurately offset the real-time deformation delay; then, the adjusted deformation field Φ' is applied to the preoperative first image data and the surgical path scheme generated based on the data, so that the preoperative image and the surgical path are deformed according to Φ', and finally the anatomical structures are aligned with the intraoperative first image data; through the similarity verification, if it does not meet the standard, the deviation parameter mapping and deformation field adjustment steps are repeated until the intraoperative navigation accuracy requirement is met.

[0130] Based on the above steps, the deformation field is adjusted according to the deviation parameter through non-rigid registration, which effectively compensates for the intraoperative real-time deformation delay, realizes the dynamic adaptation of the preoperative planning and the intraoperative anatomical structure change, and avoids the navigation deviation caused by the traditional static planning that cannot adapt to the intraoperative change.

[0131] Based on the above technical solution, the accurate deformation field is constructed by pixel-level analysis of the objective function to capture the preoperative and intraoperative non-linear deformation, the deformation field is converted into quantized stable deviation parameters through Kalman filtering, and then the deformation field is adjusted based on the deviation parameters through non-rigid registration to compensate for the real-time deformation delay, which realizes the dynamic alignment of the preoperative image and the intraoperative anatomical structure through the cooperation of the three, and solves the problem that the preoperative static planning in the prior art cannot adapt to the intraoperative anatomical change.

[0132] The above mainly introduces the scheme of the embodiment of the present application from the perspective of device implementation. It can be understood that each device, for example, a maxillofacial surgery planning and navigation device, contains at least one of the corresponding hardware structure and software module for executing each function. Those skilled in the art should easily realize that the units and algorithm steps of each example described in combination with the embodiments disclosed in the present text can be realized in the form of hardware or the combination of hardware and computer software. Whether a certain function is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Professional technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0133] The embodiments of the present application can divide the functional units of the maxillofacial surgery planning navigation device according to the above method examples. For example, each functional unit can be divided according to each function, or two or more functions can be integrated in one processing unit. The integrated unit can be realized in the form of hardware or in the form of a software functional unit. It should be noted that the division of units in the embodiments of the present application is illustrative, and is only a logical functional division. In actual implementation, there can be another division manner.

[0134] In the case of using integrated units, Figure 6 A possible structural schematic diagram of the maxillofacial surgery planning navigation device (denoted as maxillofacial surgery planning navigation device 60) involved in the above embodiments is shown, which includes a processing unit 601 and a communication unit 602, and can also include a storage unit 603. Figure 6 The structural schematic diagram shown can be used to illustrate the structure of the maxillofacial surgery planning navigation device involved in the above embodiments.

[0135] When Figure 6 When the structural schematic diagram is used to illustrate the structure of the maxillofacial surgery planning navigation device involved in the above embodiments, the processing unit 601 is used to control and manage the action of the maxillofacial surgery planning navigation device, the communication unit 602 is used for communication between the maxillofacial surgery planning navigation device and other devices, and the storage unit 603 is used to store the program code and data of the maxillofacial surgery planning navigation device.

[0136] For example, the communication unit 602 is used to obtain first image data of a region to be operated, and the first image data is X-ray projection sequence data of a maxillofacial region.

[0137] The processing unit 601 is configured to determine second image data by preprocessing the first image data based on an image processing algorithm, the image processing algorithm comprising a 2D Transform denoising model, CLAHE, and an improved Laplace pyramid algorithm; perform three-dimensional reconstruction and segmentation on the second image data to obtain a three-dimensional data model; analyze the three-dimensional data model in combination with an AI intelligent auxiliary diagnosis model and a manual diagnosis by a doctor to generate a diagnosis report; input the three-dimensional data model and the diagnosis report into a double-channel model to output a surgical path scheme and a bone plate scheme; the double-channel model comprises a personalized path planning branch and an intelligent bone plate recommendation branch, the personalized path planning branch is configured to generate the surgical path scheme by constructing a three-dimensional coordinate system and using a gradually optimal fast expanding random tree path planning algorithm, and the intelligent bone plate recommendation branch is configured to recommend the bone plate scheme based on a knowledge graph recommendation model according to the three-dimensional data model and the diagnosis report, the bone plate scheme comprising a recommended most suitable bone plate and a corresponding combination mode, and the knowledge graph recommendation model is a model trained by a large amount of maxillofacial bone fracture bone plate selection report data; in the operation, the preoperative and intraoperative first image data are compensated for real-time deformation delay by using non-rigid registration, and AR real-time navigation technology is combined to realize maxillofacial surgery planning navigation.

[0138] In a possible implementation, the processing unit 601 is further configured to determine the second image data by preprocessing the first image data based on the image processing algorithm, including: performing denoising processing on the first image data based on a 2D Transform denoising model, the 2D Transform denoising model being a model trained by a large amount of X-ray data; enhancing the local region contrast of the denoised image data by using CLAHE; compensating the outline, edge and line of the image data with enhanced contrast by using an improved Laplace pyramid algorithm to obtain the second image data; and replacing the Gaussian filter of each layer in the traditional Laplace pyramid with a guided filter in the improved Laplace pyramid algorithm, constructing a guided filter pyramid and a Laplace pyramid, and retaining the edge features through downsampling, upsampling, guided filtering and difference calculation operations.

[0139] In a possible implementation, the guided filter pyramid is constructed to satisfy the following formula:

[0140] wherein, is an original image, that is, the image data with enhanced contrast, k is an index of a layer of the guided filter pyramid, is an initial layer of the guided filter pyramid, is a downsampling operation, is a guided filter pyramid image of the kth layer, For the guided filter pyramid image of layer k+1, To guide the filtering process, This is the guide image for the k-th layer guided filtering pyramid image. The radius of the filter kernel. This is the regularization parameter.

[0141] In one possible implementation, the Laplace pyramid is constructed to satisfy the following formula:

[0142] in, This refers to the k-th Laplace layer in the Pyramid of Laplace. This represents the total number of levels in the Pyramid of Laplace. To guide the filtering of the k+1 layer pyramid image Extended operations, , For upsampling, This is the top-level image of the guided filtering pyramid.

[0143] In one possible implementation, processing unit 601 is further configured to generate surgical path schemes using an asymptotically optimal fast expanding random tree path planning algorithm, including: S1. The initial exploration tree satisfies the following formula:

[0144] in, To explore the tree, To explore the node set of the tree, To explore the edge set of the tree (i.e., the search path), This is the initial position of the surgical instruments. The path cost is the starting point.

[0145] S2. Find the nearest node that satisfies the following formula:

[0146] in, For random sampling points, To explore tree node sets Zhongyu The nearest node, q is any node in the set of nodes in the exploration tree, and argmin(·) is the variable that makes the function take the minimum value.

[0147] S3. The generation of new nodes satisfies the following formula:

[0148] in, to generate a new node, as a step parameter.

[0149] S4, calculating the path cost of the new node , finding the domain node of in the node set , reconnecting the parent node by adjusting the edge set to optimize the path cost of the exploration tree ; when the target node is included in the node set V or the preset iteration number is reached, the algorithm is terminated, the path from to the target node is backtracked based on the edge set E of the exploration tree T, and the surgical path scheme is generated.

[0150] In a possible implementation, the processing unit 601 is further configured to recommend a bone plate scheme according to the three-dimensional data model and the diagnosis report based on a knowledge graph recommendation model, including: constructing a diagnosis-bone plate interaction matrix, the interaction matrix being a matrix representing historical selection interaction between a fracture type and a bone plate, where a matrix element represents the selection association of a specific fracture type and a specific bone plate; combining the bone block data in the three-dimensional data model and the fracture type information in the diagnosis report, extracting the corresponding fracture type entity, bone plate entity and the association relationship therebetween in the knowledge graph; learning the feature representation and scoring function of the fracture type and the bone plate through the knowledge graph recommendation model, and then predicting the interaction probability of the current fracture type and each bone plate through the knowledge graph recommendation model based on the interaction matrix and the association relationship in the knowledge graph; recommending the most suitable bone plate and the corresponding combination mode from high to low according to the interaction probability to form a bone plate scheme.

[0151] In a possible implementation, the scoring function satisfies the following formula:

[0152] wherein u is a fracture type entity, i is a bone plate entity, and r is an association relationship between u and i; is an embedding vector of the fracture type entity u, is an embedding vector of the bone plate entity i, is an embedding vector of the relationship r; represents a vector inner product operation, is an interaction probability score.

[0153] ​​​In a possible implementation, the processing unit 601 is further configured to compensate for real-time deformation delay of the preoperative and intraoperative first image data by using non-rigid registration, including: performing pixel-level correspondence analysis on the preoperative and intraoperative first image data, and constructing a deformation field representing a nonlinear deformation relationship between the preoperative and intraoperative first image data; converting the deformation field into bias parameters by using a Kalman filtering algorithm, the bias parameters including a displacement vector and a rotation angle of a local region; and adjusting the deformation field based on the bias parameters by using non-rigid registration, to compensate for the real-time deformation delay.

[0154] In a possible implementation, the construction of the deformation field representing the nonlinear deformation relationship between the preoperative and intraoperative first image data satisfies the following formula:

[0155] wherein, is the deformation field, and wherein x represents each point in the image, represents a displacement vector of each point, is the preoperative first image data, is the intraoperative first image data, is the intraoperative image after being transformed by the deformation field, is a measurement function for measuring and similarity, is a bending energy penalty function, is a regularization weight.

[0156] The processing unit 601 can be a processor or a controller, and the communication unit 602 can be a communication interface, a transceiver, a transceiver, a transceiver circuit, a transceiver device, or the like. The communication interface is a general term, and can include one or more interfaces. The storage unit 603 can be a memory. When the maxillofacial surgery planning and navigation device 60 is a chip, the processing unit 601 can be a processor or a controller, and the communication unit 602 can be an input interface and / or an output interface, a pin, or a circuit, etc. The storage unit 603 can be a storage unit (for example, a register, a cache, etc.) in the chip, or can be a storage unit (for example, a read-only memory (ROM), a random access memory (RAM), etc.) located outside the chip.

[0157] The communication unit can also be referred to as a transceiver unit. The antenna and the control circuit with transceiver function in the maxillofacial surgery planning navigation device 60 can be regarded as a communication unit 602 of the maxillofacial surgery planning navigation device 60, and the processor with processing function can be regarded as a processing unit 601 of the maxillofacial surgery planning navigation device 60. Optionally, the device for realizing the receiving function in the communication unit 602 can be regarded as a communication unit, which is used to execute the receiving steps in the embodiments of the present application, and the communication unit can be a receiver, a receiver, a receiving circuit, etc. The device for realizing the sending function in the communication unit 602 can be regarded as a sending unit, which is used to execute the sending steps in the embodiments of the present application, and the sending unit can be a transmitter, a transmitter, a sending circuit, etc.

[0158] Figure 6 The integrated units in the above embodiments can be stored in a computer readable storage medium if they are realized in the form of software function modules and sold or used as independent products. Based on such understanding, the technical solutions of the embodiments of the present application or the part of the prior art that essentially contributes or the whole or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor to execute all or part of the steps of the method described in the embodiments of the present application. The storage medium for storing the computer software product includes a U disk, a mobile hard disk, a read-only memory, a random access memory, a magnetic disk or an optical disk, and various media that can store program codes.

[0159] Figure 6 The units in the above embodiments can also be referred to as modules, for example, the processing unit can be referred to as a processing module.

[0160] The embodiments of the present application also provide a hardware structure schematic diagram of a maxillofacial surgery planning navigation device (denoted as a maxillofacial surgery planning navigation device 70), see Figure 7 The maxillofacial surgery planning navigation device 70 includes a processor 701, and optionally further includes a memory 702 connected with the processor 701.

[0161] In the first possible implementation, see Figure 7The maxillofacial surgery planning navigation device 70 further includes a transceiver 703. The processor 701, the memory 702 and the transceiver 703 are connected through a bus. The transceiver 703 is configured to communicate with other devices or communication networks. Optionally, the transceiver 703 can include a transmitter and a receiver. The device in the transceiver 703 for realizing the receiving function can be regarded as a receiver, and the receiver is configured to perform the steps of receiving in the embodiments of the present application. The device in the transceiver 703 for realizing the sending function can be regarded as a transmitter, and the transmitter is configured to perform the steps of sending in the embodiments of the present application.

[0162] Based on the first possible implementation manner, Figure 7 The structural schematic diagram shown can be used to show the structure of the maxillofacial surgery planning navigation device involved in the above embodiments.

[0163] In the above embodiments, the maxillofacial surgery planning navigation device can be implemented by a system on chip. In this case, the actions performed by the maxillofacial surgery planning navigation device can be realized by the system on chip, and the specific actions performed can be referred to the above and will not be described here. Figure 7 The structural schematic diagram shown can be used to show the structure of the maxillofacial surgery planning navigation device involved in the above embodiments.

[0164] In the implementation process, the steps in the method provided by the embodiments of the present application can be completed by the integrated logic circuit of hardware in the processor or the instructions in the form of software. The steps of the method disclosed by the embodiments of the present application can be directly embodied as the execution completed by the hardware processor, or executed by the combination of hardware and software modules in the processor.

[0165] The processor in the present application can include but is not limited to at least one of the following: central processing unit (CPU), microprocessor, digital signal processor (DSP), microcontroller unit (MCU), or artificial intelligence processor and various computing devices running software, each of which can include one or more cores for executing software instructions to perform operations or processing. The processor can be a separate semiconductor chip, or can be integrated with other circuits to form a semiconductor chip, for example, it can form a SoC (system on chip) with other circuits such as coding and decoding circuits, hardware acceleration circuits or various bus and interface circuits, or it can be integrated as a built-in processor in an ASIC. The ASIC integrated with the processor can be packaged separately or packaged together with other circuits. In addition to including cores for executing software instructions to perform operations or processing, the processor can further include necessary hardware accelerators, such as field programmable gate array (FPGA), PLD (programmable logic device), or logic circuits for implementing special logic operations.

[0166] The memory in the embodiments of the present application can include at least one of the following types: read-only memory (ROM) or other types of static storage devices that can store static information and instructions, random access memory (RAM) or other types of dynamic storage devices that can store information and instructions, and electrically erasable programmable read-only memory (EEPROM). In some scenarios, the memory can also be a compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compact discs, laser discs, optical discs, digital versatile discs, Blu-ray discs, etc.), magnetic disk storage medium or other magnetic storage device, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and capable of being accessed by a computer, but is not limited thereto.

[0167] The embodiments of the present application also provide a computer readable storage medium including instructions, which, when executed on a computer, cause the computer to perform any of the above methods.

[0168] The embodiments of the present application also provide a computer program product including instructions, which, when executed on a computer, cause the computer to perform any of the above methods.

[0169] The embodiments of the present application also provide a chip, which includes a processor and an interface circuit, the interface circuit is coupled with the processor, the processor is configured to execute computer programs or instructions to implement the above method, and the interface circuit is configured to communicate with other modules outside the chip.

[0170] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented using software programs, implementation can be, in whole or in part, in the form of a computer program product. This computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, computer instructions can be transmitted from one website, computer, server, or data center to another via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium accessible to a computer or a data storage device containing one or more servers, data centers, etc., that can be integrated with the medium. The available media can be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., DVDs), or semiconductor media (e.g., solid-state drives (SSDs)).

[0171] Although this application has been described herein in conjunction with various embodiments, those skilled in the art, by reviewing the accompanying drawings, disclosure, and appended claims, will understand and implement other variations of the disclosed embodiments in carrying out the claimed application. In the claims, the word "comprising" does not exclude other components or steps, and "a" or "an" does not exclude multiple instances. A single processor or other unit can implement several functions listed in the claims. While different dependent claims may recite certain measures, this does not mean that these measures cannot be combined to produce good results.

[0172] Although this application has been described in conjunction with specific features and embodiments, it is obvious that various modifications and combinations can be made thereto without departing from the spirit and scope of this application. Accordingly, this specification and drawings are merely exemplary illustrations of this application as defined by the appended claims, and are considered to cover any and all modifications, variations, combinations, or equivalents within the scope of this application. Clearly, those skilled in the art can make various alterations and modifications to this application without departing from the spirit and scope of this application. Thus, if such modifications and modifications of this application fall within the scope of the claims of this application and their equivalents, this application is also intended to include such modifications and modifications.

Claims

1. A method for planning and navigation in maxillofacial surgery, characterized in that, include: Acquire first image data of the area to be operated on, wherein the first image data is X-ray projection sequence data of the maxillofacial region; The first image data is preprocessed based on an image processing algorithm to determine the second image data. The image processing algorithm includes a 2D Transformer denoising model, CLAHE, and an improved Laplace pyramid algorithm. The second image data is reconstructed and segmented in three dimensions to obtain a three-dimensional data model; By combining the AI-assisted diagnostic model with the physician's manual diagnostic analysis of the 3D data model, a diagnostic report is generated. The three-dimensional data model and the diagnostic report are input into a dual-channel model, which outputs a surgical path plan and a bone plate plan. The dual-channel model includes a personalized path planning branch and an intelligent bone plate recommendation branch. The personalized path planning branch is used to generate a surgical path plan by constructing a three-dimensional coordinate system and using an asymptotically optimal fast expanding random tree path planning algorithm. The intelligent bone plate recommendation branch is used to recommend bone plate plans based on the three-dimensional data model and the diagnostic report using a knowledge graph recommendation model. The bone plate plan includes recommending the most suitable bone plate and the corresponding combination mode. The knowledge graph recommendation model is a model trained by a large number of maxillofacial bone fracture bone plate selection report data. During the operation, non-rigid registration was used to compensate for the real-time deformation delay of the first image data before and during the operation, and AR real-time navigation technology was combined to realize the planning and navigation of maxillofacial surgery.

2. The method according to claim 1, characterized in that, The step of preprocessing the first image data based on an image processing algorithm to determine the second image data includes: The first image data is denoised based on the 2DTransformer denoising model, which is a model trained with a large amount of X-ray data. CLAHE is used to enhance the local contrast of the noise-reduced image data. The improved Laplace pyramid algorithm compensates for the contours, edges, and lines of the contrast-enhanced image data to obtain the second image data. The improved Laplace pyramid algorithm replaces the Gaussian filtering of each layer in the traditional Laplace pyramid with guided filtering. By constructing a guided filtering pyramid and a Laplace pyramid, edge features are preserved through downsampling, upsampling, guided filtering, and difference calculation operations.

3. The method according to claim 2, characterized in that, The construction of the guided filter pyramid satisfies the following formula: in, The image is the original image, i.e., the image data after contrast enhancement, and k is the index of the guided filter pyramid layer. This serves as the initial layer of the guiding filter pyramid. For the downsampling operation, For the guided filter pyramid image of the k-th layer, For the guided filter pyramid image of layer k+1, To guide the filtering process, This is the guide image for the k-th layer guided filtering pyramid image. The radius of the filter kernel. This is the regularization parameter.

4. The method according to claim 3, characterized in that, The construction of the Laplace pyramid satisfies the following formula: in, This refers to the k-th Laplace layer in the Pyramid of Laplace. This represents the total number of levels in the Pyramid of Laplace. To guide the filtering of the k+1 layer pyramid image Extended operations, , For the upsampling, This is the top-level image of the guided filtering pyramid.

5. The method according to claim 1, characterized in that, The method of generating surgical path schemes using the asymptotically optimal fast expanding random tree path planning algorithm includes: S1. The initial exploration tree satisfies the following formula: in, For the exploration tree, To explore the node set of the tree, To explore the edge set of the tree (i.e., the search path), This is the initial position of the surgical instruments. The path cost is the starting point. S2. Find the nearest node that satisfies the following formula: in, For random sampling points, For the set of exploration tree nodes Zhongyu The nearest node, q is any node in the set of nodes in the exploration tree, and argmin(·) is the variable that makes the function take the minimum value; S3. The generation of new nodes satisfies the following formula: in, For the newly generated node, This is the step size parameter; S4. Calculate the new node. Path cost In the node set Search in Domain nodes, by adjusting edge sets Reconnect parent nodes to optimize the exploration tree The path cost; when the target node is included in the node set V or the preset number of iterations is reached, the algorithm terminates and backtracks from the edge set E of the exploration tree T. The path to the target node is used to generate the surgical path scheme.

6. The method according to claim 1, characterized in that, The knowledge graph-based recommendation model recommends bone plate schemes based on the three-dimensional data model and the diagnostic report, including: Construct a diagnosis-plate interaction matrix, which is a matrix representing the historical selection interaction between fracture type and plate, wherein the matrix elements represent the selection association between a specific fracture type and a specific plate. By combining the bone block data in the three-dimensional data model and the fracture type information in the diagnostic report, the corresponding fracture type entities, bone plate entities, and their relationships in the knowledge graph are extracted. The knowledge graph recommendation model is used to learn the feature representation and scoring function of fracture type and bone plate, and then the knowledge graph recommendation model is used to predict the relationship between the interaction matrix and the knowledge graph to determine the interaction probability between the current fracture type and each bone plate. Based on the interaction probabilities from high to low, the most suitable bone plate and corresponding combination mode are recommended to form the bone plate scheme.

7. The method according to claim 6, characterized in that, The scoring function satisfies the following formula: Wherein, u is the fracture type entity, i is the bone plate entity, and r is the relationship between u and i; Let u be the embedding vector of the fracture type entity. Let i be the embedding vector of the bone plate entity. Let r be the embedding vector of relation r; This represents the vector dot product operation. The interaction probability is scored.

8. The method according to claim 1, characterized in that, The method of using non-rigid registration to compensate for real-time deformation delay in the first image data before and during surgery includes: Pixel-level correspondence analysis was performed on the first image data before and during the operation to construct a deformation field characterizing the nonlinear deformation relationship between the two. The deformation field is converted into deviation parameters using a Kalman filter algorithm. The deviation parameters include the displacement vector and rotation angle of the local region. Non-rigid registration is used to adjust the deformation field based on the deviation parameters to compensate for real-time deformation delay.

9. The method according to claim 8, characterized in that, The deformation field that characterizes the nonlinear deformation relationship between the two satisfies the following formula: in, For the deformation field, and ,in x Represents each point in the image. This represents the displacement vector of each point. This refers to the first preoperative imaging data. This refers to the first image data during the procedure. These are intraoperative images after deformation field transformation. (·) is used for measurement and Similarity measurement function Let be the bending energy penalty function. For regularization weights.

10. A maxillofacial surgery planning and navigation system, characterized in that, include: Image acquisition module, image preprocessing module, image reconstruction and segmentation module, AI intelligent diagnosis module, personalized path planning module, intelligent bone plate recommendation module, offset analysis and correction module, and AR navigation module; The image acquisition module is used to acquire the first image data of the area to be operated on; The image preprocessing module is used to preprocess the first image data based on the image processing algorithm to determine the second image data; The image reconstruction and segmentation module is used to perform three-dimensional reconstruction and segmentation on the second image data to obtain a three-dimensional data model; The AI-assisted diagnosis module is used to combine the AI-assisted diagnosis model with the physician's manual diagnosis and analysis of the three-dimensional data model to generate a diagnostic report. The personalized path planning module is used to input the three-dimensional data model and the diagnostic report into a dual-channel model and output a surgical path plan. The intelligent bone plate recommendation module is used to input the three-dimensional data model and the diagnostic report into a dual-channel model and output a bone plate solution. The offset analysis and correction module is used to compensate for real-time deformation delay of the first image data before and during the operation by using non-rigid registration. The AR navigation module is used to combine AR real-time navigation technology to realize maxillofacial surgery planning and navigation.

Images