Orthognathic surgery precise simulation planning system based on ai image recognition

Abstract

The application relates to the field of digital medical technology and discloses an orthognathic surgery precise simulation planning system based on AI image recognition.The system acquires and processes multi-modal data including hard tissue, soft tissue surface, dynamic movement and biomechanical characteristics, constructs an individual biomechanical atlas representing physical characteristics of a patient and a myodynamic vector field representing expression movement patterns of the patient.In the static prediction stage, an AI static prediction module takes the individual biomechanical atlas as a physical constraint to generate a postoperative static facial appearance conforming to physical reality; in the dynamic reconstruction stage, a dynamic expression reconstruction module uses a nonlinear modulation engine to nonlinearly modulate the myodynamic vector field by using the atlas, simulates the damping effect of tissue physical characteristics on expression movement, and generates a high-fidelity postoperative dynamic model.The application improves the precision of surgery simulation, realizes reliable prediction of postoperative dynamic expressions, and provides doctors with an efficient interactive planning and evaluation tool.

Description

Technical Field

[0001] This invention relates to the field of digital medical technology, specifically to a precise simulation and planning system for orthognathic surgery based on AI image recognition. Background Technology

[0002] Currently, orthognathic surgery is an important means of correcting jaw and dental deformities, aiming to restore patients' occlusal function and improve facial aesthetics. The core challenge of surgical planning lies in accurately predicting how the overlying soft tissues will respond and remodel after hard tissue osteotomy. To address this challenge, computer-aided surgical simulation planning systems have become indispensable tools in clinical practice, assisting doctors in developing plans and anticipating outcomes.

[0003] To address the aforementioned surgical planning needs, existing technologies typically begin by utilizing the patient's hard tissue scan data and static facial surface scan data. After the surgeon develops a virtual osteotomy plan, the system uses algorithms to predict the postoperative static facial appearance. These algorithms are either based on simplified biological tissue models such as the finite element method or rely on geometric deformation patterns learned from a large number of historical cases. For simulating dynamic expressions, some solutions collect the patient's preoperative facial movement data. This movement data is then reapplied to the predicted postoperative static facial appearance in some way, thereby generating a dynamic preview effect.

[0004] However, the aforementioned technical solutions have limitations. Static prediction often ignores the physical characteristics of the patient's individual soft tissues, leading to inaccurate predicted deformation. Dynamic simulation is mostly a simple redirection of preoperative movement rather than a true physical simulation, failing to consider the biomechanical changes brought about by surgery, resulting in stiff and distorted facial expressions. Static prediction and dynamic simulation are physically disconnected, and the system lacks a unified, individualized physical model to constrain static morphology and modulate dynamic expressions. This disconnect reduces overall fidelity and makes it difficult to meet the high-precision requirements of clinical practice.

[0005] Therefore, the present invention provides a precise simulation planning system for orthognathic surgery based on AI image recognition to address the shortcomings of existing technologies. Summary of the Invention

[0006] To address the shortcomings of existing technologies, this invention provides a precise simulation and planning system for orthognathic surgery based on AI image recognition. This system solves the problem that existing orthognathic surgery simulation systems fail to fully consider the biomechanical characteristics of individual patient soft tissues, resulting in insufficient accuracy and lack of physical realism in the prediction of postoperative static morphology and dynamic expressions.

[0007] To achieve the above objectives, the present invention provides a precise simulation and planning system for orthognathic surgery based on AI image recognition, comprising: Data acquisition equipment is used to acquire patient data in multiple modalities; The data processing terminal is used to run the following modules: The multimodal data acquisition and registration module is used to receive and process the patient data of the multiple modalities and output the fused data; The individualized feature field construction module is used to extract individual biomechanical maps and muscle dynamic vector fields based on the fused data; The AI ​​static prediction module is used to receive the virtual surgical plan and use the individual biomechanical atlas to predict and generate the postoperative static facial appearance. The dynamic expression reconstruction module is used to simulate and generate a postoperative dynamic model based on the postoperative static facial appearance using the muscle dynamic vector field and the individual biomechanical atlas. The visualization and interactive evaluation module is used to present the postoperative static facial appearance and the postoperative dynamic model, and to receive adjustment instructions from the operator to iteratively optimize the virtual surgical plan.

[0008] By adopting the above technical solution, the AI ​​static prediction module, when predicting postoperative static facial appearance, not only relies on the virtual surgical plan but also introduces an individual biomechanical atlas characterizing the physical properties of the patient's soft tissue as a physical constraint. Furthermore, the dynamic expression reconstruction module, when simulating postoperative dynamic expressions, also utilizes this individual biomechanical atlas to nonlinearly modulate the muscle dynamic vector field representing the patient's original movement pattern. Therefore, this invention uniformly and coherently applies the patient's individual biomechanical characteristics to both static prediction and dynamic reconstruction, obtaining simulation prediction results with high fidelity and high physical realism for both postoperative static facial morphology and dynamic expressions, thus improving the accuracy and reliability of surgical planning.

[0009] Preferably, when the AI ​​static prediction module utilizes the individual biomechanical map, it uses the map as a physical constraint, which is manifested as follows: The predicted soft tissue displacement amplitude is negatively correlated with the corresponding Young's modulus value in the individual biomechanical map.

[0010] By adopting the above technical solution, the physical hardness characteristics of soft tissue are directly transformed into quantitative rules that the AI ​​model must follow when predicting displacement. This effectively avoids unrealistic excessive displacement of the model in areas of tough tissue, ensuring the physical rationality of the postoperative static facial prediction results.

[0011] Preferably, the dynamic expression reconstruction module includes a nonlinear modulation engine, which is used to nonlinearly modulate the muscle dynamic vector field using the individual biomechanical atlas.

[0012] By adopting the above technical solution, a computing unit specifically designed to couple biomechanical characteristics with facial expression movement patterns was established, providing a core functional architecture for realizing dynamic facial expression simulation based on physical characteristics.

[0013] Preferably, the nonlinear modulation engine is used to quantify the physical damping effect of tissue physical properties on facial expression movements using a nonlinear modulation function that monotonically decreases with respect to Young's modulus.

[0014] By adopting the above technical solution, a clear mathematical relationship is established, which converts the physical quantity of Young's modulus into a decay modulation factor for the amplitude of facial expression movements, so that the physical phenomenon that the stiffer the tissue, the stronger the inhibition of the amplitude of movement can be accurately simulated.

[0015] Preferably, the nonlinear modulation function is an exponential decay function.

[0016] By adopting the above technical solution, a specific and efficient computational implementation method is provided. This function can smoothly handle the transition from soft tissue to hard tissue and ensure the nonlinear characteristics of the modulation effect.

[0017] Preferably, the nonlinear modulation engine is specifically used for: The modulated muscle dynamic vector field is calculated by performing vertex-by-vertex multiplication on the modulation factor calculated by the nonlinear modulation function and the muscle dynamic vector field. The modulated muscle dynamics vector field is then applied to the postoperative static facial appearance to generate the postoperative dynamic model.

[0018] By adopting the above technical solution, a clear calculation path is defined. This path scales the vector field representing the original motion trend according to the physical stiffness of each vertex. Then, this physically corrected displacement result is applied to the postoperative static model, ultimately generating a dynamic expression model that integrates the three factors of surgical influence, individual motion pattern, and individual physical characteristics.

[0019] Preferably, when extracting the individual biomechanical map, the individualized feature field construction module specifically uses a spatial interpolation algorithm to calculate the corresponding Young's modulus value for each vertex on the soft tissue mesh based on the biomechanical characteristic data contained in the patient data of the multiple modalities.

[0020] By adopting the above technical solution, biomechanical characteristic data collected from a finite number of discrete points can be expanded into a continuous scalar field covering the entire facial soft tissue mesh, providing the necessary, global, individualized physical characteristic input for subsequent AI physical constraints and nonlinear modulation.

[0021] Preferably, when extracting the muscle dynamics vector field, the individualized feature field construction module is specifically used to extract the vertex set of the relaxed state and the vertex set of the target facial expression state from the dynamic soft tissue data contained in the multimodal patient data, and generate the muscle dynamics vector field by calculating the spatial displacement between the two.

[0022] By adopting the above technical solution, the patient's facial expression movement pattern can be decoupled and separated from its original facial shape to obtain a pure vector field that represents muscle driving ability, which can be reused on any new facial shape (such as postoperative static face).

[0023] Preferably, the iterative optimization function of the visualization and interactive evaluation module is specifically used for: The system receives adjustment instructions from the operator for the virtual surgical plan and updates the bone displacement vector field in response to the adjustment instructions. The bone displacement vector field is a quantification result of the virtual surgical plan. This triggers the AI ​​static prediction module and the dynamic expression reconstruction module, which use the updated skeletal displacement vector field as input to re-perform the calculation.

[0024] By adopting the above technical solution, a closed-loop, interactive surgical planning process has been constructed. The operator can modify the surgical plan in real time and quickly obtain dual feedback on static morphology and dynamic facial expression changes. This allows for fine-tuning and optimization of the plan through multiple iterations until the desired effect is achieved.

[0025] Preferably, the multimodal patient data includes: Hard tissue data acquired by computed tomography (CT) or cone-beam computed tomography (CBCT); Static soft tissue surface acquired by a 3D optical scanner; Dynamic soft tissue data acquired by a 4D scanner; And biomechanical property data obtained by ultrasound elastography equipment.

[0026] By adopting the above technical solution, the complete data foundation required to achieve the high-fidelity simulation described in this invention is clarified, ensuring that the system can acquire all the original information required to construct a four-in-one individualized digital model that includes hard tissue, soft tissue surface, soft tissue movement, and soft tissue physical properties.

[0027] This invention provides a precise simulation and planning system for orthognathic surgery based on AI image recognition, which has the following beneficial effects: 1. The system of this invention uses an AI static prediction module to apply an individual's biomechanical atlas as a physical constraint, ensuring the physical rationality of postoperative static facial prediction. Simultaneously, the dynamic expression reconstruction module also utilizes this atlas to nonlinearly modulate the muscle dynamic vector field. This technical solution, which uniformly applies individual biomechanical characteristics to both static prediction and dynamic reconstruction, ensures that the final output static morphology and dynamic expression are consistent with the patient's individual physical characteristics, achieving a more accurate and reliable simulation effect.

[0028] 2. The system of this invention first extracts a muscle dynamic vector field representing the patient's inherent movement pattern from the patient data. Then, through a nonlinear modulation engine, it uses an individual biomechanical atlas characterizing postoperative soft tissue properties to physically correct this vector field. This approach accurately simulates the physical damping effect of tissue stiffness on muscle movement, so that the predicted postoperative facial expressions are no longer simple reproductions of movement, but rather a true dynamic response that conforms to the new biomechanical environment.

[0029] 3. The visualization and interactive evaluation module in this invention provides operators with real-time visual feedback including static and dynamic results, and supports rapid iterative optimization of virtual surgical plans through adjustment commands. This closed-loop interactive workflow enables doctors to efficiently explore the advantages and disadvantages of different surgical designs, intuitively assess their impact on aesthetics and function, thereby developing more ideal surgical plans before surgery and communicating expected outcomes more clearly to patients. Attached Figure Description

[0030] Figure 1 This is a schematic diagram of the functional modules of the AI ​​image recognition-based orthognathic surgery precision simulation planning system of the present invention; Figure 2 This is a flowchart illustrating the accurate simulation planning method provided in an embodiment of the present invention. Detailed Implementation

[0031] The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0032] This embodiment provides a precise simulation planning system for orthognathic surgery based on AI image recognition, as well as a corresponding method. The system introduces the patient's individualized biomechanical properties (i-BMP) as a physical constraint for the AI ​​model and uses these properties to nonlinearly modulate the decoupled muscle dynamic vector field (MFV), thereby achieving high-fidelity postoperative static and dynamic facial simulation.

[0033] See attached document Figure 1 The AI ​​image recognition-based orthognathic surgery precision simulation planning system can be deployed on a medical workstation or on a cloud server. The system includes one or more processors and a memory that stores computer program instructions. When the one or more processors execute the computer program instructions, they implement the steps of the subsequent method embodiments of the present invention.

[0034] The system's hardware environment includes data acquisition equipment and data processing terminals.

[0035] Data acquisition equipment is used to acquire the raw data required to build a personalized virtual model of a patient. The data acquisition equipment includes: equipment for acquiring hard tissue data. CT (computed tomography) or CBCT (cone-beam computed tomography) equipment; used to acquire static soft tissue surface data. A 3D optical scanner; used to acquire dynamic soft tissue data. 4D scanners; and for acquiring biomechanical property data. Ultrasonic elastography devices, such as shear wave elastography (SWE) devices.

[0036] The data processing terminal is a computer device with high-performance computing capabilities, such as a medical workstation or server equipped with a graphics processing unit (GPU). The data processing terminal is used to run the core functional modules of this system to support high-throughput 3D data computing, AI model inference, and physical modulation operations.

[0037] In terms of functional modules, the system specifically includes a data acquisition interface, a data processing module, an AI prediction engine, a dynamic reconstruction module, and a visual interactive interface; The data acquisition interface is used to connect to and receive raw data from the aforementioned data acquisition devices. ), and transmit the data to the data processing module; The data processing module is used to preprocess the collected raw data, register multimodal data, and construct individualized feature fields. Specifically, the data processing module includes a multimodal data registration unit, an i-BMP generation unit, and an MFV decoupling unit.

[0038] In this embodiment, the AI ​​prediction engine is a conditional response AI (CR-AI) engine. This engine receives a virtual surgical plan (quantized as a bone displacement vector field) planned by the doctor. The system also includes an individual biomechanical map (i-BMP) generated by the data processing module, and based on the above inputs, predicts and generates the static soft tissue morphology after surgery.

[0039] The dynamic reconstruction module is used to simulate dynamic facial expressions. This module loads i-BMP and decoupled MFV, calculates the modulated dynamic vector through a nonlinear modulation engine, and reconstructs the postoperative dynamic facial expressions.

[0040] A visual interactive interface is used to present the preoperative model, postoperative static simulation model, and postoperative dynamic simulation animation to the operator (e.g., a doctor). At the same time, the interface receives the operator's adjustment instructions for the virtual surgical plan in order to achieve closed-loop iterative optimization of the plan.

[0041] See attached document Figure 1 The functional modules of the system can be implemented by one or more processors in the data processing terminal executing computer program instructions in memory.

[0042] The multimodal data acquisition and registration module is used to receive hard tissue data from the data acquisition interface. Static soft tissue surface Dynamic soft tissue data and biomechanical property data This module establishes a reference three-dimensional coordinate system, for example, using... The data is referenced to the world coordinate system. This module is further used to perform 3D registration, for example, using an iterative nearest-point algorithm or a registration algorithm based on anatomical feature points. and The corresponding soft tissue contours in the data are aligned to generate a preoperative baseline 3D model that includes the bone and soft tissue surfaces. This module will also Sequence and The spatial coordinates of the data are mapped to this reference three-dimensional coordinate system.

[0043] The individualized feature field construction module processes registered multimodal data to extract key feature fields for subsequent AI computation and dynamic simulation. This module specifically includes an i-BMP generation unit and an MFV decoupling unit.

[0044] The i-BMP generation unit is used to receive discrete biomechanical property data. (i.e., a series of spatial coordinate points and their corresponding Young's modulus measurements) and The soft-tissue mesh. This cell is used for spatial interpolation algorithms (such as Kriging interpolation) to... Each vertex on the soft tissue mesh Calculate the corresponding Young's modulus value, thereby generating an individual biomechanical map. This map is a scalar field defined on the vertices of a soft-tissue mesh. For example, when using Kriging interpolation, the vertices... Young's modulus The calculation is as follows: ; in, yes The total number of discrete measurement points in the middle; It is the first Young's modulus values ​​at each measurement point; It is based on the vertex With all measurement points The Kriging interpolation weights are calculated based on the spatial relationships between them.

[0045] The MFV decoupling unit is used to process registered dynamic soft tissue data. This unit utilizes the consistency of vertex topology in the 4D data stream to extract the relaxed state surface (whose vertex set is denoted as ) from the sequence. ) and the target expression state surface (e.g., the maximum smile state, whose vertex set is denoted as ). ).

[0046] This unit computes each vertex on the mesh. The displacement vector from the relaxed state to the target facial expression state is the muscle dynamics vector field. : ; in, It is the vertex Spatial coordinates of the vertex set; It is the vertex exist Spatial coordinates of the vertex set.

[0047] In this embodiment, the AI ​​static prediction module is a Conditional Response AI (CR-AI) engine. This module receives the skeletal displacement vector field transmitted by the visualization interaction module. (Should (This is a quantitative result of the doctor's virtual surgical planning). This module also loads the data generated by the individualized feature field construction module. The AI ​​engine (For example, the input features of a graph neural network GNN) include grid topology, etc. Defined hard tissue boundary conditions, and by Physical properties of each vertex provided After training, the engine solves the problem in... Under physical constraints, by The resulting soft tissue displacement, and output of postoperative static soft tissue displacement. : ; This physical constraint is manifested in: when the vertex of When the value is high, the AI ​​model predicts the displacement of the vertex. The amplitude is correspondingly suppressed. This module further calculates the postoperative static vertex set. : ; in, Preoperative baseline 3D model The set of soft tissue vertices.

[0048] The dynamic facial expression reconstruction module is used to reconstruct the postoperative static model output by the AI ​​static prediction module. (Its vertex set is) Based on this, it simulates dynamic facial expressions after surgery. This module includes a non-linear modulation engine. This engine is used for loading... Generated by the MFV decoupling unit ,as well as The nonlinear modulation engine defines a nonlinear modulation function. The function For a question about Young's modulus A monotonically decreasing function, used to characterize tissue stiffness. For muscle movement The physical damping effect. For example, the function It can be implemented as an exponentially decaying function: ; in, It is by Provided Young's modulus; It is a non-negative calibration coefficient used to control attenuation sensitivity.

[0049] The engine calculates the modulated dynamic displacement vector per vertex. : ; Subsequently, the dynamic expression reconstruction module will Apply to The postoperative dynamic model was obtained by calculation. vertex set : ; The visualization and interactive evaluation module is used to present multi-dimensional simulation results to the operator on the display interface. This module is used at least for: side-by-side presentation of... and A 3D view for static evaluation; and side-by-side playback of real preoperative dynamic data. Animation and postoperative simulation dynamics (i.e.) arrive The module uses interpolated animation for dynamic evaluation. It further receives operator adjustments to the virtual osteotomy plan, which will update... This triggers the AI ​​static prediction module and the dynamic expression reconstruction module to update the results. The calculation is re-executed to achieve closed-loop iterative optimization of the planning scheme.

[0050] See attached document Figure 2 The method steps described in this embodiment can be derived from... Figure 1 The data processing terminal in the system shown executes a method that includes a multimodal data acquisition and preprocessing step, which aims to acquire raw data of different sources and modalities required to construct a patient-specific virtual model. First, acquire hard tissue data. The patient's skull is scanned using a CT (computed tomography) or CBCT (cone-beam computed tomography) scanner. This scan acquires hard tissue data including anatomical structures such as the maxilla and mandible. , It is usually stored in the DICOM (Digital Imaging and Communications in Medicine) standard format; Simultaneously, obtain static soft tissue surface Using a 3D optical scanner, such as a structured light scanner or a time-of-flight (ToF) camera, the 3D surface morphology of the patient's facial soft tissue in a natural, relaxed state is captured. This morphological data is recorded as a static soft tissue surface. They are typically stored as STL or OBJ 3D mesh file formats; Furthermore, acquire dynamic soft tissue data. Using a 4D scanning system that continuously captures a three-dimensional surface at a specific time frame rate (e.g., 30 frames per second), the patient is guided to perform a standardized sequence of facial expressions during the acquisition process, such as transitioning from a natural relaxed state to a maximum smile, holding it for several seconds, and then returning to a natural relaxed state. It is a series of time-ordered three-dimensional surface meshes, and this data sequence records the complete spatiotemporal process of facial expression changes; Next, biomechanical property data were obtained. In this embodiment, a non-invasive ultrasound elastography technique is used, specifically a shear wave elastography (SWE) device. The operator uses an ultrasound probe to scan multiple key soft tissue areas of the patient's face, including but not limited to the zygomatic fat pad, cheek, perioral area, and chin.

[0051] For each measurement, the SWE device generates an acoustic radiation force pulse within the tissue, exciting a shear wave, and calculates the tissue's Young's modulus by tracking the propagation speed of the shear wave. (Unit: kPa) It is a discrete dataset consisting of a series of data points, each containing its coordinates in three-dimensional space (typically relative to the ultrasound probe coordinate system) and the Young's modulus measured at those coordinates. value; After the data collection is completed, the data from the above four modalities ( Preliminary preprocessing may be performed, including... Denoising and thresholding of the data are performed to initially extract the skeletal contour. and Noise removal or hole filling for grid data, and the Outlier removal is performed on the data points, and all preprocessed data is transmitted to the data processing module to perform subsequent registration and feature field construction steps.

[0052] See attached document Figure 2 In this embodiment, after the multimodal data acquisition and preprocessing steps, the method continues to perform the three-dimensional model registration and individual feature field construction steps. The purpose of this step is to fuse heterogeneous data of different modalities into a unified three-dimensional coordinate system and generate the key input feature fields required for subsequent AI prediction and dynamic simulation. This step first involves performing a preoperative baseline model. The construction of hard tissue data is based on multimodal data acquisition and preprocessing steps. The defined coordinate system is the reference coordinate system, and a three-dimensional registration algorithm, such as the iterative nearest point algorithm, is used to register the static soft tissue surface. and The soft tissue surface contours segmented from the data are registered with high precision. This registration process utilizes anatomically stable and non-deformable areas of the face (such as the forehead and nasal root) as registration references to ensure registration accuracy. After registration, a preoperative baseline 3D model integrating hard tissue structure and static soft tissue surface is generated. ; Subsequently, the individual biomechanical map (i-BMP) is generated, a process that integrates discrete biomechanical property data acquired in the multimodal data acquisition and preprocessing steps. (i.e., a series of spatial coordinate points and their corresponding Young's modulus) Value), converted to the value defined in A continuous scalar field on a soft tissue mesh; first, through a calibration process, ... The spatial coordinates of the data points are transformed from their acquisition coordinate system (e.g., the ultrasonic probe coordinate system) to... In the reference coordinate system; Next, a spatial interpolation algorithm, such as Kriging interpolation, is used based on discrete coordinates mapped to the reference coordinate system. Data points, for Each vertex on the soft tissue mesh Estimate the corresponding Young's modulus value, and finally generate an individual biomechanical map. It is defined in Functions on soft tissue mesh vertices Its calculation method can be expressed as: ; in, yes Any vertex on a soft tissue mesh; yes The total number of discrete measurement points in the dataset; It is the first Young's modulus values ​​at each measurement point; It is based on the vertex With all measurement points The Kriging interpolation weights, calculated from the spatial autocorrelation between them, are... The atlas quantifies the soft tissue physical stiffness of individual patients in different facial regions; Furthermore, the muscle dynamic vector field (MFV) is decoupled, a process that handles dynamic soft tissue data acquired during multimodal data acquisition and preprocessing. The sequence, firstly, takes the starting frame of the 4D sequence, i.e., the relaxation state frame. ,and The static soft tissue surface was registered to ensure that the dynamic data was consistent with the baseline model in the initial state. Using 4D scan data in time series By maintaining the consistency of vertex topology (i.e., vertex IDs), the set of vertices in the relaxed state is extracted. (from) The vertex set of the target facial expression state (e.g., maximum smile) and the target facial expression state. (from) By calculating each corresponding vertex from arrive Spatial displacement generates muscle dynamic vector field For any vertex Its muscle dynamic vector The calculation is as follows: ; in, It is the vertex exist Spatial coordinates of the vertex set; It is the vertex exist The spatial coordinates of the vertex set, The vector field is stored, representing the individual patient's facial expression movement pattern, and is decoupled from the original preoperative morphology for use in subsequent dynamic reconstruction steps.

[0053] See attached document Figure 2 In this embodiment, after the three-dimensional model registration and individual feature field construction steps, the method continues to perform virtual surgery and conditional response AI static prediction steps. The purpose of this step is to predict and generate the postoperative static facial appearance based on the doctor's surgical plan and combined with the patient's individual biomechanical characteristics.

[0054] This step begins with virtual surgical planning. The operator (e.g., a physician) uses a visual interactive interface to view the preoperative baseline 3D model generated in the 3D model registration and individual feature field construction steps. On the included hard tissue model, virtual osteotomy operations (such as LeFort I osteotomy, bilateral mandibular sagittal splitting (BSSO)) are performed, and the operator moves the osteotomized bone blocks to the target planned position in the interface. The system captures this planning operation and quantizes it into a skeletal displacement vector field. ,Should The vector field defines the displacement of hard tissue as a boundary condition for soft tissue deformation; Subsequently, the system activates the AI ​​static prediction module, namely the Conditional Response AI (CR-AI) engine. The engine It was used to receive (but not limited to) two key inputs: the bone displacement vector field generated by the virtual surgical planning step. And the individual biomechanical atlas generated by the three-dimensional model registration and individual feature field construction steps. ; As an AI model Input of hard tissue boundary conditions, As an AI model In one embodiment, the AI ​​model takes soft tissue physical constraints as input. A graph neural network (GNN) architecture can be used, which is based on... Based on the soft tissue mesh topology, for each vertex in the mesh , The received input features include (but are not limited to): the vertex and Spatial relationships (e.g., displacement of the nearest bone point), and by The provided vertex Young's modulus at the location ; The engine The task is to solve the problem by Driven and subject to Postoperative static displacement of soft tissue under physical constraints The calculation process can be expressed as follows: ; This constraint mechanism is specifically manifested as follows: After training, its predictions Vector magnitude and input Value (i.e.) ) shows a negative correlation, that is, when When it is higher (indicating that the tissue is harder or tighter), even The resulting tensile force is relatively large, and the predicted force at this location is... The amplitude will also be suppressed accordingly; conversely, when At a lower level (indicating softer or looser tissue), The range is relatively large.

[0055] exist Engine outputs static displacement of soft tissue After (a vector field defined on all soft tissue vertices), the system applies it to the preoperative baseline 3D model. soft tissue vertex set superior.

[0056] Through this application operation, the system calculates and generates a postoperative static prediction model. and its corresponding vertex set The calculation formula is as follows: ; Should The model is a postoperative static facial appearance prediction result that has incorporated the individual biomechanical characteristics of the patient, which can be used for subsequent dynamic simulation steps or for doctors to conduct static assessments.

[0057] See attached document Figure 2 In this embodiment, after the virtual surgery and conditional response AI static prediction steps are completed, the dynamic expression reconstruction step based on i-BMP nonlinear modulation is continued. The purpose of this step is to build upon the postoperative static prediction model generated in the virtual surgery and conditional response AI static prediction steps, which already reflects the individual's physical characteristics. Based on this, the model was further simulated to demonstrate its dynamic response when performing facial expressions (e.g., smiling), which was constrained by both the patient’s original movement patterns and individual biomechanical characteristics. This dynamic facial expression reconstruction step is performed by a non-linear modulation engine; The nonlinear modulation engine is activated in the postoperative static prediction model. and its vertex set (Calculated and generated by virtual surgery and conditional response AI static prediction steps) Once available, it can be automatically triggered by the system or triggered by the operator issuing dynamic simulation commands through a visual interactive interface.

[0058] Once started, the nonlinear modulation engine is first used to load three key datasets required to perform dynamic simulation calculations, all of which were generated and stored in the aforementioned steps: First, load the postoperative static vertex set generated by the virtual surgery and conditional response AI static prediction steps. ,Should This step serves as the initial morphological basis for postoperative dynamic simulation, namely the postoperative relaxation state (time). ) surface; Second, load and store the muscle dynamics vector field that is decoupled from the 3D model registration and individual feature field construction steps. ,Should Vector field ( This represents the patient's original, unrestrained facial expression movement pattern; Third, load the individual biomechanical map generated and stored by the three-dimensional model registration and individual feature field construction steps. ,Should Define each vertex on the soft tissue mesh Young's modulus This map will serve as a physical constraint for... Perform nonlinear modulation; The engine loads the three datasets mentioned above. After that, the next calculation stage begins, which involves defining the quantization function of physical damping; The core of this step is defining a nonlinear modulation function. This function Its function is to map individual biomechanical data. The physical quantity defined in the standard (i.e., Young's modulus) This is converted into a scalar modulation factor, which will subsequently be used to scale the muscle dynamics vector field. The amplitude was used to simulate the damping effect of tissue physical properties on facial expression movements.

[0059] Nonlinear modulation function Defined as a property of Young's modulus The monotonically decreasing function ensures the Young's modulus of the organization. The higher the value (i.e., the stiffer or tighter the tissue), the lower the calculated modulation factor (i.e., the stronger the motion attenuation); conversely, The lower the value (i.e., the softer or looser the tissue), the higher the modulation factor (i.e., the weaker the motion decay).

[0060] In one specific implementation of this embodiment, the nonlinear modulation function It is defined using an exponential decay function, and the formula for calculating this function is as follows: ; in, It is an individual biomechanical map Provided soft tissue mesh vertices Young's modulus value at the location; It is the base of the natural logarithm; It is a non-negative attenuation coefficient. As a calibrable system parameter, it is used for control. The sensitivity of the value to motion decay. The value can be set based on prior clinical statistics or material mechanics experimental data.

[0061] According to this function definition, when the vertex of When the value approaches 0 (representing extremely loose tissue), The value approaches This indicates that no attenuation occurs to the motion vector when... When the value increases (indicating increased tissue stiffness), The value will decrease monotonically and approach 0, indicating an increase in the attenuation of the motion vector.

[0062] The nonlinear modulation function Once defined, the nonlinear modulation engine will call this function in subsequent steps to calculate the specific modulation factor for each vertex.

[0063] After the physical damping quantization step was completed and the nonlinear modulation function was defined, Afterward, the nonlinear modulation engine continues to perform the calculation steps of the modulated muscle dynamic vector field.

[0064] The purpose of this step is to decouple the 3D model registration from the individual feature field construction step, which represents the original motion pattern, from the muscle dynamic vector field. The individual biomechanical map representing the physical characteristics of the individual is generated during the registration with the 3D model and the construction of the individual feature field. To combine.

[0065] This calculation process is used in the postoperative static prediction model. On a soft-tissue mesh, it is executed vertex-by-vertex; For any vertex on the soft tissue mesh The nonlinear modulation engine performs the following operations: 1. From individual biomechanical maps In the middle, obtain the vertex. Corresponding Young's modulus Right now ; 2. Call the nonlinear modulation function defined in the quantization step of physical damping. ,by Using the input, calculate the vertex. Corresponding scalar modulation factor ; 3. Obtain the vertex from the data decoupled and stored from the 3D model registration and individual feature field construction steps. Corresponding primitive muscle dynamic vector ; 4. Perform a scalar-vector multiplication operation between the scalar modulation factor and the original muscle dynamic vector.

[0066] The output of this operation is the vertex. Modulated muscle dynamic vector The calculation formula is as follows: ; in: It is calculated and applied to the vertices. Modulated muscle dynamic vector; It is a nonlinear modulation function defined in the quantization step of physical damping; The vertices are obtained from i-BMP. Young's modulus at the location value; The vertices are obtained from the MFV decoupling steps. The original muscle dynamic vector at that location.

[0067] Through the All vertices on the soft tissue mesh By performing the above calculations, the system generates a complete, modulated muscle dynamics vector field. .

[0068] Should Each vector in the vector field The magnitude and direction of the amplitude have been affected by the vertex. Individual biomechanical characteristics ( Nonlinear modulation of ), for example, if the vertex lie in The marked high-hardness area, its The value will approach 0, causing it to The vector amplitude is greatly attenuated. The vector field will be applied as the final dynamic displacement to the postoperative static model in the next step.

[0069] Modulated muscle dynamics vector field After the calculations are complete, the nonlinear modulation engine continues to execute the postoperative dynamic model. The generation steps.

[0070] This step is the final stage of dynamic facial expression reconstruction. Its purpose is to apply the dynamic displacement, which has been nonlinearly modulated by the individual's biomechanical characteristics, to the postoperative static model output by the AI ​​static prediction module, thereby generating the final postoperative dynamic prediction model.

[0071] The nonlinear modulation engine modulates the post-muscle dynamic vector field. The modulated muscle dynamic vector field calculated in the calculation step Modulated muscle dynamic vector field The postoperative static vertex set calculated in the calculation step , This represents the soft tissue displacement resulting from surgical planning at a static level and has been incorporated into individual biomechanical atlases. constraint.

[0072] For any vertex on the soft tissue mesh The system will use this vertex exist Spatial coordinates of the vertex Corresponding modulated muscle dynamic vector Perform vector addition to calculate the vertex. Spatial coordinates of postoperative dynamic facial expressions (e.g., smiling).

[0073] The output of this operation is the postoperative dynamic prediction model. vertex set The calculation formula is as follows: ; in: It is the vertex Spatial coordinates in postoperative dynamic facial expression state; It is the vertex Spatial coordinates under postoperative static prediction state; In the modulated muscle dynamic vector field The vertex obtained by nonlinear modulation in the calculation step The muscle dynamic vector at that location.

[0074] Through the All vertices on the soft tissue mesh The above calculations are performed to generate a complete three-dimensional mesh model that reflects the dynamic facial expressions after surgery. ,Should The model is an integrated result of this method with both static displacement and dynamic motion physical constraints.

[0075] In generation Subsequently, the nonlinear modulation engine was further used in... and Between these frames, a series of intermediate frame models are generated using linear or nonlinear interpolation algorithms. These intermediate frame models together constitute a continuous postoperative dynamic facial expression animation, which is used by the operator for 4D dynamic evaluation on a visual interactive interface.

[0076] See attached document Figure 2 In this embodiment, the postoperative static prediction model is generated immediately after the dynamic facial expression reconstruction step based on i-BMP nonlinear modulation is completed. and postoperative dynamic prediction model Next, the simulation results visualization and iterative evaluation steps will be performed.

[0077] The purpose of this step is to provide operators (e.g., doctors) with a multi-dimensional visualization of the simulation results and to provide a closed-loop interactive mechanism for adjusting and optimizing surgical planning.

[0078] The visualization and interactive evaluation module is used to present simulation results on its display interface.

[0079] For static evaluation, this module is used to present, at least side-by-side, the preoperative baseline 3D model generated during the 3D model registration and individual feature field construction steps on the display interface. And the postoperative static prediction model generated in the virtual surgery and conditional response AI static prediction steps. This module allows operators to perform synchronous rotation, scaling, and translation operations on two 3D models, enabling them to compare and evaluate the symmetry, harmony, and morphological improvement of the postoperative static facial appearance from any viewpoint.

[0080] For dynamic assessment, this module is used to play at least two dynamic sequences side-by-side on the display interface: the first dynamic sequence is the preoperative real dynamic soft tissue data acquired during the multimodal data acquisition and preprocessing steps. Animation; the second dynamic sequence is the postoperative dynamic model. The postoperative simulated dynamic facial expression animation generated during the generation step demonstrates the model's transformation from a postoperative static vertex set. Transition to postoperative dynamic vertex set The interpolation process.

[0081] The operator can use this dynamic comparison to assess the naturalness, range of motion, symmetry, and presence of any abnormalities in the simulated facial expressions (such as a smile) after surgery. Physical constraints can lead to limited movement or stiff facial expressions.

[0082] This step further provides a closed-loop iterative optimization process. If the operator determines, based on the results of the above static or dynamic evaluation, that the current surgical plan has not achieved the expected results (e.g., insufficient improvement in static morphology, or unexpected facial expressions displayed in dynamic simulation), the operator can intervene through a visual interactive interface. The intervention involves returning to the virtual surgical planning step and adjusting the position, angle, or amount of bone fragment movement of the virtual osteotomy. After receiving the revised surgical plan, the system requantizes it into an updated skeletal displacement vector field. ; Subsequently, the system automatically used the updated version. and original As input, the AI ​​static prediction step is re-executed (to generate updated results). and ) and dynamic facial expression reconstruction steps (to generate updated ) and ).

[0083] The system will calculate the generated and updated Model and The simulation animation is then presented to the operator again. This process can be repeated until the operator is satisfied with both the static and dynamic simulation results and finally determines the surgical plan.

[0084] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. An orthognathic surgery precise simulation planning system based on AI image recognition, characterized in that, include: Data acquisition equipment is used to acquire patient data in multiple modalities; The data processing terminal is used to run the following modules: The multimodal data acquisition and registration module is used to receive and process the patient data of the multiple modalities and output the fused data; The individualized feature field construction module is used to extract individual biomechanical maps and muscle dynamic vector fields based on the fused data; The AI ​​static prediction module is used to receive the virtual surgical plan and use the individual biomechanical atlas to predict and generate the postoperative static facial appearance. The dynamic expression reconstruction module is used to simulate and generate a postoperative dynamic model based on the postoperative static facial appearance using the muscle dynamic vector field and the individual biomechanical atlas. The visualization and interactive evaluation module is used to present the postoperative static facial appearance and the postoperative dynamic model, and to receive adjustment instructions from the operator to iteratively optimize the virtual surgical plan.

2. The AI-based image recognition-based orthognathic surgery precision simulation planning system according to claim 1, characterized in that, The AI ​​static prediction module is specifically configured to use the individual biomechanical map as a physical constraint when utilizing the individual biomechanical map, and the physical constraint is manifested as follows: The predicted soft tissue displacement amplitude is negatively correlated with the corresponding Young's modulus value in the individual biomechanical map.

3. The AI-based image recognition-based orthognathic surgery precision simulation planning system according to claim 1, characterized in that, The dynamic expression reconstruction module includes a nonlinear modulation engine, which is used to nonlinearly modulate the muscle dynamic vector field using the individual biomechanical map.

4. The AI-based image recognition-based orthognathic surgery precision simulation planning system according to claim 3, characterized in that, The nonlinear modulation engine is used to quantify the physical damping effect of tissue physical properties on facial expression movements using a nonlinear modulation function that monotonically decreases with respect to Young's modulus.

5. The AI-based image recognition-based orthognathic surgery precision simulation planning system according to claim 4, characterized in that, The nonlinear modulation function is an exponential decay function.

6. The AI-based image recognition-based orthognathic surgery precision simulation planning system according to claim 4, characterized in that, The nonlinear modulation engine is specifically used for: The modulated muscle dynamics vector field is calculated by performing vertex-by-vertex multiplication with the modulation factor calculated by the nonlinear modulation function, and then the modulated muscle dynamics vector field is applied to the postoperative static facial appearance to generate the postoperative dynamic model.

7. The AI-based image recognition-based orthognathic surgery precision simulation planning system according to claim 1, characterized in that, The individualized feature field construction module is specifically used for extracting the individual biomechanical map as follows: Using a spatial interpolation algorithm, the corresponding Young's modulus value is calculated for each vertex on the soft tissue mesh based on the biomechanical property data contained in the patient data of the various modalities.

8. The AI-based image recognition-based orthognathic surgery precision simulation planning system according to claim 1, characterized in that, The individualized feature field construction module is specifically used for extracting the muscle dynamic vector field as follows: The muscle dynamic vector field is generated by extracting the vertex set of the relaxed state and the vertex set of the target facial expression state from the dynamic soft tissue data contained in the multimodal patient data, and by calculating the spatial displacement between the two.

9. The AI-based image recognition-based orthognathic surgery precision simulation planning system according to claim 1, characterized in that, The iterative optimization function of the visualization and interactive evaluation module is specifically used for: The system receives adjustment instructions from the operator for the virtual surgical plan and updates the bone displacement vector field in response to the adjustment instructions. The bone displacement vector field is a quantization result of the virtual surgical plan. The AI ​​static prediction module and the dynamic expression reconstruction module are triggered to re-perform calculations using the updated skeletal displacement vector field as input.

10. The AI-based image recognition-based orthognathic surgery precision simulation planning system according to claim 1, characterized in that, The multimodal patient data includes: Hard tissue data acquired by computed tomography (CT) or cone-beam computed tomography (CBCT); Static soft tissue surface acquired by a 3D optical scanner; Dynamic soft tissue data acquired by a 4D scanner; And biomechanical property data obtained by ultrasound elastography equipment.

Images