A method and system for controlling the instrument path in cleft lip and palate repair surgery

Abstract

The application relates to the technical field of medical instrument control, and provides a cleft lip and palate repair surgical instrument path control method and system, which comprises the following steps: acquiring operation signal information of a doctor's operation handle in a cleft lip and palate repair surgical instrument, and extracting operation characteristics; acquiring surgical field image information provided by an endoscope, and identifying relative relationship information between the instrument operated by the doctor and an anatomical structure; fusing the operation characteristics and the relative relationship information, performing doctor operation intention judgment, and obtaining operation intention judgment information; in the process of performing the doctor operation intention judgment, when it is identified that the operation signal information has an abnormal condition, the doctor operation intention judgment is preferentially performed according to the relative relationship information to obtain the operation intention judgment information; and the control mode of the instrument operated by the doctor is dynamically adjusted according to the operation intention judgment information. The application has the effects of improving the accuracy and safety of cleft lip and palate repair surgery and reducing the risk of accidental damage to fragile tissues.

Description

Technical Field

[0001] This invention relates to the technical field of medical device control, specifically to a method and system for controlling the path of surgical instruments in cleft lip and palate repair surgery. Background Technology

[0002] In modern cleft lip and palate repair surgery, an advanced instrument path control method is widely used to improve the precision and safety of the procedure. This method relies heavily on three-dimensional anatomical data acquired from the patient preoperatively using high-resolution imaging equipment. Based on this data, a precise digital model is constructed, generating a pre-defined surgical plan containing hundreds of discrete path point sequences. This plan aims to ensure that the instruments operate along the optimal and safest path, avoiding important blood vessels and nerve structures.

[0003] However, the distal end of the surgical instrument struggles between the surgeon's manual instructions and the system's automatic error correction commands, resulting in frequent micro-vibrations and lags. This unintended instrument dynamics not only significantly increase the surgeon's workload, but more seriously, in delicate suturing areas close to critical blood vessels and nerves, this unstable instrument behavior greatly increases the risk of accidental injury. The root of this problem lies in the lack of a mechanism in existing pathway control methods to dynamically and reliably update or arbitrate the effectiveness of pre-set pathways, enabling them to understand and accept intentional pathway adjustments made by the surgeon based on real-time clinical judgment, rather than viewing them all as systematic errors requiring correction.

[0004] To address the aforementioned issues, existing technologies urgently need improvement. Summary of the Invention

[0005] This application discloses a method and system for controlling the path of surgical instruments in cleft lip and palate repair surgery. It aims to solve the technical problems in existing methods for controlling the path of surgical instruments in cleft lip and palate repair surgery, such as misalignment of the actual movement trajectory of the instruments with the preset path due to preoperative registration deviation and intraoperative tissue deformation, as well as the inability of the system to distinguish between intentional adjustments and unintentional errors by the doctor, which cause instrument shaking and lag, thus increasing surgical risks.

[0006] The technical solution of this application is as follows:

[0007] In a first aspect, this application discloses a method for controlling the instrument path in cleft lip and palate repair surgery, including:

[0008] During cleft lip and palate repair surgery, the operation signal information of the surgeon's operating handle in the cleft lip and palate repair surgical instruments is obtained, and the operation features are extracted from the operation signal information.

[0009] Acquire surgical field images provided by the endoscope, and identify the relative relationship between the instruments manipulated by the surgeon and the anatomical structures from the surgical field images;

[0010] By integrating operational features and relative relationship information, the doctor's operational intent is determined to obtain operational intent determination information. During the process of determining the doctor's operational intent, when an abnormality is identified in the operational signal information, the doctor's operational intent is determined first based on the relative relationship information to obtain operational intent determination information.

[0011] Based on the operator's intent, the control mode of the instruments being operated by the doctor is dynamically adjusted.

[0012] This technical solution effectively integrates the doctor's operating signals from the operating handle with the surgical field image information. When the operating signal is abnormal, it prioritizes visual information to determine the intent, thereby dynamically adjusting the instrument control mode. This solves the problem that existing systems cannot distinguish between intentional adjustments and unintentional errors by doctors, and improves the accuracy and safety of surgical instrument path control.

[0013] Furthermore, the steps to determine the doctor's operational intent by integrating operational characteristics and relative relationship information, and obtaining operational intent determination information, include:

[0014] The reliability of surgical field image information is evaluated to obtain a reliability score;

[0015] Based on the reliability score, adjust the weight of operational characteristics and relative relationship information in the process of judging the doctor's operational intent;

[0016] Adjust the threshold for judging the surgeon's intention based on the current type of cleft lip and palate repair surgery and the anatomical region it is located.

[0017] Based on the adjusted judgment weights and judgment thresholds, the doctor's operational intent is judged to obtain operational intent judgment information;

[0018] When the operational characteristics and relative information are unreliable or conflicting, maintain the current control mode of the instrument operated by the physician.

[0019] This technical solution can dynamically adjust the judgment weights of operational features and relative relationship information based on the reliability of surgical field image information, and adjust the judgment thresholds in combination with surgical type and anatomical region, making the doctor's judgment of operational intent more accurate and flexible, avoiding erroneous judgments when information is unreliable or conflicting, thereby improving the robustness of the control system.

[0020] More specifically, in some implementation schemes, the doctor's operational intent is determined by integrating operational features and relative relationship information to obtain operational intent determination information. During the process of determining the doctor's operational intent, when an abnormality is identified in the operational signal information, the doctor's operational intent is determined primarily based on the relative relationship information. The steps to obtain the operational intent determination information include:

[0021] The quality of surgical field image information is monitored in real time, and the reliability is assessed based on the quality of the surgical field image information to obtain a reliability score;

[0022] When the reliability score is lower than the preset score threshold, the focus of judging the doctor's intention is shifted to the operation features, and a backup judgment rule set is activated. The backup judgment rule set analyzes the smoothness, duration, and directional consistency of the operation features. The judgment threshold of the operation features is adjusted in combination with the anatomical region and operation type of the current cleft lip and palate repair surgery.

[0023] Based on the adjusted judgment threshold, the doctor's operational intent is judged to obtain operational intent judgment information;

[0024] The reliability score for continuously restoring surgical field image information is given when the surgical field image information is below a preset scoring threshold.

[0025] When the reliability score recovers, the focus of judging the doctor's intention will be redistributed to the fusion judgment of the operation characteristics and relative relationship information.

[0026] This technical solution enables the timely shift of judgment focus to operational features when the quality of surgical field image information deteriorates, and the activation of backup judgment rule sets to ensure accurate judgment of the doctor's intentions even when visual information is limited. The judgment is then re-integrated after visual information is restored, thereby ensuring the continuity and safety of the surgical procedure.

[0027] Based on the above, this application further proposes that the steps for dynamically adjusting the control mode of the instrument operated by the doctor based on the information of the operator's intention include:

[0028] Obtain information on the contact force between the end of the instrument operated by the doctor and the tissue in the anatomical region where the surgery is currently taking place;

[0029] Based on tissue contact force information, assess the tissue mechanical properties of the anatomical region; tissue mechanical properties include tissue stiffness and tissue elasticity.

[0030] Based on the biomechanical properties of the tissue, the compliance parameters in the control mode of the device operated by the physician are dynamically adjusted; the compliance parameters include the proportional gain and integral gain used to control the stiffness of the device, as well as the virtual stiffness and virtual damping.

[0031] This technical solution enables real-time assessment of tissue mechanical properties based on the contact force information between the instrument tip and the tissue, and dynamic adjustment of compliance parameters in the instrument control mode. This allows the instrument's stiffness and damping to adapt to the characteristics of different tissues, thereby improving the precision and safety of operation and reducing the risk of damage to fragile tissues.

[0032] As an optional approach, the steps for dynamically adjusting the control mode of the instrument operated by the doctor based on the information of the operator's intention include:

[0033] Real-time analysis of the movement trend of the doctor's operating handle to determine the movement trajectory and target area of ​​the end effector of the instrument operated by the doctor within a preset future time period;

[0034] By combining the preoperative three-dimensional anatomical model information provided by the surgical navigation system, the type of anatomical region that the end of the instrument operated by the doctor is about to enter and the typical surgical operation type corresponding to the anatomical region can be identified.

[0035] Before entering the anatomical area, the compliance parameters in the instrument control mode operated by the doctor are pre-adjusted according to the tissue characteristics of the anatomical area and the requirements of the surgical operation.

[0036] This technical solution allows for the prediction of the movement trajectory and target area of ​​the instrument tip, combined with preoperative anatomical model information, to pre-adjust the compliance parameters in the instrument control mode. This enables the optimization of control strategies before the instrument enters a specific anatomical area, further enhancing the predictability and safety of the surgery.

[0037] To enhance functionality, the steps for real-time analysis of the movement trends of the doctor's operating handle to determine the trajectory and target area of ​​the instrument's end effector within a preset future timeframe include:

[0038] Acquire force sensor data from the doctor's operating handle;

[0039] Based on force sensor data, the mechanical interaction characteristics of the doctor's operating handle in contact with tissue are identified; the mechanical interaction characteristics include the rate of change of contact force and the trend of torque change.

[0040] By combining biomechanical interaction features and the movement trend of the doctor's operating handle, the movement trajectory and target area of ​​the end effector of the instrument operated by the doctor are predicted within a preset future time. When the biomechanical interaction features indicate that the instrument operated by the doctor is about to come into contact with the tissue, the prediction accuracy of the movement trajectory and target area is adjusted based on the biomechanical interaction features.

[0041] This technical solution enables the identification of mechanical interaction characteristics by analyzing force sensor data and predicting the motion trajectory and target area of ​​the instrument end effector by combining motion trends. In particular, it prioritizes the adjustment of prediction accuracy based on mechanical interaction characteristics just before contact with tissue, thereby improving the accuracy of prediction and providing a more reliable basis for the pre-adjustment of instrument control mode.

[0042] Building upon the above, this application further proposes that, when the biomechanical interaction characteristics indicate that the instrument being manipulated by the physician is about to come into contact with tissue, the steps for prioritizing the adjustment of the motion trajectory and the prediction accuracy of the target area based on the biomechanical interaction characteristics include:

[0043] Real-time monitoring of the volatility, signal-noise level, and deviation from historical mechanical characteristics of mechanical interaction features is used to assess the reliability of the mechanical interaction features.

[0044] When the reliability of the mechanical interaction features is lower than a preset threshold, the mechanical interaction features are weighted averaged or filtered in combination with the movement trend of the doctor's operating handle to reduce the impact of abnormal fluctuations and obtain the processed mechanical interaction features.

[0045] Based on the processed mechanical interaction characteristics, it is determined whether the instrument operated by the doctor is about to come into contact with the tissue, and the contact judgment result information is obtained;

[0046] Based on the contact judgment results, adjust the prediction accuracy of the motion trajectory and target area.

[0047] This technical solution enables real-time assessment of the reliability of mechanical interaction features and addresses any insufficient reliability, thereby ensuring the accuracy of contact judgment and further improving the accuracy of motion trajectory and target area prediction, thus enhancing the safety of the surgery.

[0048] Furthermore, when the biomechanical interaction characteristics indicate that the instrument being manipulated by the physician is about to make contact with the tissue, the steps for prioritizing the adjustment of the motion trajectory and the prediction accuracy of the target area based on the biomechanical interaction characteristics include:

[0049] Real-time acquisition of information on the contact force between the end of the instrument operated by the doctor and the tissue in the current anatomical area;

[0050] By analyzing the rate of change and frequency components of tissue contact force information over time, mechanical interaction characteristics are obtained.

[0051] By combining the information of marked tissue regions in the preoperative three-dimensional anatomical model provided by the surgical navigation system, and the spatial relative position of the end of the instrument operated by the doctor with the marked tissue region, it is determined whether the end of the instrument operated by the doctor is close to or enters the marked tissue region, thus obtaining spatial judgment features.

[0052] When the analysis of mechanical interaction features based on spatial judgment features indicates that the instrument operated by the doctor is about to come into contact with the marked vulnerable tissue, and the end of the instrument is close to or enters the unmarked potential vulnerable area in the preoperative three-dimensional anatomical model, the specific change pattern of the mechanical interaction features is matched with the preset vulnerable tissue mechanical response pattern to identify the type of unmarked potential vulnerable area.

[0053] Based on the type of unmarked potential vulnerable areas, the prediction accuracy of motion trajectory and target area is dynamically adjusted, and the virtual stiffness and virtual damping parameters of the instruments manipulated by the physician are updated.

[0054] This technical solution can identify unmarked potential vulnerable areas by integrating mechanical interaction features, spatial judgment features, and preoperative three-dimensional anatomical model information. Based on the type of vulnerability, the prediction accuracy and the virtual stiffness and virtual damping parameters of the instruments can be dynamically adjusted. This allows unknown risk areas to be discovered and avoided during surgery, greatly improving the safety of the operation.

[0055] Based on the above, this application further proposes that when the analysis of mechanical interaction features based on spatial judgment features indicates that the instrument manipulated by the doctor is about to come into contact with a marked vulnerable tissue, and the end of the instrument is close to or enters an unmarked potential vulnerable area in the preoperative three-dimensional anatomical model, the steps of matching specific change patterns of mechanical interaction features with preset vulnerable tissue mechanical response patterns to identify the type of unmarked potential vulnerable area include:

[0056] Real-time monitoring of the mechanical interaction characteristics of the end effector of the instrument operated by the doctor when it comes into contact with the tissue; the mechanical interaction characteristics include the rate of change of contact force over time and the frequency component.

[0057] By combining real-time surgical field image information, visual features of the tissue surrounding the end of the instruments operated by the surgeon can be identified.

[0058] By fusing mechanical interaction features with visual features, a multimodal interactive fingerprint is obtained.

[0059] Cross-compare the multimodal interaction fingerprint with the pre-defined multimodal response fingerprint of vulnerable tissues;

[0060] Based on the current surgical stage, the identified anatomical regions, and the doctor's historical operating habits, the weights of mechanical and visual features in the multimodal interactive fingerprint are dynamically adjusted, and the matching thresholds between the multimodal interactive fingerprint and the multimodal response fingerprints of several preset vulnerable tissues are adjusted to identify the types of unlabeled potential vulnerable regions.

[0061] This technical solution can form a multimodal interactive fingerprint by integrating mechanical and visual features, and compare it with a preset vulnerable tissue response fingerprint. At the same time, the weights and matching thresholds are dynamically adjusted to more accurately identify the types of unmarked potential vulnerable areas, providing more refined protection for surgical safety.

[0062] Secondly, this application also discloses a surgical instrument path control system for cleft lip and palate repair surgery, used to perform surgical instrument path control for cleft lip and palate repair surgery, including:

[0063] The operation feature extraction module is used to acquire the operation signal information of the surgeon's operating handle in the cleft lip and palate repair surgery, and extract operation features from the operation signal information.

[0064] The relationship information recognition module is used to acquire surgical field image information provided by the endoscope and to identify the relative relationship information between the instruments operated by the doctor and the anatomical structures from the surgical field image information.

[0065] The operation intent judgment module is used to integrate operation features and relative relationship information to judge the doctor's operation intent and obtain operation intent judgment information. When abnormalities are detected in the operation signal information during the doctor's operation intent judgment process, the operation intent judgment is performed based on the relative relationship information first to obtain operation intent judgment information.

[0066] The control mode adjustment module is used to dynamically adjust the control mode of the instruments operated by the doctor based on the information of the operation intention.

[0067] This technical solution provides a system for implementing the aforementioned surgical instrument path control method for cleft lip and palate repair surgery. Through modular design, it achieves the functions of operation feature extraction, relationship information recognition, operation intention judgment, and control mode adjustment, providing hardware and software support for the precise control of surgical instruments, thereby improving the overall efficiency and safety of the surgery.

[0068] Beneficial effects

[0069] This application discloses a surgical instrument path control method for cleft lip and palate repair surgery. By acquiring real-time operation signals from the surgeon's operating handle and surgical field images from the endoscope during the procedure, and extracting operation features and identifying the relative relationships between the instruments and anatomical structures, it achieves multimodal information fusion. When judging the surgeon's operational intent, this method intelligently integrates operation features and relative relationship information. Especially when there are abnormalities in the operation signal information, it prioritizes judgment based on the relative relationship information, effectively solving problems in existing technologies such as inaccurate instrument path control, instrument vibration, and lag caused by preoperative registration deviations, intraoperative tissue deformation, and the system's inability to distinguish between intentional adjustments and unintentional errors. By dynamically adjusting the control mode of the instruments operated by the surgeon based on the judged operational intent information, this application enables the actual movement trajectory of the instruments to better match the correct adjustments made by the surgeon based on real-time changes in the surgical field. This avoids the system misinterpreting the surgeon's intentional avoidance actions as system errors requiring correction, thus significantly improving the accuracy and safety of the surgery, reducing the risk of accidental damage to fragile tissues, alleviating the surgeon's workload, and ultimately improving the success rate of cleft lip and palate repair surgery and the patient's prognosis. Attached Figure Description

[0070] Figure 1 This is a flowchart of a surgical instrument path control method for cleft lip and palate repair according to one embodiment of the present invention;

[0071] Figure 2 This is one of the flowcharts of a surgical instrument path control method for cleft lip and palate repair surgery according to another embodiment of the present invention;

[0072] Figure 3 This is a second flowchart of a method for controlling the path of surgical instruments in cleft lip and palate repair surgery, as described in another embodiment of the present invention.

[0073] Figure 4 This is a system block diagram of a surgical instrument path control system for cleft lip and palate repair according to another embodiment of the present invention;

[0074] Explanation of reference numerals in the attached figures:

[0075] 1. Instrument path control system for cleft lip and palate repair surgery; 11. Operation feature extraction module; 12. Relationship information recognition module; 13. Operation intention judgment module; 14. Control mode adjustment module. Detailed Implementation

[0076] The technical solutions of this application will now be clearly and completely described with reference to the accompanying drawings. Obviously, the described embodiments are merely some embodiments of this application, and not all embodiments. The components of this application described and shown in the accompanying drawings can generally be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely to illustrate selected embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.

[0077] It should be noted that similar reference numerals and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures. Furthermore, in the description of this application, terms such as "first," "second," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0078] In cleft lip and palate repair surgery, traditional instrument path control methods rely heavily on preoperative planning. However, due to intraoperative registration deviations, tissue deformation, and the system's misjudgment of the doctor's manual adjustments, there are discrepancies between the actual movement of the instruments and the real anatomical structure, which can cause instrument vibration and lag, increasing surgical risks.

[0079] In response, this application proposes a method for controlling the instrument path in cleft lip and palate repair surgery, combined with... Figure 1 As shown, it includes:

[0080] S1, during the cleft lip and palate repair surgery, acquire the operation signal information of the doctor's operating handle in the cleft lip and palate repair surgical instrument, and extract the operation features from the operation signal information;

[0081] S2, acquire surgical field image information provided by the endoscope, and identify the relative relationship between the instruments operated by the doctor and the anatomical structures from the surgical field image information;

[0082] S3, integrates operation features and relative relationship information to determine the doctor's operation intention and obtain operation intention determination information; during the process of determining the doctor's operation intention, when an abnormality is identified in the operation signal information, the doctor's operation intention is determined first based on the relative relationship information to obtain operation intention determination information.

[0083] S4 dynamically adjusts the control mode of the instruments operated by the doctor based on the information obtained from the operator's intention.

[0084] The term "cleft lip and palate repair surgical instruments" as used in this application generally refers to precision surgical tools used in cleft lip and palate repair surgery, which may include, but are not limited to, robotic-assisted surgical instruments and endoscopic surgical instruments. These instruments are typically equipped with sensors that can collect "operational signal information" such as the movement and force feedback of the surgeon's operating handle in real time. "Operational characteristics" refer to key parameters extracted from these raw operation signal information that reflect the surgeon's intentions, such as the speed, acceleration, applied force, smoothness of the movement trajectory, duration of the operation, and directional consistency of the handle. An "endoscope" is a medical optical instrument used to provide surgical field images in minimally invasive surgery. It can transmit real-time images of the surgical area to a monitor, allowing the surgeon to clearly observe the interaction between the surgical instruments and the patient's anatomical structures. "Surgical field image information" refers to the real-time video stream or image sequence captured by the endoscope. "Information on the relative relationship between the instruments manipulated by the surgeon and the anatomical structures" refers to the spatial relationship, such as the position, distance, and angle, of the instrument tip relative to surrounding tissues, blood vessels, nerves, and other anatomical structures, identified from the surgical field image information through image processing and computer vision technology. "Physician's intention determination" refers to the system's comprehensive analysis of operational characteristics and relative information to infer the type of operation the physician currently desires the instrument to perform (e.g., cutting, suturing, traction, avoidance, etc.) and its expected direction and target of movement. "Intention determination information" is the result of this determination, used to guide subsequent instrument control. "Abnormal situations" may include, but are not limited to, loss of operating handle signal, excessive signal noise, or a significant discrepancy between the physician's actions and the expected model. "Control mode" refers to the way the instrument responds to the physician's operational commands; for example, it can be rigid control, compliant control, force feedback control, etc., and its parameters (such as stiffness, damping, compliance, etc.) can be dynamically adjusted according to surgical needs.

[0085] The core of the surgical instrument path control method for cleft lip and palate repair surgery proposed in this application lies in the accurate judgment of the surgeon's intention and the dynamic adjustment of the instrument control mode.

[0086] Specifically, during cleft lip and palate repair surgery, the first step is to acquire the operational signal information from the surgeon's operating handles in the surgical instruments and extract operational features from them. For example, multi-axis force sensors and encoders mounted on the surgeon's operating handles can be used to collect real-time data on the surgeon's three-dimensional force, torque, position, and posture. This raw data is then fed into a signal processing module, which uses algorithms such as low-pass filtering and Kalman filtering to remove noise and further calculates the speed, acceleration, curvature of the motion trajectory, magnitude and direction of the operating force, and duration of the operation. These are the extracted "operational features." As a preferred implementation, the extraction of operational features can also be achieved through machine learning models. For example, a recurrent neural network (RNN) or a long short-term memory network (LSTM) can be trained, taking the raw sensor data as input and outputting an operational feature vector after feature engineering. This vector can more abstractly represent the surgeon's operating habits and intentions.

[0087] Simultaneously, it is necessary to acquire surgical field images provided by the endoscope and identify the relative relationships between the instruments manipulated by the surgeon and the anatomical structures. For example, the endoscope can transmit high-definition video streams in real time. These video streams are fed into an image processing and recognition module. This module can use deep learning-based image segmentation algorithms (such as U-Net and Mask R-CNN) to identify and segment key anatomical structures in the images in real time, such as the ends of surgical instruments, target tissues (such as lip muscles and palatal bones), blood vessels, and nerves. Subsequently, by calculating the centroid, bounding box, and orientation vector of these segmented regions, the "relative relationship information" such as the distance, angle, and relative position between the instrument end and these anatomical structures can be quantified. For example, the Euclidean distance from the instrument tip to the nearest blood vessel, or the angle between the instrument axis and the normal to the tissue surface, can be calculated.

[0088] Furthermore, this application integrates operational features and relative relationship information to determine the doctor's operational intent, thereby obtaining operational intent determination information. During the process of determining the doctor's operational intent, when an anomaly is detected in the operational signal information, the system prioritizes determining the doctor's operational intent based on the relative relationship information, thus obtaining operational intent determination information. For example, extracted operational features (such as handle speed and force) and identified relative relationship information (such as the distance between the instrument and the blood vessel, and the contact state between the instrument and the tissue) can be used as input to a multimodal fusion model, such as an attention-based neural network model or a Bayesian network. This model, by learning from a large amount of doctor's operational data, can determine whether the doctor's current intent is to perform delicate suturing, tissue cutting, or avoid critical structures. Therefore, the obtained operational intent determination information may include "delicate suturing," "tissue resection," and "avoiding blood vessels." When an anomaly is detected in the operational signal information, such as a sudden loss of handle sensor data or severe vibration, the system will prioritize relying on the relative relationship information provided by the endoscope for judgment. For example, even if the handle signal is abnormal, if the image shows that the instrument tip is approaching the blood vessel at an extremely slow speed, the system can still determine that the doctor's intention is "precise avoidance" rather than "misoperation".

[0089] Finally, based on the operator's intent, the system dynamically adjusts the control mode of the instruments being operated by the physician. For example, if the intent indicates that the physician is performing "delicate suturing," the system can adjust the instrument's control mode to a high-precision, low-stiffness compliant mode to allow for more precise manipulation and provide stronger force feedback to prevent excessive force. If the intent is determined to be "tissue resection," the system can adjust to a high-stiffness, high-response mode to ensure efficient and stable cutting. This dynamic adjustment can include changing parameters such as the instrument's virtual stiffness, virtual damping, and force feedback gain to better match the instrument's response characteristics with the physician's current operational needs and the characteristics of the surgical area.

[0090] Optional, combined Figure 2 As shown, the steps in step S3, which involve fusing operational features and relative relationship information to determine the doctor's operational intent and obtain operational intent determination information, include:

[0091] S31, assess the reliability of the surgical field image information and obtain a reliability score;

[0092] S32, Based on the reliability score, adjust the judgment weight of operational characteristics and relative relationship information in the process of judging the doctor's operational intention;

[0093] S33, Adjust the threshold for judging the doctor's intention based on the current type of cleft lip and palate repair surgery and the anatomical area it is located.

[0094] S34, Based on the adjusted judgment weights and judgment thresholds, the doctor's operational intent is judged to obtain operational intent judgment information;

[0095] S35, when the operating characteristics and relative relationship information are both unreliable or conflicting, maintain the current control mode of the instrument operated by the physician.

[0096] Specifically, assessing the reliability of surgical field image information and obtaining a reliability score refers to real-time quality evaluation of the surgical field image information provided by the endoscope. This evaluation can be based on quantitative analysis of visual characteristics such as image sharpness, brightness, contrast, noise level, and the presence of occlusion or blurring, thereby generating a reliability score that reflects the trustworthiness of the image information. For example, image processing algorithms can be used to calculate the edge sharpness, signal-to-noise ratio, or texture clarity of a specific area of ​​the image and map them to a score range of 0 to 100, with higher scores indicating more reliable image information.

[0097] The adjustment of the weighting of operational features and relative relationship information in the doctor's operational intent judgment process based on the reliability score can be understood as dynamically allocating the influence of operational features and relative relationship information in the final operational intent judgment according to the quality of the surgical field image information. When the image information reliability score is high, the weight of relative relationship information (derived from the image) can be increased, allowing it to play a greater role in the judgment; conversely, when the image information reliability score is low, the weight of relative relationship information can be decreased, while the weight of operational features (derived from the doctor's operating handle) can be increased accordingly to compensate for the impact of insufficient image information. This weight adjustment ensures that the system can prioritize more reliable information sources under different information quality conditions.

[0098] In practical applications, the threshold for judging the surgeon's intention is adjusted based on the type of cleft lip and palate repair surgery and the anatomical region in which it is performed. Specifically, the system dynamically adjusts the threshold required to judge the surgeon's intention based on the specific stage of the surgery (e.g., incision, dissection, suturing) and the anatomical region where the instruments are located (e.g., hard palate, soft palate, lip muscles). For example, in delicate suturing operations, a lower threshold may be needed to identify subtle changes in instrument movement; while in large-scale dissection operations, a higher threshold may be needed to avoid misjudgment. This adjustment allows the system to better adapt to the specific requirements of different surgical scenarios.

[0099] Furthermore, when operational characteristics and relative information are unreliable or conflicting, maintaining the current control mode of the instrument operated by the physician serves as a safety mechanism. This means that if neither of the two primary information sources can provide reliable information, or if there are contradictory indications between them, the system will avoid making uncertain control mode adjustments and instead maintain the instrument's current stable state to prevent adverse consequences due to misjudgment. This can be understood as a "safety standstill" or "conservative mode," awaiting information recovery or further clarification from the physician.

[0100] In some preferred embodiments, a specific example is given below. Suppose that during a cleft lip and palate repair surgery, the surgeon is performing a delicate dissection of the soft palate tissue. At this time, due to tissue bleeding or water condensation, the image clarity of the endoscopic surgical field suddenly decreases. The system, through real-time evaluation, detects that the reliability score of the surgical field image information is below a preset threshold. Based on this, the system immediately reduces the weight of relative relationship information (derived from the image) in the surgeon's intention assessment and correspondingly increases the weight of the surgeon's operating characteristics of the handle (e.g., handle movement speed, direction, force feedback, etc.).

[0101] Specifically, the system adjusts the threshold for judging the doctor's intention based on the current type of "soft palate dissection" operation and the anatomical region of the soft palate. This makes the system more sensitive to the doctor's subtle, smooth dissection movements with the handle, while setting a higher threshold for rapid, large-amplitude non-dissection movements. Even when the image information is blurry, as the doctor continues the dissection operation, the system can still accurately determine the doctor's dissection intention primarily based on the smooth, continuous dissection movements of the handle.

[0102] Furthermore, if, at any moment, the surgical field image information is completely lost, and the surgeon's operating signal also experiences brief abnormal fluctuations (e.g., due to external interference), causing both the operational features and relative relationship information to be judged as unreliable or conflicting, the system will immediately trigger a safety mechanism. This mechanism maintains the current control mode of the instrument being operated by the surgeon, keeping its current position and stiffness settings unchanged until the information returns to normal or the surgeon provides explicit instructions through other means. This approach effectively avoids inappropriate instrument adjustments due to misjudgment in situations of high information uncertainty, thus ensuring surgical safety. Once the image information becomes clear and the reliability score recovers, the system will refocus its judgment on the fusion of operational features and relative relationship information, restoring its adaptive judgment capability.

[0103] Optional, combined Figure 3As shown, S3 integrates operational features and relative relationship information to determine the doctor's operational intent, obtaining operational intent determination information. During the process of determining the doctor's operational intent, when an abnormality is detected in the operational signal information, the relative relationship information is used first to determine the doctor's operational intent. The steps to obtain the operational intent determination information include:

[0104] A1, real-time monitoring of the quality of surgical field image information, and assessment of reliability based on the quality of surgical field image information to obtain a reliability score;

[0105] A2. When the reliability score is lower than the preset score threshold, the focus of judging the doctor's intention is shifted to the operation features, and the backup judgment rule set is activated. The backup judgment rule set analyzes the smoothness, duration, and directional consistency of the operation features. Combined with the anatomical region and operation type of the current cleft lip and palate repair surgery, the judgment threshold of the operation features is adjusted.

[0106] A3. Based on the adjusted judgment threshold, the doctor's operational intent is judged to obtain operational intent judgment information;

[0107] A4, a reliability score for continuously restoring surgical field image information when the surgical field image information is below a preset scoring threshold;

[0108] A5, when the reliability score recovers, the focus of judging the doctor's operational intent is redistributed to the fusion judgment of operational characteristics and relative relationship information.

[0109] Specifically, real-time monitoring of the quality of surgical field image information refers to the system continuously acquiring the image stream provided by the endoscope and quantitatively analyzing visual indicators such as image sharpness, brightness, contrast, noise level, blurriness, and the presence of occlusion or reflection. For example, image processing algorithms (such as Fourier transform, edge detection, signal-to-noise ratio calculation, etc.) can be used to evaluate image quality and convert it into a reliability score. This score can be a value between 0 and 1, where 1 indicates excellent image quality and 0 indicates completely unusable image. A preset scoring threshold can be set according to the actual surgical environment and the required accuracy; for example, it can be set to 0.6, where an image is considered unreliable when the score is below this value.

[0110] When the reliability score falls below a preset threshold, the focus of determining the doctor's intention shifts to the operational features. This means the system reduces the weight of surgical field image information in determining the doctor's intention and correspondingly increases the weight of operational features extracted from the doctor's operating handle signals. For example, in the weighting model, the weight of image features can be dynamically adjusted to 0.2, while the weight of operational features can be increased to 0.8. Simultaneously, a backup set of rules is activated. This set is designed for operational features and is used to infer the doctor's intention by analyzing the inherent patterns of operational features when visual information is limited. Specifically, the backup set of rules analyzes the smoothness of operational features (e.g., the smoothness of the movement trajectory, avoiding sudden, unnatural jitter), duration (e.g., the length of time a certain action lasts, distinguishing between brief adjustments and continuous operations), and directional consistency (e.g., the stability of the instrument's movement direction, determining whether it is linear or rotational movement). These features reflect the stability and purposefulness of the doctor's operation. By considering the anatomical region of the cleft lip and palate repair surgery (e.g., the soft palate, hard palate, or alveolar bone region) and the type of operation (e.g., cutting, suturing, dissection, or hemostasis), the judgment threshold for operation characteristics can be adjusted. For example, in areas requiring fine suturing, a higher requirement for smoothness of operation is placed, and the judgment threshold will be more stringent; while in areas requiring coarse dissection, the requirement for smoothness may be relatively lenient.

[0111] While the surgical field image information falls below a preset scoring threshold, the system continuously attempts to restore the reliability score of the surgical field image information. This can be achieved in several ways; for example, if the image blurriness is due to endoscope lens contamination, the system can trigger the endoscope cleaning function; if the brightness is insufficient, the light source intensity can be adjusted; if there is obstruction, the system can prompt the doctor to adjust the endoscope position. The system continuously monitors the effectiveness of these restoration measures and updates the reliability score in real time. When the reliability score recovers above the preset threshold, the system will refocus its judgment on the doctor's operational intent on a fusion judgment of operational features and relative relationship information, that is, it will revert to a conventional judgment mode based on multimodal information fusion to make full use of all available information.

[0112] In some preferred embodiments, a specific example is given below. Suppose that during a cleft lip and palate repair surgery, the surgeon is performing a delicate soft palate tissue dissection. Due to the surgeon's prolonged operation causing slight hand tremors, the operating signal information of the surgeon's manipulator fluctuates abnormally. At this time, the system will preferentially rely on the surgical field image information provided by the endoscope to determine the surgeon's intention. However, during the dissection process, a small amount of bleeding or moisture may cause the endoscopic field of view to gradually blur, causing the reliability score of the surgical field image information to gradually decrease from 0.9 to 0.5, which is lower than the preset score threshold of 0.6.

[0113] At this point, the proposed solution immediately shifts the focus of determining the doctor's intention to the operational characteristics. The system activates a backup set of rules and begins to analyze the smoothness of the doctor's manipulator's trajectory, the duration of the instrument's movement in a specific direction, and its directional consistency. For example, despite slight tremors, if the overall trend of the instrument's movement is smooth and continuous in a certain direction, and it is in the soft palate region, this usually indicates that the doctor is performing a dissection procedure. The system adjusts the judgment threshold of the operational characteristics based on the current location in the soft palate region and the type of dissection procedure; for example, allowing a certain degree of smoothness fluctuation but requiring high directional consistency.

[0114] Simultaneously, the system continuously attempts to restore the reliability score of the surgical field image information. For example, it can automatically activate the endoscope's defogging function or prompt the surgeon to wipe the endoscope. As the defogging function takes effect, image quality gradually recovers, and the reliability score rises back to 0.8. Once the score recovers, the system immediately reassigns the focus of the surgeon's intention assessment to a fusion of operational features and relative information, reverting to a mode that utilizes both visual and operational signals for judgment, thereby obtaining a more comprehensive and accurate intention assessment result. Through this dynamic adaptation and recovery mechanism, the system can continuously provide stable and reliable instrument path control even in complex and changing surgical environments.

[0115] Optionally, the steps of dynamically adjusting the control mode of the instrument operated by the doctor based on the information of the operator's intention include:

[0116] Obtain information on the contact force between the end of the instrument operated by the doctor and the tissue in the anatomical region where the surgery is currently taking place;

[0117] Based on tissue contact force information, assess the tissue mechanical properties of the anatomical region; tissue mechanical properties include tissue stiffness and tissue elasticity.

[0118] Based on the biomechanical properties of the tissue, the compliance parameters in the control mode of the device operated by the physician are dynamically adjusted; the compliance parameters include the proportional gain and integral gain used to control the stiffness of the device, as well as the virtual stiffness and virtual damping.

[0119] Specifically, acquiring information about the tissue contact force between the end effector of the surgeon's instrument and the anatomical region being operated on involves integrating a high-precision force sensor at the instrument's end effector or employing vision-based force estimation methods to acquire real-time three-dimensional force vector data during instrument-tissue contact. This contact force information reflects the intensity and direction of the physical interaction between the instrument and the tissue. Assessing the tissue mechanics characteristics of the anatomical region based on this contact force information can be understood as analyzing the variation of contact force with instrument displacement or deformation, combined with a pre-defined tissue mechanics model, to calculate parameters such as tissue stiffness and elasticity. For example, the viscoelasticity of the tissue can be inferred by measuring the force response under known displacement or analyzing the frequency components and attenuation characteristics of the force signal. Tissue stiffness reflects the tissue's resistance to deformation, while tissue elasticity describes the tissue's ability to recover its original shape after being deformed by force. In practical applications, dynamically adjusting the compliance parameter in the surgeon's instrument control mode based on tissue mechanics characteristics aims to match the instrument's response characteristics with the characteristics of the currently operated tissue. The compliance parameter is a key parameter in robot control used to define the degree of system response to external forces or displacements. Specifically, proportional gain and integral gain are the core parameters for controlling the stiffness of a control device, affecting the response speed and steady-state accuracy of force control. The proportional gain determines the proportional relationship between force error and control output, while the integral gain is used to eliminate steady-state error. Virtual stiffness and virtual damping are concepts in virtual force field control. By simulating physical characteristics through software, the device exhibits specific stiffness and damping characteristics when interacting with the environment. For example, when virtual stiffness is high, the device behaves more "stiffly" and has greater resistance to displacement; when virtual damping is high, the device's movement is smoother and has greater resistance to changes in velocity.

[0120] In some preferred embodiments, a specific example is given below. Suppose that during cleft lip and palate repair surgery, the end of the instrument manipulated by the surgeon first needs to perform delicate suturing of the soft palate tissue, and then needs to dissect the periosteum of the hard palate.

[0121] When the instrument tip contacts the soft palate tissue, the system uses a force sensor to obtain information about the low tissue contact force and analyzes that the soft palate tissue has low stiffness and high elasticity. Based on these tissue mechanics characteristics, the system dynamically adjusts the compliance parameters in the instrument control mode, for example, reducing virtual stiffness and increasing virtual damping. As a result, the instrument exhibits a more "soft" and "compliant" characteristic in the soft palate region, allowing the surgeon to experience less resistance during delicate suturing, avoiding unnecessary damage to the fragile soft palate tissue, and maintaining operational smoothness.

[0122] Subsequently, as the instrument tip moves to the periosteum region of the hard palate, the system acquires higher tissue contact force information and assesses that the hard palate periosteum has high rigidity and low elasticity. At this point, the system adjusts compliance parameters accordingly, for example, increasing virtual stiffness and decreasing virtual damping. As a result, the instrument exhibits a more "rigid" and "direct" characteristic in the hard palate periosteum region, allowing the surgeon to apply more effective force during periosteal dissection, ensuring efficiency and thoroughness of the procedure, while preventing the instrument from slipping on hard surfaces or generating unnecessary vibrations.

[0123] Optionally, the steps of dynamically adjusting the control mode of the instrument operated by the doctor based on the information of the operator's intention include:

[0124] Real-time analysis of the movement trend of the doctor's operating handle to determine the movement trajectory and target area of ​​the end effector of the instrument operated by the doctor within a preset future time period;

[0125] By combining the preoperative three-dimensional anatomical model information provided by the surgical navigation system, the type of anatomical region that the end of the instrument operated by the doctor is about to enter and the typical surgical operation type corresponding to the anatomical region can be identified.

[0126] Before entering the anatomical area, the compliance parameters in the instrument control mode operated by the doctor are pre-adjusted according to the tissue characteristics of the anatomical area and the requirements of the surgical operation.

[0127] Specifically, real-time analysis of the movement trend of the doctor's operating handle refers to collecting kinematic data such as the position, velocity, and acceleration of the operating handle, and combining this data with force sensor data (e.g., torque, contact force), using motion prediction algorithms (e.g., based on Kalman filtering, neural networks, or Gaussian process regression) to model the doctor's operating intentions. This allows for the prediction of the possible movement path and final target area of ​​the end effector of the instrument being operated by the doctor within a future time period (e.g., a preset future duration of 0.1 to 1 second). This target area can be a specific point, line, or surface within the surgical area.

[0128] In this system, the surgical navigation system, by combining preoperative 3D anatomical model information, identifies the type of anatomical region that the surgeon's instrument tip will enter and the corresponding typical surgical operation type. This can be understood as follows: the surgical navigation system typically includes a 3D anatomical model constructed from the patient's preoperative CT, MRI, and other imaging data. This model can pre-mark important anatomical structures (such as blood vessels, nerves, muscles, bones, and mucous membranes) and their fragility or mechanical properties. By registering and comparing the real-time position of the surgeon's instrument tip in the surgical space with this 3D anatomical model, the system can identify the specific type of anatomical region that the instrument tip will contact or enter (e.g., muscle tissue, bone tissue, or mucous membrane tissue). Based on the preset attributes of this region or the typical surgical operation type defined by the surgeon in the preoperative plan (e.g., cutting, suturing, dissection, or hemostasis), the system provides a basis for subsequent control mode adjustments.

[0129] In practical applications, before entering an anatomical region, the compliance parameters in the instrument control mode are pre-adjusted based on the tissue characteristics of the region and the requirements of the surgical procedure. The aim is to ensure the instrument is adaptable before contacting the target area. Tissue characteristics can include mechanical properties such as tissue stiffness, elasticity, toughness, and brittleness, while surgical procedure requirements may involve specific limitations on operating force, speed, and precision. Compliance parameters are key parameters in robot control used to describe the compliance of the instrument's interaction with the environment. They typically include proportional and integral gains for controlling instrument stiffness, as well as virtual stiffness and virtual damping. For example, when the instrument is about to enter a fragile mucosal tissue area, virtual stiffness and virtual damping can be pre-reduced to make the instrument more "soft" and "compliant," allowing for gentler action on the tissue during actual contact and avoiding damage. Conversely, when the instrument is about to act on bone tissue, virtual stiffness can be appropriately increased to provide a more stable operating platform.

[0130] Optionally, the steps of analyzing the movement trend of the doctor's operating handle in real time to determine the movement trajectory and target area of ​​the instrument's end effector within a preset future time period include:

[0131] Acquire force sensor data from the doctor's operating handle;

[0132] Based on force sensor data, the mechanical interaction characteristics of the doctor's operating handle in contact with tissue are identified; the mechanical interaction characteristics include the rate of change of contact force and the trend of torque change.

[0133] By combining biomechanical interaction features and the movement trend of the doctor's operating handle, the movement trajectory and target area of ​​the end effector of the instrument operated by the doctor are predicted within a preset future time. When the biomechanical interaction features indicate that the instrument operated by the doctor is about to come into contact with the tissue, the prediction accuracy of the movement trajectory and target area is adjusted based on the biomechanical interaction features.

[0134] Acquiring force sensor data from the doctor's operating handle refers to the real-time collection of force and torque information applied by the doctor during operation using force sensors integrated into the handle. These force sensors can be piezoelectric sensors, strain gauge sensors, or optical force sensors, etc., and their purpose is to quantify the physical interaction between the doctor and the instrument. Identifying the mechanical interaction characteristics when the doctor's operating handle contacts the tissue refers to analyzing and extracting features that characterize the physical contact state between the instrument tip and the tissue based on the acquired force sensor data. These mechanical interaction characteristics specifically include the rate of change of contact force over time and the trend of torque over time. The rate of change of contact force can reflect the suddenness or smoothness of the contact between the instrument tip and the tissue, while the trend of torque change can indicate the resistance distribution and direction when the instrument tip moves inside or on the surface of the tissue. Combining the mechanical interaction characteristics and the movement trend of the doctor's operating handle, predicting the movement trajectory and target area of ​​the instrument tip within a preset future time period refers to fusing and analyzing the mechanical interaction characteristics with the kinematic data (such as position, velocity, and acceleration) of the doctor's operating handle to more comprehensively predict the movement path and the final anatomical area reached by the instrument tip in a short period of time (preset future time). The preset future duration can be set according to the type of surgery and the precision of the operation, for example, from 0.1 seconds to 1 second. When the biomechanical interaction features indicate that the instrument being manipulated by the surgeon is about to make contact with the tissue, the prediction accuracy of the motion trajectory and target area is adjusted based on the biomechanical interaction features. This means that at the critical moment when the instrument tip is approaching or has already begun to make contact with the tissue, the system will give higher weight to the biomechanical interaction features, and even use them as the main basis to correct or refine the prediction results. This is because, at the time of contact, mechanical information often reflects the actual interaction state between the instrument and the tissue more accurately than pure kinematic information, thus providing a more accurate prediction.

[0135] In some preferred embodiments, a specific example is given below. Suppose a surgeon is performing soft palate dissection during cleft lip and palate repair surgery, and the instrument tip is gradually approaching the critical muscle tissue of the velopharyngeal closure ring. In conventional prediction methods, the system might predict the instrument's trajectory solely based on the speed and direction of the surgeon's manipulator. However, as the instrument tip approaches the muscle tissue, force sensors on the surgeon's manipulator capture subtle changes in contact force and torque in real time. For example, when the instrument tip slightly touches the tissue, force sensor data shows that the contact force begins to increase, possibly accompanied by a specific torque change trend, indicating that the instrument is attempting to penetrate or separate the tissue. At this point, the solution of this application identifies these mechanical interaction characteristics and determines that the instrument is about to contact the tissue. Based on this determination, the system prioritizes using these mechanical interaction characteristics to adjust the instrument tip's trajectory and the prediction accuracy of the target area in the near future. For example, if the mechanical characteristics indicate that the instrument is contacting the tissue with excessive force, the system immediately corrects the predicted trajectory, making it more inclined towards smooth gliding rather than puncture, and may narrow the predicted range of the target area to avoid accidental injury. This priority prediction mechanism based on mechanical interaction characteristics enables the instrument path control to adapt more precisely and in real time to the actual tissue interaction, thereby ensuring the accuracy and safety of surgical procedures.

[0136] Optionally, when the biomechanical interaction characteristics indicate that the instrument being manipulated by the physician is about to contact the tissue, the steps of prioritizing the adjustment of the motion trajectory and the prediction accuracy of the target area based on the biomechanical interaction characteristics include:

[0137] Real-time monitoring of the volatility, signal-noise level, and deviation from historical mechanical characteristics of mechanical interaction features is used to assess the reliability of the mechanical interaction features.

[0138] When the reliability of the mechanical interaction features is lower than a preset threshold, the mechanical interaction features are weighted averaged or filtered in combination with the movement trend of the doctor's operating handle to reduce the impact of abnormal fluctuations and obtain the processed mechanical interaction features.

[0139] Based on the processed mechanical interaction characteristics, it is determined whether the instrument operated by the doctor is about to come into contact with the tissue, and the contact judgment result information is obtained;

[0140] Based on the contact judgment results, adjust the prediction accuracy of the motion trajectory and target area.

[0141] Specifically, real-time monitoring of the volatility, signal-noise level, and deviation from historical mechanical characteristics aims to dynamically assess the quality of these characteristics. Volatility refers to the magnitude of change in the mechanical interaction characteristics over a short period; excessive volatility may indicate data instability. Signal-noise levels can be quantified through frequency domain analysis or statistical methods; high noise levels reduce data usability. Deviation from historical mechanical characteristics reflects whether the current data conforms to the expected pattern; abnormal deviations may indicate data anomalies. Through comprehensive analysis of these indicators, a reliability score for the mechanical interaction characteristics can be obtained.

[0142] When the reliability of the mechanical interaction features falls below a preset threshold, it indicates that the original mechanical data may contain interference or anomalies. In this case, by combining the movement trend of the doctor's operating handle, the mechanical interaction features are weighted and averaged or filtered. The aim is to remove or reduce the influence of noise and abnormal fluctuations, thereby obtaining more stable and reliable processed mechanical interaction features. Weighted averaging can assign different weights based on the reliability of the data, while filtering can employ methods such as low-pass filtering and median filtering to smooth the data.

[0143] In practical applications, based on the processed mechanical interaction characteristics, it is possible to more accurately determine whether the instrument being manipulated by the doctor is about to come into contact with the tissue. For example, when the rate of change of the processed contact force or the trend of the change of the torque reaches a specific threshold, it can be determined that contact is imminent. The contact judgment information obtained from this will serve as the basis for adjusting the accuracy of motion trajectory and target area prediction. For example, when it is determined that contact is imminent, the prediction accuracy can be improved to plan the instrument path more precisely.

[0144] Optionally, when the biomechanical interaction characteristics indicate that the instrument being manipulated by the physician is about to contact the tissue, the steps of prioritizing the adjustment of the motion trajectory and the prediction accuracy of the target area based on the biomechanical interaction characteristics include:

[0145] Real-time acquisition of information on the contact force between the end of the instrument operated by the doctor and the tissue in the current anatomical area;

[0146] By analyzing the rate of change and frequency components of tissue contact force information over time, mechanical interaction characteristics are obtained.

[0147] By combining the information of marked tissue regions in the preoperative three-dimensional anatomical model provided by the surgical navigation system, and the spatial relative position of the end of the instrument operated by the doctor with the marked tissue region, it is determined whether the end of the instrument operated by the doctor is close to or enters the marked tissue region, thus obtaining spatial judgment features.

[0148] When the analysis of mechanical interaction features based on spatial judgment features indicates that the instrument operated by the doctor is about to come into contact with the marked vulnerable tissue, and the end of the instrument is close to or enters the unmarked potential vulnerable area in the preoperative three-dimensional anatomical model, the specific change pattern of the mechanical interaction features is matched with the preset vulnerable tissue mechanical response pattern to identify the type of unmarked potential vulnerable area.

[0149] Based on the type of unmarked potential vulnerable areas, the prediction accuracy of motion trajectory and target area is dynamically adjusted, and the virtual stiffness and virtual damping parameters of the instruments manipulated by the physician are updated.

[0150] Specifically, real-time acquisition of tissue contact force information between the end of the instrument manipulated by the doctor and the current anatomical area can be achieved by integrating a miniature force sensor or force / torque sensor array at the end of the instrument. These sensors can measure three-dimensional force or torque data when the instrument contacts the tissue with high precision. The analysis of the rate of change and frequency components of tissue contact force information over time can employ signal processing techniques, such as Fourier transform, wavelet analysis, or time-frequency analysis, to extract key features reflecting tissue mechanical properties, such as the instantaneous rate of change of contact force and vibration frequency, thereby obtaining the mechanical interaction characteristics.

[0151] Furthermore, by combining the information of the marked tissue regions in the preoperative three-dimensional anatomical model provided by the surgical navigation system, and the spatial relative relationship between the current position of the end of the instrument operated by the doctor and the marked tissue regions, image registration technology can be used to align the real-time instrument position with the preoperative model. The distance and relative orientation between the end of the instrument and the preset vulnerable tissue regions (such as blood vessels, nerve bundles, and thin membranes) in the model can be calculated through spatial geometry algorithms. This allows for the determination of whether the instrument is close to or has entered these marked vulnerable regions, and the generation of spatial judgment features.

[0152] Specifically, when the analysis of mechanical interaction features based on the aforementioned spatial judgment characteristics indicates that the instrument manipulated by the physician is about to come into contact with a marked vulnerable tissue, and the instrument's tip is approaching or entering an unmarked potential vulnerable area in the preoperative 3D anatomical model, the system will activate an advanced recognition mechanism. Specifically, the specific change patterns of mechanical interaction features refer to the unique curves or signal characteristics of how parameters such as contact force and torque change over time during instrument-tissue contact. These patterns can be matched with preset vulnerable tissue mechanical response patterns, which are established based on extensive clinical data and biomechanical experiments and include typical mechanical response characteristics exhibited by different types of vulnerable tissues (such as blood vessels, nerves, and glands) when exposed to instrument contact. The matching process can employ machine learning algorithms, such as support vector machines, neural networks, or pattern recognition algorithms, to identify the type of unmarked potential vulnerable area.

[0153] Therefore, based on the type of unmarked potentially vulnerable area identified, the system can dynamically adjust the trajectory of the instrument manipulated by the physician and the prediction accuracy of the target area. For example, if a vascular area is identified, the prediction accuracy may be significantly improved, while the virtual stiffness of the instrument is reduced and the virtual damping is increased to ensure a gentler operation and avoid vascular rupture. The updating of virtual stiffness and virtual damping parameters aims to change the force feedback characteristics of the instrument, allowing the physician to experience different levels of resistance or compliance during operation, thereby guiding the physician to operate in a safer and more precise manner.

[0154] Optionally, when the analysis of mechanical interaction features based on spatial judgment features indicates that the instrument manipulated by the physician is about to come into contact with a marked vulnerable tissue, and the instrument tip is close to or enters an unmarked potential vulnerable area in the preoperative three-dimensional anatomical model, the step of matching specific change patterns of mechanical interaction features with preset vulnerable tissue mechanical response patterns to identify the type of unmarked potential vulnerable area includes:

[0155] Real-time monitoring of the mechanical interaction characteristics of the end effector of the instrument operated by the doctor when it comes into contact with the tissue; the mechanical interaction characteristics include the rate of change of contact force over time and the frequency component.

[0156] Specifically, mechanical interaction characteristics refer to the mechanical signals generated when the instrument tip comes into contact with tissue, such as those acquired by force sensors mounted on the instrument tip or the doctor's operating handle. The rate of change of contact force over time can reflect the instantaneous response speed of tissue to the force applied by the instrument. For example, when the instrument contacts harder tissue, the contact force may rise rapidly; while when it contacts softer or easily torn tissue, the increase in contact force may be slower or accompanied by a sudden drop. Frequency components can reveal the characteristics of tissue vibration or instrument-tissue friction. For example, different tissue types may produce different vibration frequency responses when subjected to instrument action. Monitoring these mechanical characteristics can be achieved through high-speed sampling and signal processing techniques (such as Fourier transform).

[0157] By combining the real-time surgical field image information mentioned above, the visual characteristics of the tissue surrounding the end of the instrument being operated by the surgeon can be identified.

[0158] The surgical field of view is provided in real time by the endoscope, and visual features may include, but are not limited to, tissue color, texture, gloss, blood vessel distribution, surface smoothness, and deformation under instrument action. For example, fragile tissue may exhibit characteristics such as a redder color, finer texture, denser blood vessels, or significant deformation under slight touch. These visual features can be extracted and identified using image processing algorithms (such as edge detection, texture analysis, and color segmentation) or deep learning-based image recognition models.

[0159] By fusing mechanical interaction features with visual features, a multimodal interactive fingerprint is obtained.

[0160] Specifically, multimodal interactive fingerprints are a comprehensive feature representation formed by integrating data from different sensors (force sensors and endoscopes). Fusion methods can include feature-level fusion (concatenating extracted mechanical and visual feature vectors into a longer vector), decision-level fusion (performing preliminary judgments on mechanical and visual features separately, then weighting or logically combining the judgment results), or deep learning fusion (using multi-input neural networks to directly process raw or pre-processed multimodal data). This fusion aims to leverage the complementarity of different modal information to overcome the limitations of a single modality, thereby providing more comprehensive and robust organizational interaction information.

[0161] Cross-compare the multimodal interaction fingerprint with the pre-defined multimodal response fingerprint of vulnerable tissues;

[0162] The pre-defined multimodal response fingerprint of vulnerable tissues is established in advance through extensive experimental data or clinical experience, encompassing typical combinations of mechanical and visual characteristics exhibited by different types of vulnerable tissues when interacting with the device. Cross-matching can be achieved through various pattern recognition or machine learning algorithms, such as support vector machines (SVM), neural networks, decision trees, or distance-based similarity matching algorithms, to determine which pre-defined vulnerable tissue type best matches the current multimodal interaction fingerprint.

[0163] Based on the current surgical stage, the identified anatomical regions, and the doctor's historical operating habits, the weights of mechanical and visual features in the multimodal interactive fingerprint are dynamically adjusted, and the matching thresholds between the multimodal interactive fingerprint and the multimodal response fingerprints of several preset vulnerable tissues are adjusted to identify the types of unlabeled potential vulnerable regions.

[0164] Specifically, at different surgical stages (e.g., incision, dissection, suturing), surgeons may prioritize different aspects of tissue mechanics and visual perception, thus allowing for dynamic adjustment of the weights of these two features. For instance, visual features may be more important during delicate dissection, while mechanical features may be more crucial when assessing tissue toughness. Identified anatomical regions (e.g., soft palate, hard palate, lip muscles) possess different tissue characteristics, resulting in varying mechanical and visual response patterns. The system can adjust weights and matching thresholds based on the current region. Surgeon's historical operating habits refer to the system learning and recording typical mechanical and visual interaction patterns of a specific surgeon when handling different tissues, enabling personalized adjustments to ensure the recognition results better align with the surgeon's style and preferences. This dynamic adjustment mechanism makes the recognition process more adaptive and accurate.

[0165] This application also discloses a surgical instrument path control system for cleft lip and palate repair surgery, used to perform surgical instrument path control for cleft lip and palate repair surgery, combined with... Figure 4 As shown, the instrument path control system 1 for cleft lip and palate repair surgery includes:

[0166] The operation feature extraction module 11 is used to acquire the operation signal information of the doctor's operating handle in the cleft lip and palate repair surgery instrument during the cleft lip and palate repair surgery, and extract operation features from the operation signal information.

[0167] The relationship information recognition module 12 is used to acquire surgical field image information provided by the endoscope and to identify the relative relationship information between the instruments operated by the doctor and the anatomical structures from the surgical field image information.

[0168] The operation intention judgment module 13 is used to integrate operation features and relative relationship information to judge the doctor's operation intention and obtain operation intention judgment information. During the process of judging the doctor's operation intention, when an abnormal situation is identified in the operation signal information, the doctor's operation intention is judged first based on the relative relationship information to obtain operation intention judgment information.

[0169] The control mode adjustment module 14 is used to dynamically adjust the control mode of the instrument operated by the doctor based on the information of the operation intention.

[0170] The "cleft lip and palate repair surgical instrument path control system" mentioned in this application is an intelligent system that integrates multiple functional modules, designed to assist doctors in performing cleft lip and palate repair surgery and ensure the accuracy and safety of instrument operation.

[0171] The operation feature extraction module can be a hardware circuit unit, such as a digital signal processor (DSP) or microcontroller, configured to receive sensor signals from the doctor's operating handle and execute a preset algorithm to extract operation features. This module can also be a software program running on a general-purpose computer or a dedicated server, which processes and analyzes the raw operation signal information to obtain operation features. The specific methods for extracting operation features have been described in the above embodiments and will not be repeated here. It is important to emphasize that this module can capture key information such as the movement and force feedback of the doctor's operating handle in real time and accurately, providing basic data for subsequent intent determination.

[0172] The relationship information recognition module can be an image processing unit, such as a graphics processing unit (GPU) or a dedicated vision processing chip, configured to receive real-time video streams from the endoscope and execute image recognition and computer vision algorithms to identify the relative relationship information between the instruments manipulated by the surgeon and the anatomical structures. This module can also be a deep learning model running on a high-performance computing platform, trained on massive amounts of surgical image data to achieve accurate identification and positioning of instruments and anatomical structures. The specific methods for recognizing relative relationship information have been described in the above embodiments and will not be repeated here. It is important to emphasize that this module can provide objective, real-time information about the surgical area environment, compensating for potential deviations in preoperative planning and initial registration.

[0173] The operation intent determination module can be a central processing unit (CPU) or an artificial intelligence inference engine. It is configured to receive operation features output by the operation feature extraction module and relative relationship information output by the relationship information recognition module, and execute a multimodal fusion algorithm or machine learning model to determine the doctor's current operation intent. This module can also be a rule-based expert system that infers and judges input information according to preset logical rules. The specific process of judging the doctor's operation intent, including the judgment priority mechanism in abnormal situations, has been described in the above embodiments and will not be repeated here. It is important to emphasize that this module, by fusing multi-source information, can make a comprehensive and accurate judgment of the doctor's intent. Especially when a single information source is abnormal, it can prioritize the judgment based on reliable information sources to avoid misjudgment.

[0174] The control mode adjustment module can be a motion controller or a robot control system. It is configured to receive operation intent judgment information output by the operation intent judgment module and dynamically adjust the control mode of the instrument operated by the physician based on this information. This module may include a force feedback controller, compliance controller, etc., and adjust parameters such as the instrument's virtual stiffness, virtual damping, and force feedback gain to match the instrument's response characteristics with the physician's current operational needs and the characteristics of the surgical area. The specific methods of dynamic control mode adjustment have been described in the above embodiments and will not be repeated here. It is important to emphasize that this module enables the instrument's response characteristics to be dynamic, not fixed, based on the physician's real-time operational intent and the surgical environment, thereby improving the instrument's adaptability and operational smoothness.

[0175] The core innovation of the instrument path control system for cleft lip and palate repair surgery presented in this application, compared to existing technologies, lies in its modular design, which enables precise judgment of the surgeon's operational intentions and dynamic adjustment of the instrument control mode. Existing systems primarily rely on preoperative planning and fixed registration relationships, lacking adaptability to real-time changes during surgery and failing to distinguish between intentional path adjustments and unintentional system errors. This leads to a "struggle" between the instrument and the surgeon, causing vibration and lag. The system in this application, through the collaborative work of an operation feature extraction module, a relationship information recognition module, and an operation intention judgment module, can acquire and integrate the surgeon's subjective intentions and the objective environmental information of the surgical area in real time and comprehensively. In particular, when the operation intention judgment module identifies abnormalities in the operation signal information, it prioritizes judgment based on relative relationship information, effectively avoiding the problem of traditional systems misinterpreting correct surgeon operations as system errors when the surgeon manually fine-tunes or sensors malfunction. Therefore, the control mode adjustment module can dynamically adjust the instrument control mode based on accurate judgment of the surgeon's operational intentions, eliminating instrument vibration and lag, and significantly improving the accuracy, safety, and surgeon's experience during surgery. The system described in this application provides a more intelligent and safer instrument path control solution for cleft lip and palate repair surgery.

[0176] The above are merely embodiments of this application and are not intended to limit the scope of protection of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. A surgical instrument path control system for cleft lip and palate repair surgery, used for performing surgical instrument path control for cleft lip and palate repair surgery, characterized in that, The method comprises the following steps: An operation feature extraction module is used to acquire operation signal information of a doctor's operation handle in a cleft lip and palate repair surgery and extract operation features from the operation signal information; A relationship information identification module is used to acquire surgical field image information provided by an endoscope and identify relative relationship information between the instrument operated by the doctor and an anatomical structure from the surgical field image information; An operation intention judgment module is used to fuse the operation features and the relative relationship information, judge the doctor's operation intention, and obtain operation intention judgment information; During the judgment of the doctor's operation intention, when it is identified that the operation signal information has an abnormal condition, the judgment of the doctor's operation intention is preferentially performed according to the relative relationship information, and the operation intention judgment information is obtained; The reliability of the surgical field image information is further evaluated, and a reliability score is obtained; According to the reliability score, the judgment weights of the operation features and the relative relationship information in the judgment process of the doctor's operation intention are adjusted; According to the operation type and the anatomical region of the current cleft lip and palate repair surgery, the judgment threshold of the doctor's operation intention is adjusted; According to the adjusted judgment weights and the judgment threshold, the judgment of the doctor's operation intention is performed, and the operation intention judgment information is obtained; When the operation features and the relative relationship information are both unreliable or in conflict, the current control mode of the instrument operated by the doctor is maintained; A control mode adjustment module is used to dynamically adjust the control mode of the instrument operated by the doctor according to the operation intention judgment information.

2. A cleft lip and palate repair surgical instrument path control system according to claim 1, wherein, The fusion of the operation features and the relative relationship information, the judgment of the doctor's operation intention, and the operation intention judgment information obtained are as follows: The quality of the surgical field image information is monitored in real time, and the reliability is evaluated according to the quality of the surgical field image information to obtain a reliability score; When the reliability score is lower than a preset score threshold, the judgment focus of the doctor's operation intention is shifted to the operation features, and a backup judgment rule set is enabled; the backup judgment rule set analyzes the smoothness, duration, and direction consistency of the operation features; the judgment threshold of the operation features is adjusted in combination with the anatomical region and the operation type of the current cleft lip and palate repair surgery; According to the adjusted judgment threshold, the judgment of the doctor's operation intention is performed, and the operation intention judgment information is obtained; During the period when the surgical field image information is lower than the preset score threshold, the reliability score of the surgical field image information is continuously recovered; When the reliability score is recovered, the judgment focus of the doctor's operation intention is redistributed to the fusion judgment of the operation features and the relative relationship information.

3. The cleft lip and palate repair surgical instrument path control system of claim 1, wherein, The dynamic adjustment of the control mode of the instrument operated by the doctor according to the operation intention judgment information comprises the following steps: The contact force information between the tip of the instrument operated by the doctor and the tissue in the anatomical region of the current surgery is acquired. According to the tissue contact force information, tissue mechanical properties of the anatomical region are evaluated; the tissue mechanical properties include tissue stiffness and tissue elasticity; According to the tissue mechanical properties, compliance parameters in a control mode of the instrument operated by the doctor are dynamically adjusted; the compliance parameters include proportional and integral gains for controlling instrument stiffness, and virtual stiffness and virtual damping.

4. The cleft lip and palate repair surgical instrument path control system of claim 1, wherein, The dynamically adjusting the control mode of the instrument operated by the doctor according to the operation intention judgment information comprises: Real-time analysis of the motion trend of the doctor's operating handle is performed to determine the motion trajectory and target region of the tip of the instrument operated by the doctor within a preset future time length; In combination with preoperative three-dimensional anatomical model information provided by a surgical navigation system, the type of the anatomical region into which the tip of the instrument operated by the doctor is about to enter and the typical surgical operation type corresponding to the anatomical region are identified; Before entering the anatomical region, the compliance parameters in the control mode of the instrument operated by the doctor are adjusted in advance according to the tissue properties of the anatomical region and the requirements of the surgical operation type.

5. A cleft lip and palate repair surgical instrument path control system according to claim 4, wherein, The real-time analysis of the motion trend of the doctor's operating handle to determine the motion trajectory and target region of the tip of the instrument operated by the doctor within a preset future time length comprises: Obtaining force sensor data of the doctor's operating handle; According to the force sensor data, mechanical interaction characteristics when the doctor's operating handle contacts with the tissue are identified; the mechanical interaction characteristics include contact force change rate and moment change trend; In combination with the mechanical interaction characteristics and the motion trend of the doctor's operating handle, the motion trajectory and target region of the tip of the instrument operated by the doctor within a preset future time length are predicted; when the mechanical interaction characteristics indicate that the instrument operated by the doctor is about to contact with the tissue, the prediction accuracy of the motion trajectory and target region is adjusted preferentially according to the mechanical interaction characteristics.

6. A cleft lip and palate repair surgical instrument path control system according to claim 5, wherein, The preferentially adjusting the prediction accuracy of the motion trajectory and target region according to the mechanical interaction characteristics when the mechanical interaction characteristics indicate that the instrument operated by the doctor is about to contact with the tissue comprises: Real-time monitoring of fluctuation, signal noise level and deviation degree from historical mechanical characteristics of the mechanical interaction characteristics is performed to evaluate the reliability of the mechanical interaction characteristics; When the reliability of the mechanical interaction characteristics is lower than a preset threshold, the mechanical interaction characteristics are weighted and averaged or filtered in combination with the motion trend of the doctor's operating handle to reduce the influence of abnormal fluctuations, to obtain processed mechanical interaction characteristics; According to the processed mechanical interaction characteristics, whether the instrument operated by the doctor is about to contact with the tissue is determined to obtain contact judgment result information; According to the contact judgment result information, the prediction accuracy of the motion trajectory and target region is adjusted.

7. A cleft lip and palate repair surgical instrument path control system according to claim 6, wherein, The preferentially adjusting the prediction accuracy of the motion trajectory and target region according to the mechanical interaction characteristics when the mechanical interaction characteristics indicate that the instrument operated by the doctor is about to contact with the tissue comprises: Real-time obtaining of tissue contact force information of the tip of the instrument operated by the doctor and the tissue of the current anatomical region; Analysis of the rate of change and frequency components of the tissue contact force information over time to obtain mechanical interaction characteristics; The space judgment feature is obtained by judging whether the tip of the instrument controlled by the doctor is close to or enters the marked tissue region according to the information of the marked tissue region in the preoperative three-dimensional anatomical model provided by the surgical navigation system and the spatial relative relationship between the current position of the tip of the instrument controlled by the doctor and the marked tissue region; When the result of analyzing the mechanical interaction feature based on the space judgment feature indicates that the instrument controlled by the doctor is about to contact the marked fragile tissue, and the position of the tip of the instrument is close to or enters the unmarked potential fragile region in the preoperative three-dimensional anatomical model, a specific change mode of the mechanical interaction feature is matched with a preset fragile tissue mechanical response mode to identify the type of the unmarked potential fragile region; According to the type of the unmarked potential fragile region, the prediction accuracy of the motion trajectory and the target region is dynamically adjusted, and the virtual stiffness and virtual damping parameters of the instrument controlled by the doctor are updated.

8. A cleft lip and palate repair surgical instrument path control system according to claim 7, wherein, The matching of the specific change mode of the mechanical interaction feature with the preset fragile tissue mechanical response mode to identify the type of the unmarked potential fragile region when the result of analyzing the mechanical interaction feature based on the space judgment feature indicates that the instrument controlled by the doctor is about to contact the marked fragile tissue, and the position of the tip of the instrument is close to or enters the unmarked potential fragile region in the preoperative three-dimensional anatomical model includes: The mechanical interaction feature when the tip of the instrument controlled by the doctor contacts the tissue is monitored in real time; the mechanical interaction feature includes the rate of change of contact force with time and the frequency component; The visual feature of the tissue around the tip of the instrument controlled by the doctor is identified in combination with the real-time surgical field image information; The mechanical interaction feature and the visual feature are fused to obtain a multi-modal interaction fingerprint; The multi-modal interaction fingerprint is cross-compared with a preset multi-modal response fingerprint of fragile tissue; According to the current surgical stage, the identified anatomical region, and the historical operation habit of the doctor, the weights of the mechanical interaction feature and the visual feature in the multi-modal interaction fingerprint are dynamically adjusted, and the matching threshold of the multi-modal interaction fingerprint and the preset multi-modal response fingerprints of several fragile tissues is adjusted to identify the type of the unmarked potential fragile region.

Images