Construction method for individualized virtual laparoscopic intraoperative risk feedback model and feedback method for individualized virtual laparoscopic intraoperative risk feedback model

Abstract

A construction method for an individualized virtual laparoscopic intraoperative risk feedback model and a feedback method for an individualized virtual laparoscopic intraoperative risk feedback model, belonging to the field of medical robotics. The construction method for an intraoperative risk feedback model comprises: constructing a multi-dimensional risk assessment model based on individual patient characteristics that comprehensively considers organ morphology, a physical distance of a mechanical arm, clinical risk levels of organs, and historical risk accumulation, so as to quantify risk scores for different organs during virtual laparoscopic surgery; improving a conventional attention mechanism on the basis of the risk scores, and designing a risk attention module; embedding the risk attention module into an encoder and a decoder of a conventional U-Net segmentation model to obtain an RU-Net segmentation model; and using the RU-Net segmentation model to achieve organ segmentation, highlight risk areas, and dynamically generate visual feedback such as adaptive transparency adjustment and risk warnings. The present application improves the comprehensiveness and accuracy of intraoperative risk analysis, and enables dynamic real-time visual feedback, significantly improving the reliability of virtual laparoscopic surgery.

Description

Construction of individualized virtual laparoscopic intraoperative risk feedback model and feedback method TECHNICAL FIELD

[0001] The present application relates to the technical field of medical robots, in particular to a construction of individualized virtual laparoscopic intraoperative risk feedback model and feedback method. BACKGROUND

[0002] With the rapid development of medical imaging technology and computer-aided surgery, virtual laparoscopic surgery based on individualized medicine plays an increasingly important role in modern surgery. Individualized medicine emphasizes the development of precise surgical plans for the unique anatomical features and pathological conditions of each patient, which is of great significance to improve the safety and effectiveness of surgery. However, traditional virtual surgery systems still have many challenges in individualized risk identification and real-time feedback. Existing technologies usually rely on static risk assessment methods, which are difficult to dynamically adjust according to individual differences of patients, and cannot adapt to the changing risk environment during surgery. In addition, traditional image segmentation models often lack the ability to accurately identify individual high-risk areas when dealing with complex anatomical structures specific to patients. Therefore, how to identify and avoid high-risk areas based on individual patient characteristics during preoperative planning and surgery to achieve truly individualized precision surgery has become a key problem that needs to be solved in the current medical field.

[0003] The Chinese patent application file with publication number CN118571477A discloses a deep learning perioperative risk analysis method and system fusing a high-dimensional self-attention mechanism. The method includes obtaining risk prediction results based on a risk analysis model, and the encoder of the risk analysis model contains a position encoding module. The relative position information based on time information is added to each index data part of the input perioperative data, so that the risk analysis model considers the numerical significance and periodic trend of the patient's vital signs. The trend of the vital signs can provide important information about the patient's physiological state, so that the risk analysis model can better evaluate the patient's physiological state and improve the prediction accuracy. In the risk score calculation step, the risk prediction results are weighted and calculated based on risk weight information to obtain the risk score. The main purpose is to calculate the risk score based on position information and vital signs. It does not model and analyze the importance of organs, the morphological characteristics of organs (volume, surface area, shape complexity, etc.), the cumulative effect of historical risks, etc., resulting in low risk prediction accuracy, incomplete and inaccurate risk analysis, and lack of dynamic adjustment and real-time visual feedback.

[0004] Therefore, there is an urgent need to develop a new feedback model and feedback method to better meet the risk identification and real-time feedback needs of virtual laparoscopic surgery robots during surgery. SUMMARY

[0005] The present application aims to at least solve one of the technical problems in the related art. To this end, the present application provides a personalized virtual laparoscopic surgery risk feedback model construction and feedback method. The method first constructs a targeted risk assessment model through three-dimensional reconstruction and analysis of the patient's individual medical image data, and then can dynamically identify and avoid high-risk areas according to the patient's specific abdominal tissue and organ structure and environment during the operation, which is more suitable for the dynamically changing risk environment in the individualized operation process. This individualized risk feedback mechanism can provide customized surgical safety protection for different patients, significantly improving the accuracy and safety of the operation.

[0006] To achieve the above-mentioned purpose, in a first aspect, the present application provides a personalized virtual laparoscopic surgery risk feedback model construction method, comprising:

[0007] Construct a multi-dimensional risk assessment model based on the individual characteristics of the patient, comprehensively consider the organ morphology, mechanical arm physical distance, organ clinical risk level and historical risk accumulation, and quantify the risk score of different organs in the virtual laparoscopic surgery process.

[0008] Based on the risk score, improve the conventional attention mechanism and design a risk attention module;

[0009] Embed the risk attention module into the encoder and decoder of the traditional U-Net segmentation model to obtain the RU-Net segmentation model;

[0010] Use the RU-Net segmentation model to realize organ image segmentation, highlight the risk area, and dynamically generate adaptive transparency adjustment and risk warning feedback.

[0011] Preferably, the multi-dimensional risk assessment model comprises:

[0012] Distance dynamic evaluation mechanism, dynamically updating the risk score according to the physical distance, speed and acceleration of the mechanical arm and the risk area;

[0013] Morphological feature evaluation mechanism, dynamically adjusting the risk score by calculating the volume, surface area and shape change of the target organ;

[0014] Historical risk accumulation evaluation mechanism, comprehensively considering the cumulative risk effect in the operation process to generate a comprehensive risk score.

[0015] Preferably, the risk attention module dynamically adjusts the attention weight using the risk score, and the specific implementation comprises:

[0016] Extract the initial feature map F from the input three-dimensional medical image, and generate the query matrix (Q), the key matrix (K) and the value matrix (V) through a first convolution layer: Q=W q• F, K = W k • F, V = W v • F;

[0017] wherein W q ,W k ,W v is a learnable weight matrix;

[0018] The risk attention module dynamically adjusts the attention weight of the feature map according to the risk score, and the features of the high-risk area are given higher weights to ensure that the segmentation model extracts more abundant features in these areas. The attention weight matrix is calculated by the following formula:

[0019] wherein:

[0020] A(Q, K): attention weight matrix;

[0021] R(t): comprehensive risk function;

[0022] d k : feature dimension.

[0023] Preferably, before the image segmentation, the input individual patient CT image sequence is preprocessed to enhance the display of the target tissue structure in the individualized case; after the image segmentation, a three-dimensional reconstruction algorithm is executed to construct a three-dimensional model through the Marching Cubes algorithm, realizing the accurate visualization of the target anatomical structure.

[0024] Preferably, the morphological feature evaluation is calculated by the following formula:

[0025] wherein:

[0026] V(t): time-varying volume change feature, time-varying volume refers to the organ volume changing with time, when there is collision or compression, the organ volume will change;

[0027] w1: volume feature weight of the risk area;

[0028] w2: surface area feature weight of the risk area;

[0029] A(t): surface complexity of the risk area, used to reflect the irregularity of the organ surface, indicating the possible lesion area;

[0030] w3 is the shape change rate weight, the integral term is used to capture the dynamic change of the organ shape over time, wherein the shape complexity function is defined as

[0031] w4 is the shape complexity weight, Geometric features describing the shape of the organ, wherein P(t) is the projected perimeter and r(t) is the equivalent radius.

[0032] Preferably, the distance dynamic evaluation mechanism is calculated by the following formula:

[0033] Wherein:

[0034] L(t) is the actual physical distance of the robotic arm from the risk region at the current time t;

[0035] ε=0.001 is a conventional coefficient;

[0036] μ is a speed influence factor, reflecting the influence degree of the current robotic arm dynamic speed on the risk;

[0037] v is an acceleration influence factor, reflecting the influence degree of acceleration on the risk;

[0038] is a basic term, representing the reciprocal of the instantaneous distance between the surgical instrument and the risk region, the shorter the distance, the higher the risk score, showing non-linear growth;

[0039] is a first-order derivative term of distance with respect to time, representing the instantaneous speed of the robotic arm relative to the risk region;

[0040] is a second-order derivative term of distance with respect to time, representing the acceleration of the robotic arm relative to the risk region.

[0041] Preferably, the historical risk accumulation evaluation is calculated by the following formula:

[0042] Wherein:

[0043] e -λ(t-τ) is an exponential decay kernel function, indicating the decay effect of historical risk over time, wherein t-τ is the time difference and λ is the risk decay rate parameter;

[0044] R(τ)=R d (τ)+R m (τ)+R v (τ)+R a (τ) is a comprehensive risk function for recording the instantaneous risk value at time τ, wherein R d (τ) is a distance-related risk, R m (τ) is a shape-related risk, R v (τ) is a speed-related risk, R a (τ) is an acceleration-related risk.

[0045] In a second aspect, the present application also provides a feedback method of an individualized virtual laparoscopic surgery risk feedback model, comprising the following steps:

[0046] (1) An adaptive threshold mechanism is set to dynamically adjust the risk level according to the risk change during the operation process, and the corresponding visual feedback is set according to the risk level;

[0047] (2) The visual feedback maps different risk levels to corresponding visual attributes, including color, transparency, brightness, and flicker, which are used to convey risk information for the operator to identify;

[0048] (3) According to the change of risk level, the intensity of visual feedback is dynamically adjusted in real time, including transparency adjustment, color change or flicker prompt, to provide accurate risk warning information to assist the operation.

[0049] Preferably, the adaptive threshold function calculation formula is as follows:

[0050] Wherein:

[0051] Ti(t) is the adaptive threshold function;

[0052] is the basic threshold, the second term represents the risk accumulation effect during the fixed time period Δt, and the third term reflects the change trend of the risk, and u1 and u2 are the weight coefficients of the accumulation effect and the change rate, respectively.

[0053] Preferably, the transparency adjustment method comprises the following steps:

[0054] (1) A risk assessment model is constructed to calculate a real-time comprehensive risk score according to the distance between the virtual mechanical arm and the anatomical structure, the importance and morphological characteristics of the organ;

[0055] (2) A transparency processing threshold is set, and when the distance between the surgical instrument and the high-risk area is less than the preset threshold, the transparency dynamic adjustment is triggered;

[0056] (3) The transparency level is dynamically adjusted based on the risk score, and the change range and intensity of the transparency are adaptively adjusted;

[0057] (4) A gradual mechanism of transparency adjustment is set to realize the smooth transition of transparency by gradual change, avoiding the visual jump caused by the change of transparency.

[0058] In a third aspect, the present application also provides an individualized virtual laparoscopic surgery risk feedback system, comprising:

[0059] An input module is configured to input patient individual characteristic parameters, including medical image data, organ morphological characteristics, mechanical arm distance parameters, and historical risk parameters accumulated during a surgical procedure.

[0060] An output module is configured to output a dynamically adjusted risk score, visual feedback parameters, and control the visual display module to dynamically adjust the transparency, color, brightness, and flicker frequency of the high-risk area.

[0061] A control module is configured to generate a comprehensive risk score based on the input individual characteristics and real-time surgical parameters, control the RU-Net segmentation model to dynamically segment key anatomical structures of the patient, and dynamically adjust the transparency and warning effect of the visual prompt based on the risk level.

[0062] In a fourth aspect, the present application also provides a processor executable program which can be directly executed by a processor or compiled and then executed by a processor, for executing the construction method and feedback method of the individualized virtual laparoscopic intraoperative risk feedback model as described above.

[0063] In a fifth aspect, the present application also provides a storage medium having a non-volatile program code executable by a processor or compiled and then executable by a processor stored thereon, for executing the construction method and feedback method of the individualized virtual laparoscopic intraoperative risk feedback model as described above.

[0064] Based on the above technical solution, the present application has at least the following beneficial effects relative to the prior art:

[0065] 1. The risk feedback model constructed based on patient individual characteristics can provide customized surgical safety protection for different patients

[0066] Traditional virtual surgery systems usually rely on static risk assessment methods in individual risk identification, which is difficult to dynamically adjust according to patient individual differences. The risk feedback model constructed based on patient individual characteristics in the present application can fully exhibit the unique pathological characteristics of each patient by three-dimensional reconstruction and analysis of the patient's medical image data, and can dynamically identify and avoid high-risk areas according to the patient's specific abdominal tissue and organ structure and environment during the surgical procedure. It is more suitable for the dynamically changing risk environment in individualized surgery, and can provide customized surgical safety protection for different patients, significantly improving the precision and safety of surgery.

[0067] 2. Improve the segmentation accuracy of high-risk areas and more accurately identify and quantify high-risk areas

[0068] The attention mechanism in conventional medical image segmentation models assigns the same weight to all regions, which cannot distinguish the risk levels of different organs. The present application constructs a multi-dimensional risk assessment model to calculate the risk score from multiple aspects such as organ morphology, mechanical arm physical distance, and historical risk, and assigns a higher weight to high-risk regions (such as the liver). The improved risk attention module can significantly improve the segmentation accuracy of high-risk regions while weakening the segmentation intensity of low-risk regions (such as distant blood vessels), thereby more accurately identifying and quantifying high-risk regions and providing a scientific basis for dynamic adjustment in the surgical environment.

[0069] 3. Realize risk-guided dynamic segmentation and enhance feature extraction intensity of key risk regions

[0070] The present application proposes a risk attention module (Risk-Attention Module) for improving the encoder and decoder in the traditional U-Net segmentation model, thereby constructing a new risk U-Net segmentation model (RU-Net). This segmentation model can emphasize feature extraction of high-risk regions when segmenting organs, dynamically guide the concentration of computing resources to high-risk regions during segmentation, provide more efficient and accurate risk warning segmentation capability, and improve the reliability of segmentation results of high-risk regions, thereby providing solid technical support for improving surgical quality and reducing intraoperative risks.

[0071] 4. Optimize computing efficiency

[0072] Compared with the traditional attention mechanism that treats all regions equally, the present application optimizes the allocation of computing resources through a risk scoring mechanism, concentrating more computing power on high-risk regions. This design not only reduces the computational overhead of low-risk regions but also improves overall computing efficiency, enabling the model to meet the demand for high efficiency while maintaining high accuracy.

[0073] 5. Propose a new adaptive threshold mechanism for dynamic and real-time visual feedback

[0074] The present application proposes a new adaptive threshold mechanism that dynamically adjusts risk levels according to changes in risk during surgery. This adaptive mechanism can automatically adjust the alert threshold according to different stages and risk characteristics of surgery, providing more sensitive visual feedback in high-risk stages and appropriately reducing alert intensity in low-risk stages. The system maps different risk levels to corresponding visual attributes (such as color, transparency, brightness, etc.), which significantly improves the visual feedback effect in virtual surgery scenarios, providing clear and intuitive risk guidance for doctors and greatly improving operation safety and convenience.

[0075] Other features and advantages of the present application will be set forth in the following specification, and in part will be apparent from the description, or can be learned by practice of the application, the purposes and other advantages of the present application will be realized and attained by the structure particularly pointed out in the written description and claims hereof. BRIEF DESCRIPTION OF DRAWINGS

[0076] FIG. 1 is a general roadmap of the present application;

[0077] FIG. 2 is an exemplary risk attention mechanism guided RU-Net structure in the present application;

[0078] FIG. 3(A) is an exemplary three-dimensional model opaque effect diagram in a virtual laparoscopic environment in the present application;

[0079] FIG. 3(B) is an exemplary three-dimensional model transparent effect diagram in a virtual laparoscopic environment in the present application;

[0080] FIG. 4(A) is an exemplary three-dimensional model opaque and no position pop-up effect diagram in a virtual laparoscopic environment in the present application;

[0081] FIG. 4(B) is an exemplary three-dimensional model transparent and position pop-up effect diagram in a virtual laparoscopic environment in the present application. DETAILED DESCRIPTION

[0082] In order to make the purposes, technical solutions and advantages of the present application more clear, the present application is further described in detail below in combination with specific embodiments and with reference to the drawings.

[0083] The terms used in the embodiments of the present application are only for the purpose of describing specific embodiments, and are not intended to limit the embodiments of the present application. The singular forms "a", "an" and "the" used in the embodiments of the present application are also intended to include the plural forms, unless the context clearly indicates otherwise.

[0084] In view of the deficiencies of the prior art, the purpose of the present application is to provide an individualized virtual laparoscopic intraoperative risk feedback model construction and feedback method. By introducing an adaptive attention mechanism and a multi-dimensional risk assessment model, the system can dynamically identify and avoid high-risk areas during surgery.

[0085] Specifically, as shown in FIG. 1, the overall roadmap of the intraoperative risk feedback model proposed in this application is described. The model is based on the individual characteristics of the patient, taking into account multiple dimensions of risk factors such as organ shape, physical distance between mechanical arm and target tissue, clinical risk level of organ and historical risk accumulation, etc. Through a multi-dimensional risk assessment model, the risk score of different organs in virtual laparoscopic surgery is realized. On this basis, the conventional attention mechanism is improved, a risk attention module is designed and embedded into the encoder and decoder of the traditional U-Net model, a new RU-Net segmentation model is constructed for precise organ image segmentation and dynamic highlighting of high-risk areas. At the same time, combined with the risk level, self-adaptive transparency adjustment and visual feedback are generated, and finally an intraoperative risk feedback system for real-time risk warning is constructed.

[0086] Embodiment one

[0087] The application proposes a method for constructing an individualized virtual laparoscopic intraoperative risk feedback model, comprising:

[0088] 1. Image segmentation and anatomical region identification

[0089] 1.1 Image preprocessing

[0090] Before image segmentation, the input individual patient CT image sequences are preprocessed to improve segmentation accuracy and efficiency. Z-score normalization is used to convert pixel values ​​of CT images acquired from different devices and at different times into a standard normal distribution. Normalization and standardization techniques are combined to ensure consistency and comparability of image data from different periods for the same patient. To specifically enhance the display of target tissue structures in individualized cases, window width and window level parameters are adjusted according to the CT value range characteristics of different anatomical locations. For example, a lung window setting (window width 1500 HU, window level -600 HU) is used for lung lesions, while a mediastinal window setting (window width 350 HU, window level 40 HU) is used for mediastinal structures, ensuring clear visibility of key anatomical structures in different cases. For image denoising, considering the differences in scanning conditions and image noise characteristics among different patients, an adaptive combination strategy of anisotropic diffusion filtering and bilateral filtering is adopted. By analyzing the signal-to-noise ratio characteristics of each case image, filtering parameters are dynamically adjusted to effectively suppress noise signals in individual CT images while preserving edge detail information of lesion areas to the greatest extent. To enhance the contrast of CT grayscale images for each case, this application employs an adaptive histogram equalization (CLAHE) algorithm. This algorithm adaptively segments the image and dynamically adjusts the contrast enhancement parameters based on the grayscale distribution characteristics of local regions. Simultaneously, it combines Gamma correction technology to personalize parameter optimization for the unique grayscale distribution characteristics of different patients' images, ensuring that while maintaining the overall naturalness of the image, it highlights the tissue details of the region of interest. This individualized image preprocessing scheme can fully adapt to the differences in image features among different patients, laying the foundation for subsequent accurate segmentation.

[0091] 1.2 Risk Factor-Guided Adaptive Risk Attention Module

[0092] In this application, to ensure that the model pays special attention to high-risk areas (such as major blood vessels) during segmentation, an adaptive risk attention module based on surgical risk scores is designed, which improves the conventional attention mechanism by dynamically adjusting attention weights to enhance the feature extraction intensity of key anatomical structures, directly contributing to the assurance of surgical safety. The specific process is as follows:

[0093] (1) Self-attention feature extraction from CT images

[0094] First, an initial feature map F is extracted from the input 3D medical image, and a query matrix Q, a key matrix K, and a value matrix V are generated through a first-order convolutional layer: Q = W q ·F,K=W k ·F,V=W v ·F;

[0095] Among them W q W k W v It is a learnable weight matrix.

[0096] (2) Attention weight allocation mechanism

[0097] The risk attention module dynamically adjusts the attention weights of the feature maps based on the risk score R(t). Features in high-risk regions are assigned higher weights to ensure that the organ segmentation model extracts richer features from these regions. The formula for calculating the attention weights is:

[0098] Where Q and K are the query and key matrices, respectively, A(Q,K) is the attention weight matrix, R(t) is the comprehensive risk function, and dk is the feature dimension. This attention enhancement mechanism can dynamically adjust the organ segmentation model's focus on different anatomical structures, especially in high-risk areas, where the model automatically enhances feature extraction intensity. Guided by risk scoring, the model can accurately identify and avoid potential risks in complex surgical environments, providing reliable support for surgical procedures.

[0099] (3) Feature weighting

[0100] The risk attention module guided by risk factors is used to weight the V matrix to obtain the enhanced feature map F′ and the final feature map F. f The formula is as follows: F′=A(Q,K)·V; F f =F+F′;

[0101] This risk attention module dynamically adjusts attention weights, allowing the model to pay special attention to critical structures that may affect surgical safety during segmentation. The model exhibits higher accuracy and robustness when handling intricate anatomical structures such as major blood vessels and the liver. This design not only improves the model's segmentation accuracy but also provides reliable support for surgical planning and real-time decision-making.

[0102] 1.3 Segmentation Model Construction

[0103] In constructing the final segmentation model, this application improves the traditional U-Net segmentation model by introducing the aforementioned Risk Attention module, resulting in a novel RU-Net segmentation model to enhance its performance in risk-aware segmentation. The Risk Attention module, as a key module, dynamically adjusts the model's focus on different anatomical regions. By embedding the Risk Attention module into the encoder and decoder of the traditional U-Net segmentation model, the RU-Net segmentation model is obtained. This model not only captures multi-scale image features but also highlights the feature information of key risk regions during feature fusion. The Risk Attention module is used in the downsampling path to highlight the detailed features of key anatomical regions, and in the upsampling path to enhance the feature fusion capability of key risk regions. Through these innovative designs, the RU-Net segmentation model provides more efficient and accurate risk-aware segmentation capabilities in virtual surgical navigation systems, providing solid technical support for improving surgical quality and reducing intraoperative risks.

[0104] Figure 2 shows the RU-Net network structure of this application. A risk attention module is embedded in the encoder and decoder of the traditional U-Net segmentation model to improve the extraction of detailed features of key risk areas in the downsampling stage of the segmentation model. At the same time, features from skip-connections are fused in the upsampling stage to achieve selective detail recovery of key areas. Ultimately, the model focuses on the detailed identification of risk areas with limited computational load.

[0105] 1.4 Individualized 3D Model Construction

[0106] CT images are segmented into different tissue structures, and then a 3D reconstruction algorithm is executed to construct a 3D model, achieving precise visualization of the target anatomical structure. For each patient's unique anatomical features, a moving cubes algorithm is used for 3D surface reconstruction. By dynamically adjusting the isosurface threshold, the reconstructed model ensures that it accurately reflects the morphological characteristics of individual diseased tissues. During reconstruction, mesh density and smoothing parameters are adjusted according to the size, shape, and spatial relationship of organs and tissues in different patients, ensuring reconstruction accuracy while optimizing computational efficiency. For lesion areas, local mesh refinement techniques are used to improve the geometric details of key areas; while for non-critical areas, appropriate mesh simplification strategies are employed to achieve rational allocation of computational resources. This individualized 3D reconstruction and analysis scheme can fully reveal the unique pathological characteristics of each patient, providing strong technical support for precision medicine.

[0107] 2. Construction of Virtual Surgery Risk Assessment Model

[0108] This application proposes a multi-dimensional risk assessment model, where each dimension represents a key factor. It comprehensively considers factors such as organ morphology, the physical distance to the robotic arm, and accumulated historical risks to calculate a risk score for different organs during surgery.

[0109] Distance is the primary consideration; the shorter the distance, the higher the risk level. Secondly, the importance of the region is determined by anatomical knowledge and clinical experience; for example, the aorta and major blood vessels are given higher weight. Morphological characteristics include the region's volume, surface area, and shape complexity, which can affect the difficulty and risk of the surgical procedure. The cumulative effect of historical risks during the surgical process is also considered, with recent risks having a greater impact. In summary, the comprehensive risk function R(t) represents the comprehensive risk score at time t. This model considers the current state and historical cumulative risks, achieving a comprehensive assessment of the risks of the virtual laparoscopic surgical path through multi-dimensional feature fusion, defined as follows: R(t)=α·D(t)+β·P(t)+γ·M(t)+σ·H(t)+λ·Ω(R);

[0110] Where α is a fixed weight for distance factors during real-time interaction in virtual surgery, reflecting the importance of the distance between the virtual robotic arm and different tissues and organs, with a value between 0.4 and 0.7. β is a fixed weight for organ importance, reflecting the risk weight of different organs during surgery. γ represents a fixed weight for the radiomics characteristics of the current organ, as the degree of influence of morphological features on surgical risk. σ is the weight for historical risk assessment within a fixed time period during virtual surgery. λ is a regularization coefficient used to control the complexity of the model.

[0111] (1) Distance dynamic assessment mechanism

[0112] D(t) represents the distance dynamic evaluation function, which fully considers the physical distance, relative velocity, and acceleration between the virtual robotic arm and the risk area. The formula is as follows:

[0113] Where L(t) is the actual physical distance between the robotic arm and the risk area at time t; ε = 0.001 is the conventional coefficient; μ is the speed influence factor, reflecting the degree of influence of the current dynamic speed of the robotic arm on the risk; and v is the acceleration influence factor, reflecting the degree of influence of acceleration on the risk. As a basic component, it represents the reciprocal of the instantaneous distance between the surgical instrument and the risk area. The shorter the distance, the higher the risk score, and it increases non-linearly. The first derivative of distance with respect to time represents the instantaneous velocity of the robotic arm relative to the risk area. A positive value of the derivative indicates moving away from the risk area, while a negative value indicates moving closer to the risk area. Taking the absolute value indicates that regardless of whether the robotic arm is approaching or moving away, the rapid movement of the virtual surgical arm will increase the intraoperative risk and is used to predict potential collision risks. The term is the second derivative of distance with respect to time, representing the acceleration of the robotic arm relative to the risk area. This term is used to reflect the degree of abrupt changes in motion state. A larger acceleration indicates motion instability, and taking the absolute value indicates that abrupt changes in any direction increase the risk. It aims to assess the stability of surgical operations and provide early warning of sudden movements.

[0114] This multi-derivative evaluation model, designed for the virtual surgical scenario of this application, enables the risk assessment system to not only consider static distance but also predict and evaluate the risks brought about by dynamic movement, dynamically update the risk score, and provide more comprehensive and forward-looking safety assurance.

[0115] (2) Mechanism for assessing the importance of organs in the abdominal environment

[0116] The abdominal cavity involves the interaction of various human tissues and organs. By comprehensively considering the risk levels of different organs and establishing a scientific organ importance grading system, more accurate risk assessment guidance can be provided for surgical procedures, thereby maximizing surgical safety. The organ importance scoring function in this application can be expressed as:

[0117] Where a0 represents the polynomial constant term, a i δ represents the weighting coefficient for each level. i (O) is an indicator function for organ type. This application classifies abdominal organ types into four levels corresponding to δ. i The four different outputs of (O) are as follows: 1) Level 1 (lethal): aorta, major blood vessels, heart; 2) Level 2 (important): liver, kidneys, spleen; 3) Level 3 (minor): gallbladder, appendix; 4) Level 4 (common): adipose tissue, connective tissue. i The weighting coefficients for each level are determined in descending order to quantify the risk coefficients at different levels.

[0118] (3) Morphological feature evaluation mechanism

[0119] M(t) comprehensively considers the real-time changes in radiomics features such as organ volume, surface area, shape change rate, and complexity, and its impact on the risk coefficient is expressed by the following formula:

[0120] Where M(t) represents the morphological feature evaluation function, t represents time, w1 is the volume feature weight of the risk area, V(t) is the time-varying volume change feature, which refers to the organ volume changing over time. When there is a collision or compression, the organ volume will change. Larger organs may increase the difficulty of surgery. Volume change indicates tissue deformation or surgical progress, affecting the operating space of surgical instruments. w2 is the surface area feature weight of the risk area, A(t) is the surface complexity of the risk area, used to reflect the irregularity of the organ surface and indicate the possible lesion area. w3 is the shape change rate weight, which is an integral term. This is used to capture the dynamic changes in organ morphology over time, where the shape complexity function is defined as... This integral term can be used to accumulate and record the history of shape changes, reflecting the degree of tissue deformation and predicting possible shape change trends, thereby achieving faster risk perception. w4 is the shape complexity weight. Describe the geometric features of the organ's shape, where P(t) is the projected perimeter and r(t) is the equivalent radius.

[0121] This morphological feature scoring system can comprehensively describe the geometric characteristics of risk areas, capture dynamic changes, dynamically adjust risk scores, support real-time monitoring and early warning, and adapt to different types of surgical scenarios. Through the combination of these features, the system can more accurately assess surgical risks and provide doctors with more reliable decision support.

[0122] (4) Historical risk accumulation assessment mechanism

[0123] Considering the cumulative effect of historical risks, recent risks have a greater impact, as shown in the following formula.

[0124] Where H(t) is the historical risk cumulative assessment function, e -λ(t-τ) Let R(τ) be the exponentially decaying kernel function, representing the decay effect of historical risk over time. λ>0 is the decay rate parameter, t is time, τ represents the time at a specific moment during the operation, and t-τ is the time difference. A larger λ indicates that the historical risk decays faster, while a smaller λ indicates that the historical risk has a longer-term impact. Specifically, R(τ) = R d (τ)+R m (τ)+R v (τ)+R a (τ) is the comprehensive risk function, used to record the instantaneous risk value at time τ, involving the distance-related risk R in the aforementioned steps. d (τ), Morphology-related risk R m (τ), speed-related risk R v (τ) and acceleration-related risk R aH(t) accumulates all historical risks from the start of surgery (τ=0) to the current time (τ=t) through integration, comprehensively considering the accumulated risk effects during the operation to generate a comprehensive risk score. This historical risk accumulation mechanism can comprehensively record risk changes during the operation, provide historical risk prediction, and achieve a balance between short-term and long-term risks.

[0125] (5) Regularization term

[0126] The fundamental purpose of regularization is to control the complexity of the risk assessment model by adding constraints, making risk assessment during virtual surgery more stable and reliable. The complete regularization formula is as follows:

[0127] The first term is the space regularization term, and its expression is:

[0128] The gradient of the risk score R in the x, y, and z directions is represented by the first term, which aims to accumulate gradient information across the entire surgical space, ensuring the smoothness of the global risk distribution and comprehensively considering risk changes at all spatial locations. This ensures that risk scores in adjacent regions do not change abruptly, reflecting the gradual spatial variation of surgical risk and conforming to the continuous risk distribution characteristics in actual surgical environments. The second term is a sparse regularization term (L1 regularization), where η is a sparsity factor used to control the overall model complexity. The sparse regularization strategy in this application is used to automatically select important risk features, ignore secondary risk factors, and simplify the risk assessment model. The third term is direction-sensitive regularization, with the formula:

[0129] By selecting the maximum value of the risk assessment score gradient in different directions, it is possible to effectively identify the main directions of risk change, analyze the directional characteristics of the operable space, identify restricted directions, and assess the degree of operational freedom.

[0130] Example 2

[0131] This embodiment provides a feedback method for a personalized virtual laparoscopic surgery risk feedback model. This method employs a novel adaptive threshold mechanism that dynamically adjusts the risk level based on changes in risk during the surgical procedure and generates corresponding visual feedback based on the risk level, making the visual feedback more sensitive and accurate. Its core adaptive threshold function can be expressed as:

[0132] Among them, T i R(t) is the adaptive threshold function, where t is time, τ represents the time at a specific moment during the operation, and R(t) and R(τ) represent the combined risk functions at times t and τ, respectively. The first term represents the base threshold, the second term represents the cumulative risk effect over a fixed time period Δt, and the third term reflects the trend of risk change. u1 and u2 are the weighting coefficients for the cumulative effect and the rate of change, respectively. This adaptive threshold mechanism can automatically adjust the warning threshold according to different stages and risk characteristics of the surgery, providing more sensitive visual feedback at higher-risk stages and appropriately reducing the warning intensity at lower-risk stages.

[0133] This feedback method includes the following steps:

[0134] (1) A novel adaptive threshold mechanism is set up to dynamically adjust the risk level according to the risk changes during the operation, and corresponding visual feedback is set according to the risk level.

[0135] (2) Visual feedback and visual attribute mapping: Visual feedback is mapped to corresponding visual attributes based on the risk level. Visual attributes include color, transparency, brightness, flicker, etc., conveying risk information through multi-dimensional visual cues. For example, high-risk areas can be mapped to a visual effect of darker color and enhanced flicker, while low-risk areas can be presented with lower brightness or lighter color cues. This ensures the effectiveness of the warnings while avoiding visual fatigue that may be caused by excessive warnings, providing doctors with more precise and humane surgical safety assurance.

[0136] (3) Adjust the intensity of visual feedback in real time. The intensity of visual feedback includes color changes, brightness adjustments, flashing prompts, contour enhancement and other visual feedback methods to convey risk information in real time and provide accurate risk warning information to assist surgical operations. Furthermore, establish a surgical simulation "approach warning" feedback mechanism. For high-risk areas that exceed the warning threshold, the risk information can be conveyed in real time by triggering warning effects such as flashing or contour enhancement. This visual prompting method helps operators to be more cautious when approaching key anatomical structures and avoid potential damage risks.

[0137] Furthermore, the visual feedback method includes setting up pop-up warnings on the screen. The pop-up warning content includes the specific distance d between the surgical instrument and the critical part, as well as the specific name of the critical part. As shown in Figure 4(A), the experiment set that the primary, secondary, and tertiary vessels of the hepatic vein and portal vein in the liver need to be transparent for warning processing. The warning threshold distance was set to 5mm. During the simulated liver resection surgery, when the tip of the surgical instrument approached the tertiary vessel of the hepatic vein, the algorithm adaptively adjusted the transparency of the model and displayed the warning content in a pop-up at a certain location. In the figure, two locations (d is 5mm and 2mm) were identified where the distance from the warning vessel was less than the threshold distance. Therefore, two warning pop-ups were displayed as shown in Figure 4(B).

[0138] Furthermore, the model's transparency can be adjusted in real time by calculating the distance between surgical instruments and key structures such as important blood vessels.

[0139] Furthermore, it is possible to achieve gradual adjustment of the transparency of 3D images, so that surgical instruments have a visual transition prompt when approaching critical areas, rather than suddenly becoming transparent. For example, the transparency of other tissue parts of the model can be adaptively adjusted according to the distance "danger level". The closer the distance, the more obvious the transparency adjustment, and the farther the distance, the weaker the transparency. That is, the transparency is controlled according to the distance between the surgical tip and the key point, and a smooth transition is achieved by setting it during the transition to avoid visual jumps.

[0140] Further, the transparency adjustment method includes the following steps:

[0141] (1) Construct a risk assessment model and calculate a real-time comprehensive risk score based on the distance between the virtual robotic arm and the anatomical structure, the importance of the organ, and the morphological characteristics;

[0142] (2) Set a transparency threshold. When the distance between the surgical instrument and the high-risk area is less than the preset threshold, dynamic adjustment of transparency is triggered.

[0143] (3) Based on the risk score, the transparency level is dynamically adjusted, and the range and intensity of the transparency change are adaptively adjusted. The closer the distance, the more obvious the transparency adjustment, and the farther the distance, the weaker the transparency. That is, the transparency is controlled according to the distance between the surgical tip and the key point, so that the transparency effect of high-risk areas is gradually enhanced, thereby improving the surgical operation field of vision.

[0144] (4) Set up a gradual mechanism for adjusting transparency to achieve a smooth transition of transparency through gradual changes, and avoid visual jumps caused by changes in transparency;

[0145] (5) While adjusting the transparency, combine it with other visual cues, including color changes, contour enhancements or flashing effects, to further emphasize high-risk areas.

[0146] Through the above adaptive adjustment mechanism, the transparency of the model tissue is adaptively adjusted according to different stages of the surgery. The closer the distance, the higher the transparency and the more obvious the adjustment; the farther the distance, the lower the transparency. That is, the transparency is controlled according to the distance between the surgical tip and the key point. This ensures the safety and intuitiveness of the surgical procedure. Note: Transparency refers to the degree to which an object allows light to pass through, usually expressed as 0% (completely transparent) to 100% (completely opaque). Objects with high transparency transmit almost all light, allowing people to clearly see objects inside or behind them; while objects with low transparency absorb or reflect most of the light, making objects inside or behind them difficult to identify or completely invisible.

[0147] Based on the above embodiments, this application also proposes an individualized virtual laparoscopic intraoperative risk feedback system, the system comprising:

[0148] The input module is used to input individual patient characteristics parameters, including medical imaging data (such as CT images), organ morphological characteristics (such as volume, surface area, and shape complexity), robotic arm distance parameters, and historical risk parameters accumulated during the operation.

[0149] The output module is used to output dynamically adjusted risk scores and visual feedback parameters, as well as to control the visual display module to dynamically adjust the transparency, color, brightness, and flicker frequency of high-risk areas, assisting doctors in identifying and avoiding high-risk areas.

[0150] The control module is used to generate a comprehensive risk score based on the input individualized features and real-time surgical parameters, control the RU-Net segmentation model to dynamically segment key anatomical structures of the patient, and dynamically adjust the transparency and warning effect of visual cues according to the risk level.

[0151] In addition, this application also provides a processor executable program that can be directly executed by a processor or compiled and then executed by a processor, for performing the above-mentioned individualized virtual laparoscopic surgery risk feedback model construction and feedback method.

[0152] On the other hand, this application also provides a storage medium storing processor-executable or compiled-executable non-volatile program code, the program code being used to execute the above-described individualized virtual laparoscopic surgery risk feedback model construction and feedback method.

[0153] The foregoing has described specific embodiments of the present application. Furthermore, the processes depicted in the accompanying drawings do not necessarily require a specific or sequential order to achieve the desired results. In some embodiments, multitasking and parallel processing are possible or may be advantageous.

[0154] In the description of the embodiments of this application, the terms "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., refer to specific features, structures, materials, or characteristics described in connection with that embodiment or example, which are included in at least one embodiment or example of the embodiments of this application. In the embodiments of this application, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described may be combined in a suitable manner in any one or more embodiments or examples. Furthermore, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in the embodiments of this application, as well as the features of different embodiments or examples.

[0155] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features, excluding any ordering. Thus, features defined with "first" and "second" may explicitly or implicitly include at least one of those features and are used to distinguish them from one another. In the description of embodiments of this application, "multiple" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0156] Any process or method description in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or more executable instructions for implementing custom logic functions or processes, and the scope of preferred embodiments of this application includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order according to the functions involved, as should be understood by those skilled in the art to which embodiments of this application pertain.

[0157] The above description is only a preferred embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present application should be included within the scope of protection of the present application.

Claims

1. A method for constructing an individualized virtual laparoscopic intraoperative risk feedback model, characterized in that, The method comprises the following steps: A multi-dimensional risk assessment model based on individual characteristics of patients is constructed, considering organ morphology, physical distance of mechanical arm, clinical risk level of organ and historical risk accumulation, to quantify the risk score of different organs in the virtual laparoscopic surgery process; A risk attention module is designed based on the risk score to improve the conventional attention mechanism; The risk attention module is embedded in the encoder and decoder of the traditional U-Net segmentation model to obtain the RU-Net segmentation model; The RU-Net segmentation model is used to realize organ image segmentation, highlight the risk area, and dynamically generate adaptive transparency adjustment and risk warning feedback.

2. The construction method of claim 1, wherein, The multi-dimensional risk assessment model comprises: A distance dynamic evaluation mechanism that dynamically updates the risk score according to the physical distance, speed and acceleration of the mechanical arm relative to the risk area; A morphological feature evaluation mechanism that dynamically adjusts the risk score by calculating the volume, surface area and shape change of the target organ; A historical risk accumulation evaluation mechanism that considers the cumulative risk effect during the surgery process to generate a comprehensive risk score.

3. The construction method of claim 1, wherein, The risk attention module dynamically adjusts the attention weight using the risk score, and the specific implementation steps comprise: Initial feature maps F are extracted from the input three-dimensional medical image, and query matrix (Q), key matrix (K) and value matrix (V) are generated through a first-order convolution layer: Q = W q • F, K = W k • F, V = W v • F; where W q is a learnable weight matrix k is a learnable weight matrix v is a learnable weight matrix The risk attention module dynamically adjusts the attention weight of the feature map according to the risk score. The features of the high-risk area are given higher weights to ensure that the segmentation model extracts more abundant features in these areas. The attention weight matrix is calculated by the following formula: Wherein: A(Q,K) is the attention weight matrix; R(t) is the comprehensive risk function; d k : feature dimension.

4. The construction method of claim 1, wherein, Before the image segmentation step, the input individual patient CT image sequence is preprocessed to enhance the display of the target tissue structure in the individualized case; after the image segmentation step, a three-dimensional reconstruction algorithm is executed to construct a three-dimensional model through the Marching Cubes algorithm, realizing accurate visualization of the target anatomical structure.

5. The method of construction of claim 2, wherein, The morphological feature assessment is calculated by the following formula: Wherein: M(t) represents the morphological feature evaluation function; t represents time; V(t) is the time-varying volume change feature, and the time-varying volume refers to the organ volume that changes over time, which changes when there is collision or compression; w1 is the volume feature weight of the risk area; w2 is the surface area feature weight of the risk area; A(t) is the surface complexity of the risk area, which is used to reflect the irregularity of the organ surface and indicate the possible lesion area; w3 is a shape change rate weight, the integral term For capturing the dynamic changes of organ morphology over time, where the shape complexity function is defined as w4 is a shape complexity weight, The geometric feature of the organ shape is described, wherein P(t) is the projection perimeter and r(t) is the equivalent radius; The distance dynamic evaluation mechanism calculates by the following formula: Wherein: D(t) is the distance dynamic evaluation function; L(t) is the actual physical distance of the mechanical arm from the risk area at the current time t; ε=0.001 is a conventional coefficient; μ is the speed influence factor, reflecting the influence degree of the current mechanical arm dynamic speed on the risk; v is the acceleration influence factor, reflecting the influence degree of the acceleration on the risk; is the base term, representing the inverse of the instantaneous distance between the surgical instrument and the risk area, and the shorter the distance, the higher the risk score, which increases nonlinearly; is the first derivative term of distance with respect to time, representing the instantaneous speed of the mechanical arm relative to the risk area; is the second derivative term of distance with respect to time, representing the acceleration of the mechanical arm relative to the risk area; The historical risk accumulation assessment is calculated by the following formula: Wherein: H(t) is the historical risk accumulation evaluation function; t is time, τ represents the time at a certain moment during the operation; e -λ(t-τ) is an exponential decay kernel function representing the decay effect of historical risk over time, where t - τ is the time difference and λ is the decay rate parameter; R(t) = R d (t) + R m (t) + R v (t) + R a (t) is the integrated risk function, which records the instantaneous risk value at time t, where R d (t) is the distance-dependent risk, R m (t) is the shape-dependent risk, R v (t) is the speed-dependent risk, R a (t) is the acceleration-dependent risk.

6. A feedback method of an individualized virtual laparoscopy in risk feedback model, characterized in that, The method comprises the following steps: (1) setting an adaptive threshold mechanism, dynamically adjusting the risk level according to the risk changes during the operation, and generating corresponding visual feedback according to the risk level; (2) the visual feedback maps different risk levels to corresponding visual properties, including color, transparency, brightness and flicker; (3) according to the change of risk level, real-time dynamic adjustment of visual feedback intensity, including transparency change, color switching or flicker prompt, to provide accurate risk warning information to assist operation; The transparency adjustment method preferably comprises the following steps: (1) constructing a risk assessment model, calculating the real-time comprehensive risk score according to the distance between the virtual manipulator and the anatomical structure, the importance of the organ and the morphological characteristics; (2) set the transparency processing threshold, when the distance between the surgical instrument and the high-risk area is less than the preset threshold, trigger the dynamic adjustment of transparency; (3) dynamically adjust the transparency level based on the risk score, and adaptively adjust the change range and intensity of the transparency; (4) set the gradual mechanism of transparency adjustment, realize the smooth transition of transparency by gradual change, and avoid visual jump.

7. The feedback method of claim 6, wherein, The adaptive threshold function calculation formula is as follows: Wherein: T i (t) is an adaptive threshold function; t is time, τ represents the time at a certain moment during the operation; R(t), R(τ) represent the comprehensive risk function at t, τ time; T i 0 For the base threshold, the second term in the above equation represents the cumulative effect of risk during a fixed time period Δt, and the third term reflects the trend of risk change, and u1 and u2 are the weight coefficients of the cumulative effect and the change rate, respectively.

8. An individualized virtual laparoscopy in risk feedback system, characterized in that, It includes: The input module is used for inputting the individual characteristic parameters of the patient, including medical image data, organ morphological characteristics, manipulator distance parameters and cumulative historical risk parameters in the operation process; The output module is used for outputting the dynamically adjusted risk score, visual feedback parameter, and controlling the visual display module to dynamically adjust the transparency, color, brightness and flicker frequency of the high-risk area; The control module is used for generating a comprehensive risk score according to the input individual characteristics and real-time operation parameters, controlling the RU-Net segmentation model to dynamically segment the key anatomical structure of the patient, and dynamically adjusting the transparency and warning effect of the visual prompt according to the risk level.

9. A processor executable program capable of being directly executed by a processor or compiled and then executed by a processor, for executing the construction method and feedback method of the individualized virtual laparoscopic intraoperative risk feedback model according to any one of claims 1-7.

10. A storage medium, characterized by The storage medium stores non-volatile program code executable by a processor or executable after being compiled, and the program code is used to execute the construction method and feedback method of the individualized virtual laparoscopic intraoperative risk feedback model of claim 1-7.

Images