Method for constructing and feeding back risk feedback model in individualized virtual laparoscopic surgery

Abstract

The invention discloses a risk feedback model construction and feedback method in an individualized virtual laparoscopic surgery, and belongs to the field of medical robots. The intraoperative risk feedback model construction method comprises the steps of constructing a multi-dimensional risk assessment model based on individual features of a patient, comprehensively considering organ forms, mechanical arm physical distances, organ clinical risk levels and historical risk accumulation, and quantifying risk scores for different organs in a virtual laparoscopic surgery process; improving a conventional attention mechanism based on the risk score, and designing a risk attention module; embedding the risk attention module into an encoder and a decoder of a traditional U-Net segmentation model to obtain an RU-Net segmentation model; an RU-Net segmentation model is used to realize organ segmentation, a risk area is highlighted, and visual feedback such as adaptive transparency adjustment and risk early warning is dynamically generated. According to the method, the comprehensiveness and accuracy of risk analysis in the operation are improved, dynamic real-time visual feedback can be realized, and the reliability of the virtual laparoscopic operation is remarkably improved.

Description

Technical Field

[0001] The present application relates to the field of medical robot technology, and specifically to the construction of a risk feedback model and a feedback method for individualized virtual laparoscopic surgery. Background Art

[0002] With the rapid development of medical imaging technology and computer-assisted surgery, virtual laparoscopic surgery based on personalized medicine plays an increasingly important role in modern surgery. Personalized medicine emphasizes the development of precise surgical plans based on the unique anatomical characteristics and pathological conditions of each patient. The implementation of this concept is of great significance to improving surgical safety and treatment effects. However, traditional virtual surgery systems still face many challenges in individual risk identification and real-time feedback. Existing technologies usually rely on static risk assessment methods, which are difficult to dynamically adjust according to individual differences of patients, and cannot adapt to the changing risk environment during surgery. In addition, traditional image segmentation models often lack the ability to accurately identify individual high-risk areas when dealing with patient-specific complex anatomical structures. Therefore, how to identify and avoid high-risk areas in real time during preoperative planning and surgery based on individual patient characteristics to achieve truly personalized precision surgery has become a key issue that needs to be urgently addressed in the current medical field.

[0003] The Chinese patent application document with publication number CN118571477A discloses a deep learning perioperative risk analysis method and system integrating a dimensional self-attention mechanism. The method includes "obtaining a risk prediction result based on a risk analysis model, and the encoder of the risk analysis model includes a position encoding module, adding relative position information based on time information to each indicator data part of the input perioperative data, so that the risk analysis model simultaneously considers the numerical significance of the patient's vital signs and the changing trend within the cycle. The changing trend of the vital signs can provide important information about the patient's physiological state, so that the risk analysis model can better evaluate the patient's physiological state and improve the prediction accuracy. In the risk score calculation step, the risk prediction result is weighted based on the risk weight information to obtain a risk score." It mainly calculates the risk score based on the position information and vital signs, and does not perform modeling and analysis based on the importance of the organ, the morphological characteristics of the organ (volume, surface area, shape complexity, etc.), the cumulative effect of historical risks, etc., resulting in low risk prediction accuracy, insufficient comprehensiveness and accuracy of the risk analysis, and lack of dynamic adjustment and real-time visual feedback.

[0004] It can be seen that there is an urgent need to develop a new feedback model and feedback method to better meet the risk identification and real-time feedback needs of virtual laparoscopic surgical robots during surgery. Summary of the invention

[0005] This application aims to solve at least one of the technical problems in the related art to some extent. To this end, this application provides a method for constructing and providing feedback on an individualized virtual laparoscopic intraoperative risk feedback model. This method first performs three-dimensional reconstruction and analysis on the medical image data of the patient individual to construct a targeted risk assessment model, and then can dynamically identify and avoid high-risk areas according to the specific abdominal tissue and organ structure and environment of the patient during the operation, which is more suitable for the dynamically changing risk environment during the individualized operation process. This individualized risk feedback mechanism can provide customized surgical safety guarantees for different patients, significantly improving the accuracy and safety of the operation.

[0006] To achieve the above object, in a first aspect, this application provides a method for constructing an individualized virtual laparoscopic intraoperative risk feedback model, including:

[0007] Construct a multi-dimensional risk assessment model based on the individual characteristics of the patient, comprehensively consider the organ morphology, the physical distance of the robotic arm, the clinical risk level of the organ, and the historical risk accumulation, and quantify the risk scores for different organs during the virtual laparoscopic surgery process.

[0008] Improve the conventional attention mechanism based on the risk score, and design a risk attention module;

[0009] Embed the risk attention module into the encoder and decoder of the traditional U-Net segmentation model to obtain the RU-Net segmentation model;

[0010] Use the RU-Net segmentation model to implement organ image segmentation, highlight the risk area, and dynamically generate adaptive transparency adjustment and risk warning feedback.

[0011] Preferably, the multi-dimensional risk assessment model includes:

[0012] A distance dynamic assessment mechanism that dynamically updates the risk score according to the physical distance, speed, and acceleration of the robotic arm from the risk area;

[0013] A morphological feature assessment mechanism that dynamically adjusts the risk score by calculating the volume, surface area, and shape change of the target organ;

[0014] A historical risk accumulation assessment mechanism that comprehensively considers the cumulative risk effect during the operation process to generate a comprehensive risk score.

[0015] Preferably, the risk attention module dynamically adjusts the attention weight using the risk score, and the specific implementation includes:

[0016] Extract the initial feature map F from the input three-dimensional medical image, and generate a query matrix (Q), a key matrix (K), and a value matrix (V) through a first-order convolutional layer:

[0017] Q = Wq ·F, K = W k ·F, V = W v ·F;

[0018] Among them, W q , W k , W v is a learnable weight matrix;

[0019] The risk attention module dynamically adjusts the attention weights of the feature map according to the risk score, and higher weights are assigned to the features in the high-risk areas to ensure that the segmentation model extracts richer features in these areas. The attention weight matrix is calculated by the following formula:

[0020]

[0021] Where:

[0022] A(Q, K): Attention weight matrix;

[0023] R(t): Comprehensive risk function;

[0024] d k : Feature dimension.

[0025] Preferably, before the image segmentation, the input CT image sequence of individual patients is preprocessed to specifically enhance the display of the target tissue structure in individual cases; after the image segmentation, a three-dimensional reconstruction algorithm is executed, and a three-dimensional model is constructed through the Marching Cubes algorithm to achieve precise visualization of the target anatomical structure.

[0026] Preferably, the morphological feature evaluation is calculated by the following formula:

[0027]

[0028] Where:

[0029] V(t): Time-varying volume change feature. The time-varying volume refers to the volume of the organ that changes over time. When there is collision or compression, the organ volume will change;

[0030] w 1 : Volume feature weight of the risk area;

[0031] w 2 : Surface area feature weight of the risk area;

[0032] A(t): Is the surface complexity of the risk area, used to reflect the irregularity of the organ surface and indicate possible lesion areas;

[0033] w 3 Is the shape change rate weight, integral term For capturing the dynamic changes of organ morphology over time, where the shape complexity function is defined as

[0034] w 4 is the shape complexity weight, describes the geometric features of the organ shape, where P(t) is the projected perimeter and r(t) is the equivalent radius.

[0035] Preferably, the distance dynamic evaluation mechanism is calculated by the following formula:

[0036]

[0037] where:

[0038] L(t) is the actual physical distance of the robotic arm from the risk area at the current time t;

[0039] ε = 0.001 is a conventional coefficient;

[0040] μ is the speed impact factor, reflecting the impact degree of the current dynamic speed of the robotic arm on the risk;

[0041] v is the acceleration impact factor, reflecting the impact degree of the acceleration on the risk;

[0042] is the basic term, representing the reciprocal of the instantaneous distance between the surgical instrument and the risk area. The shorter the distance, the higher the risk score, showing non-linear growth;

[0043] is the first derivative term of the distance with respect to time, representing the instantaneous speed of the robotic arm relative to the risk area;

[0044] is the second derivative term of the distance with respect to time, representing the acceleration of the robotic arm relative to the risk area.

[0045] Preferably, the historical risk cumulative evaluation is calculated by the following formula:

[0046]

[0047] where:

[0048] e -λ(t-τ) is the exponential decay kernel function, representing the decay effect of historical risk over time. Among them, t - τ is the time difference, and λ is the risk decay rate parameter;

[0049] R(τ) = R d (τ) + R m (τ) + R v (τ) + R aR(τ) is the comprehensive risk function, which is used to record the instantaneous risk value at time τ, where R d (τ) is the distance-related risk, R m (τ) is the morphology-related risk, R v (τ) is the speed-related risk, R a (τ) is the acceleration-related risk.

[0050] In a second aspect, the present application also provides a feedback method for an individualized virtual laparoscopic surgery risk feedback model, including the following steps:

[0051] (1) Set an adaptive threshold mechanism to dynamically adjust the risk level according to the risk changes during the surgery, and set corresponding visual feedback according to the risk level;

[0052] (2) The visual feedback maps different risk levels to corresponding visual attributes, and the visual attributes include color, transparency, brightness, and flicker, which are used to transmit risk information for easy identification by the operator;

[0053] (3) According to the change of the risk level, dynamically adjust the visual feedback intensity in real time, including transparency adjustment, color change or flicker prompt, to provide accurate risk warning information to assist the surgical operation.

[0054] Preferably, the calculation formula of the adaptive threshold function is as follows:

[0055]

[0056] Where:

[0057] Ti(t) is the adaptive threshold function;

[0058] T i 0 is the basic threshold, the second term represents the risk accumulation effect during the fixed time period Δt, the third term reflects the change trend of the risk, u 1 and u 2 are the weight coefficients of the accumulation effect and the change rate respectively.

[0059] Preferably, the transparency adjustment method includes the following steps:

[0060] (1) Construct a risk assessment model, and calculate the real-time comprehensive risk score according to the distance between the virtual robotic arm and the anatomical structure, the importance of the organ, and the morphological characteristics;

[0061] (2) Set the transparency processing threshold, and trigger the dynamic adjustment of transparency when the distance between the surgical instrument and the high-risk area is less than the preset threshold;

[0062] (3) Dynamically adjust the transparency level based on the risk score, and adaptively adjust the change range and intensity of the transparency;

[0063] (4) Set a progressive mechanism for transparency adjustment to achieve smooth transition of transparency through gradual changes and avoid visual jumps caused by transparency changes.

[0064] In a third aspect, the present application also provides an individualized virtual laparoscopic surgery risk feedback system, including:

[0065] An input module for inputting patient individual characteristic parameters, including medical image data, organ morphological characteristics, robotic arm distance parameters, and historical risk parameters accumulated during the surgical process;

[0066] An output module for outputting a dynamically adjusted risk score, visual feedback parameters, and controlling the visual display module to dynamically adjust the transparency, color, brightness, and flicker frequency of high-risk areas;

[0067] A control module for generating a comprehensive risk score based on the input individualized characteristics and real-time surgical parameters, controlling the RU-Net segmentation model to dynamically segment the patient's key anatomical structures, and dynamically adjusting the transparency and warning effect of visual cues according to the risk level.

[0068] In a fourth aspect, the present application also provides a processor-executable program that can be directly executed by a processor or can be executed by a processor after compilation, and is used to execute the construction method and feedback method of the individualized virtual laparoscopic surgery risk feedback model as described above.

[0069] In a fifth aspect, the present application also provides a storage medium, on which non-volatile program code that can be executed or executed after compilation by a processor is stored, and the program code is used to execute the construction method and feedback method of the individualized virtual laparoscopic surgery risk feedback model as described above.

[0070] Based on the above technical solutions, it can be seen that the present application has at least the following beneficial effects compared with the prior art:

[0071] 1. The risk feedback model constructed based on patient individual characteristics can provide customized surgical safety guarantees for different patients.

[0072] Traditional virtual surgery systems usually rely on static risk assessment methods in terms of individualized risk identification and are difficult to make dynamic adjustments according to individual patient differences. The risk feedback model constructed in this application based on individual patient characteristics can build a targeted risk assessment model by performing 3D reconstruction and analysis on the medical image data of individual patients, which can fully display the unique pathological characteristics of each patient. Subsequently, it can dynamically identify and avoid high-risk areas according to the specific abdominal tissue and organ structure and environment of the patient during the operation, and is more suitable for the dynamically changing risk environment in the individualized operation process. It can provide customized surgical safety guarantees for different patients and significantly improve the accuracy and safety of the operation.

[0073] 2. Improve the segmentation accuracy of high-risk areas and more accurately identify and quantify high-risk areas

[0074] The attention mechanism in the conventional medical image segmentation model assigns the same weight to all regions and cannot distinguish the risk levels of different organs. In this application, a multi-dimensional risk assessment model is constructed to jointly calculate the risk score from multiple aspects such as organ morphology, physical distance of the robotic arm, and historical risk, and assign a higher weight to high-risk areas (such as the liver). The improved risk attention module can significantly improve the segmentation accuracy of high-risk areas, while weakening the segmentation intensity of low-risk areas (such as distant blood vessels), so as to more accurately identify and quantify high-risk areas and provide a scientific basis for dynamic adjustment in the surgical environment.

[0075] 3. Achieve risk-guided dynamic segmentation and enhance the feature extraction intensity of key risk areas

[0076] This application proposes a risk attention module (Risk-Attention Module) to improve the encoder and decoder in the traditional U-Net segmentation model, thereby constructing a new risk U-Net segmentation model (RU-Net). This segmentation model can pay more attention to the feature extraction of high-risk areas when segmenting organs, dynamically guide the concentration of computing resources on high-risk areas during the segmentation process, provide a more efficient and accurate risk warning segmentation ability, improve the reliability of the segmentation results of high-risk areas, and provide strong technical support for improving the surgical quality and reducing intraoperative risks.

[0077] 4. Optimize the computing efficiency

[0078] Compared with the traditional attention mechanism that treats all regions equally, this application optimizes the computing resource allocation through the risk scoring mechanism and concentrates more computing power on high-risk areas. This design not only reduces the computing overhead of low-risk areas, but also improves the overall computing efficiency, enabling the model to meet the requirements of high efficiency while maintaining high accuracy.

[0079] 5. A novel adaptive threshold mechanism is proposed to perform visual feedback dynamically and in real time.

[0080] This application proposes a novel adaptive threshold mechanism that dynamically adjusts the risk level according to the risk changes during the surgical process. This adaptive mechanism can automatically adjust the warning threshold according to different stages and risk characteristics of the surgery, providing more sensitive visual feedback in stages with higher risks and appropriately reducing the warning intensity in stages with lower risks. The system maps different risk levels to corresponding visual attributes (such as color, transparency, brightness, etc.). This design significantly improves the visual feedback effect in the virtual surgical scene, providing clear and intuitive risk guidance for doctors and greatly enhancing the operation safety and convenience.

[0081] Other features and advantages of this application will be described in the subsequent specification. Moreover, some of them will become obvious from the specification, or the objectives and other advantages of this application can be realized and obtained by implementing this application and through the structures specifically pointed out in the written specification and the accompanying drawings. Brief Description of the Drawings

[0082] Figure 1 is the overall roadmap of this application;

[0083] Figure 2 is the RU-Net structure guided by the exemplary risk attention mechanism in this application;

[0084] Figure 3(A) is the opaque effect diagram of the 3D model in the exemplary virtual laparoscopic environment of this application;

[0085] Figure 3(B) is the transparent effect diagram of the 3D model in the exemplary virtual laparoscopic environment of this application;

[0086] Figure 4(A) is the opaque and position-popup-free effect diagram of the 3D model in the exemplary virtual laparoscopic environment of this application;

[0087] Figure 4(B) is the transparent and position-popup-displaying-warning-content effect diagram of the 3D model in the exemplary virtual laparoscopic environment of this application. Detailed Description of the Embodiments

[0088] To make the objectives, technical solutions, and advantages of this application clearer and more understandable, the following further elaborates on this application in detail with reference to specific embodiments and the accompanying drawings.

[0089] The terms used in the embodiments of this application are only for the purpose of describing specific embodiments and are not intended to limit the embodiments of this application. The singular forms "a", "the", and "said" used in the embodiments of this application are also intended to include the plural forms unless the context clearly indicates otherwise.

[0090] Aiming at the deficiencies of the prior art, the purpose of this application is to provide an individualized virtual laparoscopic intraoperative risk feedback model construction and feedback method. By introducing an adaptive attention mechanism and a multi-dimensional risk assessment model, the system can dynamically identify and avoid high-risk areas during the operation.

[0091] Specifically, as Figure 1 shown, the overall roadmap of the intraoperative risk feedback model proposed in this application is described. Based on the individual characteristics of the patient, this model comprehensively considers multiple dimensions of risk factors, such as organ morphology, the physical distance between the robotic arm and the target tissue, the clinical risk level of the organ, and historical risk accumulation, etc. Through the multi-dimensional risk assessment model, the risk scores of different organs in virtual laparoscopic surgery are realized. On this basis, the conventional attention mechanism is improved, a risk attention module is designed, and it is embedded in the encoder and decoder of the traditional U-Net model to construct a new type of RU-Net segmentation model for achieving accurate organ image segmentation and dynamic highlighting of high-risk areas. At the same time, combined with the risk level, adaptive transparency adjustment and visual feedback are generated, and finally an intraoperative risk feedback system for real-time risk warning is constructed.

[0092] Embodiment 1

[0093] This application proposes an individualized virtual laparoscopic intraoperative risk feedback model construction method, including:

[0094] 1. Image segmentation and anatomical region recognition

[0095] 1.1 Image preprocessing

[0096] Before image segmentation, the input CT image sequence of individual patients is preprocessed to improve the accuracy and efficiency of segmentation. The pixel values of CT images obtained from different devices and at different times are converted into a standard normal distribution through the Z-score normalization method, and standardization and normalization techniques are combined to ensure the consistency and comparability of image data of the same patient at different times. To specifically enhance the display of target tissue structures in individual cases, the window width and window level parameters are adjusted according to the CT value range characteristics of different anatomical sites. For example, for the lung lesion area, a lung window setting (window width 1500 HU, window level -600 HU) is adopted, while for the mediastinal structure, a mediastinal window setting (window width 350 HU, window level 40 HU) is used to ensure that the key anatomical structures in different cases are clearly visible. In terms of image denoising, considering the differences in scanning conditions and image noise characteristics of different patients, an adaptive combination strategy of anisotropic diffusion filtering and bilateral filtering is adopted. By analyzing the signal-to-noise ratio characteristics of each case image, the filtering parameters are dynamically adjusted to effectively suppress the noise signal of individual CT images while maximizing the preservation of the edge detail information of the lesion area. In this application, to enhance the contrast of each case's CT grayscale image, the adaptive histogram equalization (CLAHE) algorithm is used. By performing adaptive block processing on the image, the contrast enhancement parameters are dynamically adjusted according to the grayscale distribution characteristics of the local area. At the same time, combined with the Gamma correction technology, personalized parameter optimization is carried out according to the grayscale distribution characteristics of different patients' images to ensure that while maintaining the overall naturalness of the image, the tissue structure details of the region of interest are prominently displayed. This individualized image preprocessing scheme can fully adapt to the differences in image characteristics of different patients and lay a foundation for subsequent accurate segmentation.

[0097] 1.2 Adaptive Risk Attention Module Guided by Risk Factors

[0098] In this application, to ensure that the model can specifically focus on high-risk areas (such as major blood vessels) during the segmentation process, the conventional attention mechanism is improved based on the risk score, and an adaptive risk attention module based on the surgical risk score is designed. This risk attention module enhances the feature extraction intensity of key anatomical structures by dynamically adjusting the attention weights, directly serving to ensure surgical safety. The specific process is as follows:

[0099] (1) Self-Attention Feature Extraction of CT Images

[0100] First, an initial feature map F is extracted from the input three-dimensional medical image, and a query Q (Query) matrix, a key K (Key) matrix, and a value V (Value) matrix are generated through a first-order convolutional layer:

[0101] Q = W q ·F, K = W k ·F, V = Wv ·F;

[0102] where W q , W k , W v is a learnable weight matrix.

[0103] (2) Attention weight assignment mechanism

[0104] The risk attention module dynamically adjusts the attention weights of the feature map according to the risk score R(t). Features in high-risk areas are assigned higher weights to ensure that the tissue and organ segmentation model extracts richer features in these areas. The calculation formula for the attention weights is:

[0105]

[0106] where Q and K are the query and key matrices respectively, A(Q, K) is the attention weight matrix, R(t) is the comprehensive risk function, and dk is the feature dimension. This attention improvement mechanism can dynamically adjust the attention of the organ segmentation model to different anatomical structures. Especially in high-risk areas, the model will automatically enhance the feature extraction intensity. Guided by the risk score, the model can accurately identify and avoid potential risks in a complex surgical environment, providing reliable support for surgical operations.

[0107] (3) Feature weighting

[0108] Use the risk attention module guided by risk factors to weight the V matrix to obtain the enhanced feature map F' and the final feature map F f , and the formula is as follows:

[0109] F' = A(Q, K)·V;

[0110] F f = F + F';

[0111] By dynamically adjusting the attention weights through this risk attention module, the model can specifically focus on those key structures that may affect surgical safety during the segmentation process. The model shows higher accuracy and robustness when dealing with fine anatomical structures such as major blood vessels and the liver. This design not only improves the segmentation accuracy of the model but also provides reliable support for surgical planning and real-time decision-making.

[0112] 1.3 Segmentation model construction

[0113] When constructing the final segmentation model, the present application improves the traditional U-Net segmentation model by introducing the above-mentioned Risk attention module to obtain a new type of RU-Net segmentation model, so as to enhance its model performance in risk warning segmentation. As a key module, the Risk attention module can dynamically adjust the model's attention to different anatomical regions. By embedding the Risk attention module in the encoder and decoder of the traditional U-Net segmentation model, the RU-Net segmentation model is obtained, which can not only capture multi-scale image features, but also highlight the feature information of key risk regions during the feature fusion process. The Risk attention module is used to highlight the detailed features of key anatomical regions in the downsampling path, and the Risk attention module is used to enhance the feature fusion ability of key risk regions in the upsampling path. Through these innovative designs, the RU-Net segmentation model provides more efficient and accurate risk warning segmentation capabilities in the virtual surgical navigation system, providing solid technical support for improving the surgical quality and reducing intraoperative risks.

[0114] As Figure 2 shown is the RU-Net network structure of the present application. The Risk attention module is embedded in the encoder and decoder of the traditional U-Net segmentation model to improve the extraction of detailed features of key risk regions in the downsampling link of the segmentation model. At the same time, features from skip-connection are fused in the upsampling link to achieve selective detail restoration of key regions. Finally, the model focuses on the detail recognition of risk regions under a limited computational load.

[0115] 1.4 Individualized three-dimensional model construction

[0116] CT images, after passing through the segmentation model, output different tissue structures, and then execute a three-dimensional reconstruction algorithm to construct a three-dimensional model to achieve accurate visualization of the target anatomical structure. For the unique anatomical features of each patient, the Marching Cubes algorithm is used for three-dimensional surface reconstruction. By dynamically adjusting the isosurface threshold, it is ensured that the reconstructed model can accurately reflect the morphological characteristics of individual diseased tissues. During the reconstruction process, according to the size, shape and spatial position relationship of the tissue organs of different patients, the grid density and smoothing parameters are adjusted to ensure the reconstruction accuracy while optimizing the calculation efficiency. For the diseased area, local grid refinement technology is adopted to improve the geometric detail performance of key parts; while for non-critical areas, appropriate grid simplification strategies are adopted to achieve reasonable allocation of computing resources. This individualized three-dimensional reconstruction and analysis scheme can fully display the unique pathological features of each patient and provide strong technical support for precision medicine.

[0117] 2. Construction of virtual surgery risk assessment model

[0118] This application proposes a multi-dimensional risk assessment model, where each dimension represents a key factor. By comprehensively considering factors such as organ morphology, the physical distance of the robotic arm, and historical risk accumulation, the risk scores for different organs during the surgical process are calculated comprehensively.

[0119] Distance is the primary factor to consider. The shorter the distance, the higher the risk level. Secondly, the importance of the region is determined by anatomical knowledge and clinical experience. For example, the aorta and major blood vessels are given higher weights. Morphological features include the volume, surface area, and shape complexity of the region, which can affect the difficulty and risk of surgical operations. The historical risk accumulation effect during the surgical process has a greater impact on recent risks. In summary, the comprehensive risk function R(t) represents the comprehensive risk score at time t. This model considers the current state and historical cumulative risks and achieves a comprehensive assessment of the risks of the virtual laparoscopic surgical path through multi-dimensional feature fusion, defined as follows:

[0120] R(t) = α·D(t) + β·P(t) + γ·M(t) + σ·H(t) + λ·Ω(R);

[0121] Among them, α is the fixed weight of the distance factor during the real-time interaction of the virtual surgery, reflecting the importance of the distance between the virtual robotic arm and different tissue organs, and its value ranges from 0.4 to 0.7. β is the fixed weight of the organ importance, reflecting the risk weights of different organs during the surgery. γ represents the fixed weight of the radiomics features of the current organ, as the degree of influence of morphological features on surgical risks. σ is the historical risk assessment weight within a fixed time period during the virtual surgery. λ is the regularization coefficient, used to control the complexity of the model.

[0122] (1) Distance dynamic assessment mechanism

[0123] D(t) represents the distance dynamic assessment function, which is constructed by fully considering the physical distance, relative speed, and acceleration between the virtual robotic arm and the risk region. The formula is as follows:

[0124]

[0125] Among them, L(t) is the actual physical distance between the robotic arm and the risk region at the current time t; ε = 0.001 is a conventional coefficient; μ is the speed influence factor, reflecting the influence degree of the current dynamic speed of the robotic arm on the risk; v is the acceleration influence factor, reflecting the influence degree of the acceleration on the risk. is the basic term, representing the reciprocal of the instantaneous distance between the surgical instrument and the risk region. The shorter the distance, the higher the risk score, showing a non-linear growth. is the first derivative term of distance with respect to time, representing the instantaneous velocity of the robotic arm relative to the risk area. A positive derivative value represents moving away from the risk area, a negative derivative value represents approaching the risk area, and taking the absolute value means that whether approaching or moving away, the rapid movement of the virtual surgical robotic arm will increase the intraoperative risk, which is used to predict potential collision risks. is the second derivative term of distance with respect to time, representing the acceleration of the robotic arm relative to the risk area. This term is used to reflect the degree of sudden change in the motion state. A larger acceleration indicates unstable motion, and taking the absolute value means that any sudden change in any direction increases the risk, aiming to evaluate the stability of the surgical operation and warn of sudden movements.

[0126] This multi-derivative evaluation model designed for the virtual surgery background of this application enables the risk assessment system to not only consider the static distance but also predict and evaluate the risks brought by dynamic movements, dynamically update the risk score, and provide more comprehensive and forward-looking safety protection.

[0127] (2) Abdominal cavity environment organ importance evaluation mechanism

[0128] The abdominal cavity environment involves the interaction of various human tissues and organs. By comprehensively considering the risk levels of different organs and establishing a scientific organ importance grading system, it can provide more accurate risk assessment guidance for surgical operations, thus maximizing surgical safety. The organ importance scoring function of this application can be expressed as:

[0129]

[0130] where a 0 represents the polynomial constant term, a i represents the weight coefficients of each level, and δ i (O) is the indicator function of the organ type. This application divides the abdominal cavity organ types into 4 levels corresponding to four different outputs of δ i (O), specifically: 1) Level 1 (fatal level): aorta, major blood vessels, heart; 2) Level 2 (important level): liver, kidney, spleen; 3) Level 3 (secondary level): gallbladder, appendix; 4) Level 4 (ordinary level): adipose tissue, connective tissue. a i As the weight coefficients of each level are formulated in a descending order, they are used to quantify the risk coefficients of different levels.

[0131] (3) Morphological feature evaluation mechanism

[0132] M(t) comprehensively considers the real-time changes of imaging features such as the volume, surface area, shape change rate, and complexity of the organ, and the impact on the risk coefficient. The formula is as follows:

[0133]

[0134] Among them, M(t) represents the morphological feature evaluation function, t represents time, and w 1 is the volume feature weight of the risk area, V(t) is the time-varying volume change feature, and the time-varying volume refers to the volume of the organ that changes over time. When there is collision or compression, the organ volume will change. Organs with larger volumes may increase the surgical difficulty. The volume change indicates tissue deformation or surgical progress, which affects the operating space of surgical instruments. w 2 is the surface area feature weight of the risk area, A(t) is the surface complexity of the risk area, which is used to reflect the irregularity of the organ surface and indicate possible lesion areas. w 3 is the shape change rate weight, and the integral term is used to capture the dynamic changes of the organ morphology over time, where the shape complexity function is defined as This integral term can be used to cumulatively record the shape change history, reflect the degree of tissue deformation, predict possible shape change trends, and thus achieve faster risk perception. w 4 is the shape complexity weight, describes the geometric features of the organ shape, where P(t) is the projected perimeter and r(t) is the equivalent radius.

[0135] This morphological feature scoring system can comprehensively describe the geometric characteristics of the risk area, capture the dynamic change process, dynamically adjust the risk score, support real-time monitoring and early warning, adapt to different types of surgical scenarios. Through the combination of these features, the system can more accurately evaluate the surgical risk and provide more reliable decision-making support for doctors.

[0136] (4) Historical risk cumulative assessment mechanism

[0137] Considering the cumulative effect of historical risks, the recent risks have a greater impact. The formula is as follows

[0138]

[0139] Among them, H(t) is the historical risk cumulative assessment function, and e -λ(t-τ) is the exponential decay kernel function, which represents the decay effect of historical risks over time. λ > 0 is the decay rate parameter, t is time, τ represents a specific moment in the surgical process, and t - τ is the time difference. A larger λ indicates that historical risks decay faster, and a smaller λ indicates that historical risks have a more long-term impact. In particular, R(τ) = R d (τ) + R m (τ) + R v (τ) + R a (τ) is the comprehensive risk function, which is used to record the instantaneous risk value at the moment of τ, and involves the distance-related risk R d (τ), the morphology-related risk R m (τ), the speed-related risk Rv (τ) and the acceleration-related risk R a (τ). H(t) cumulatively calculates all historical risks from the start of the surgery (τ = 0) to the current moment (τ = t) through integration, comprehensively considers the cumulative risk effects during the surgery process, and generates a comprehensive risk score. This historical risk accumulation mechanism can comprehensively record the risk changes during the surgery process, provide historical-based risk prediction, and achieve the long-term and short-term balance of risks.

[0140] (5) Regularization term

[0141] The fundamental purpose of the regularization term is to control the complexity of the risk assessment model by adding constraint conditions, making the risk assessment in the virtual surgery process more stable and reliable. The complete regularization formula is as follows:

[0142]

[0143] Among them, the first term is the spatial regularization term, and the expression is:

[0144]

[0145] represents the gradients of the risk score R in the x, y, and z directions. The purpose is to accumulate the gradient information of the entire surgery space, ensure the smoothness of the global risk distribution, and comprehensively consider the risk changes at all spatial positions. Ensure that the risk scores in adjacent regions do not mutate, reflecting the gradual change characteristics of the surgical risk in space, which conforms to the continuous distribution characteristics of risks in the actual surgical environment. The second term is the sparse regularization term (L1 regularization), and η is the sparse factor, which is used to control the complexity of the overall model. The sparse regularization strategy of this application is used to automatically select important risk features, ignore minor risk factors, and simplify the risk assessment model. The third term is the direction-sensitive regularization, and the formula is:

[0146]

[0147] By selecting the maximum value of the gradient of the risk assessment score in different directions, it can effectively identify the main risk change directions, analyze the directional characteristics of the operable space, identify the restricted directions, and evaluate the operating freedom.

[0148] Example Two

[0149] This example provides a feedback method for an individualized virtual laparoscopic intraoperative risk feedback model. This method sets a new type of adaptive threshold mechanism, dynamically adjusts the risk level according to the risk changes during the surgery process, and generates corresponding visual feedback according to the risk level, making the visual feedback more sensitive and accurate. Its core adaptive threshold function can be expressed as:

[0150]

[0151] Among them, T i (t) is an adaptive threshold function, where t is time, τ represents the time at a specific moment during the surgical procedure, and R(t) and R(τ) represent the comprehensive risk functions at times t and τ. is the basic threshold, the second term represents the risk accumulation effect during the fixed time period Δt, and the third term reflects the changing trend of the risk. u 1 and u 2 are the weight coefficients of the accumulation effect and the change rate, respectively. This adaptive threshold mechanism can automatically adjust the warning threshold according to different stages and risk characteristics of the surgery, providing more sensitive visual feedback in the stages with higher risks and appropriately reducing the warning intensity in the stages with lower risks.

[0152] This feedback method includes the following steps:

[0153] (1) Set up a new type of adaptive threshold mechanism to dynamically adjust the risk level according to the risk changes during the surgical procedure, and set corresponding visual feedback according to the risk level;

[0154] (2) Map the visual feedback to visual attributes. Map the visual feedback to corresponding visual attributes according to the risk level. Visual attributes include color, transparency, brightness, flicker, etc. Transmit risk information through multi-dimensional visual cue methods. For example, high-risk areas can be mapped to visual effects with deepened color and enhanced stroboscopic effects, and low-risk areas can be presented with lower brightness or light color cues, which not only ensures the effectiveness of the warning but also avoids visual fatigue that may be caused by over-warning, providing more accurate and user-friendly surgical safety protection for doctors;

[0155] (3) Dynamically adjust the visual feedback intensity in real time. The visual feedback intensity includes visual feedback methods such as color change, brightness adjustment, flicker prompt, and contour enhancement, to convey risk information in real time and provide accurate risk warning information to assist surgical operations; Further, establish a feedback mechanism for "proximity warning" in surgical simulation. For high-risk areas exceeding the warning threshold, warning effects such as triggering flicker or contour enhancement can be used to convey risk information in real time. This visual cue method helps the operator be more cautious when approaching key anatomical structures and avoid potential injury risks.

[0156] Further, the visual feedback method includes setting a pop-up warning on the screen. The content of the pop-up warning includes the specific distance d between the surgical instrument and the key part, as well as the specific name of the key part. As shown in Figure 4(A), in the experiment, it is set that the first-, second-, and third-order blood vessels of the hepatic vein and portal vein in the liver all need to be subjected to transparency warning processing. The warning threshold distance is set to 5 mm. During the simulated liver resection surgery, when the tip of the surgical instrument approaches the third-order blood vessel of the hepatic vein, the algorithm adaptively adjusts the transparency of the model, and a pop-up warning with the warning content is displayed at the determined position. In the figure, it is recognized that the distances from two positions (when d is 5 mm and 2 mm) to the warning blood vessel are less than the threshold distance. Therefore, two warning pop-ups are shown in Figure 4(B) for pop-up prompts.

[0157] Further, the transparency of the model can be adjusted in real time by calculating the distance between the surgical instrument and key structures such as important blood vessels.

[0158] Further, a gradual adjustment of the three-dimensional image transparency can be achieved, providing a visual transition hint when the surgical instrument approaches the key area instead of sudden transparency. For example, the transparency of other tissue parts of the model can be adaptively adjusted according to the "degree of danger" of the distance. The closer the distance, the more obvious the transparency adjustment; the farther the distance, the weaker the transparency. That is, the transparency is controlled according to the distance between the surgical tip and the key point, and a smooth transition is achieved during the conversion to avoid visual jumps.

[0159] Further, the transparency adjustment method includes the following steps:

[0160] (1) Construct a risk assessment model, and calculate the real-time comprehensive risk score according to the distance between the virtual robotic arm and the anatomical structure, the importance of the organ, and the morphological characteristics.

[0161] (2) Set the transparency processing threshold. When the distance between the surgical instrument and the high-risk area is less than the preset threshold, trigger the dynamic adjustment of transparency.

[0162] (3) Dynamically adjust the transparency level based on the risk score, adaptively adjust the change range and intensity of transparency. The closer the distance, the more obvious the transparency adjustment; the farther the distance, the weaker the transparency. That is, the transparency is controlled according to the distance between the surgical tip and the key point, so that the transparency effect of the high-risk area is gradually enhanced to improve the surgical operation field of view.

[0163] (4) Set the gradual mechanism of transparency adjustment, and achieve a smooth transition of transparency through gradual changes to avoid visual jumps caused by transparency changes.

[0164] (5) While adjusting the transparency, combine other visual hint information, including color changes, contour enhancement, or flashing effects, to further emphasize the high-risk area.

[0165] Through the above adaptive adjustment mechanism, the transparency of the model tissue part is adaptively adjusted according to different stages of the operation. The closer the distance, the higher the transparency and the more obvious the adjustment. The farther the distance, the lower the transparency, that is, the transparency is controlled according to the distance between the surgical tip and the key point. Ensure the safety and intuitiveness of the surgical process. Note: Transparency refers to the degree to which an object allows light to penetrate, usually expressed as 0% (completely transparent) to 100% (completely opaque). An object with high transparency almost completely transmits light, enabling people to clearly see the object inside or behind it; while an object with low transparency absorbs or reflects most of the light, making it difficult to identify or completely invisible the object inside or behind it.

[0166] Based on the above embodiments, the present application also proposes an individualized virtual laparoscopic intraoperative risk feedback system, which includes:

[0167] An input module for inputting patient individual characteristic parameters, including medical image data (such as CT images), organ morphological characteristics (such as volume, surface area, shape complexity), robotic arm distance parameters, and historical risk parameters accumulated during the operation;

[0168] An output module for outputting the dynamically adjusted risk score, visual feedback parameters, and controlling the visual display module to dynamically adjust the transparency, color, brightness, and flicker frequency of high-risk areas, etc., to assist doctors in identifying and avoiding high-risk areas;

[0169] A control module for generating a comprehensive risk score according to the input individualized characteristics and real-time surgical parameters, controlling the RU-Net segmentation model to dynamically segment the patient's key anatomical structures, and dynamically adjusting the transparency and warning effect of visual cues according to the risk level.

[0170] In addition, the present application also provides a processor-executable program, which can be directly executed by the processor or can be executed by the processor after compilation, and is used to execute the above-mentioned individualized virtual laparoscopic intraoperative risk feedback model construction and feedback method.

[0171] On the other hand, the present application also provides a storage medium, on which a non-volatile program code that can be executed or executed after compilation by the processor is stored, and the program code is used to execute the above-mentioned individualized virtual laparoscopic intraoperative risk feedback model construction and feedback method.

[0172] The above describes specific embodiments of the present application. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0173] In the description of the embodiments of the present application, the descriptions referring to terms such as "one embodiment", "some embodiments", "examples", "specific examples", or "some examples" mean that the specific features, structures, materials, or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the embodiments of the present application. In the embodiments of the present application, the schematic expressions of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described can be combined in any one or more embodiments or examples in a suitable manner. In addition, without contradiction, those skilled in the art can combine and combine the different embodiments or examples described in the embodiments of the present application and the features of different embodiments or examples.

[0174] In addition, the terms "first" and "second" are only used for descriptive purposes and cannot be understood as indicating or implying relative importance or implicitly specifying the quantity of the indicated technical features, excluding any ranking. Thus, the features defined with "first" and "second" can explicitly or implicitly include at least one of such features and are used to distinguish each other. In the description of the embodiments of the present application, the meaning of "a plurality" is at least two, such as two, three, etc., unless otherwise specifically and clearly defined.

[0175] Any process or method description shown in a flowchart or described in other ways herein can be understood as representing a module, segment, or part of code including one or more executable instructions for implementing a customized logic function or process, and the scope of the preferred embodiments of the embodiments of the present application includes additional implementations, where the functions can be executed in a substantially simultaneous manner or in a reverse order according to the functions involved, rather than in the order shown or discussed, which should be understood by those skilled in the art of the embodiments of the present application.

[0176] The above are only the preferred embodiments of the embodiments of the present application and are not intended to limit the embodiments of the present application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the embodiments of the present application shall be included within the scope of protection of the embodiments of the present application.

Claims

1. A method for constructing a risk feedback model for individualized virtual laparoscopy, characterized in that: The following steps are involved: Construct a multidimensional risk assessment model based on individual patient characteristics, comprehensively consider organ morphology, physical distance of the robotic arm, organ clinical risk level and historical risk accumulation, and quantify the risk scores of different organs during virtual laparoscopic surgery; Improve the conventional attention mechanism based on risk scoring and design a risk attention module; The risk attention module is embedded into the encoder and decoder of the traditional U-Net segmentation model to obtain the RU-Net segmentation model; The RU-Net segmentation model is used to achieve organ image segmentation, highlight risk areas, and dynamically generate adaptive transparency adjustment and risk warning feedback.

2. The construction method according to claim 1, characterized in that: The multi-dimensional risk assessment model includes: A dynamic distance assessment mechanism that dynamically updates the risk score based on the physical distance, speed, and acceleration between the robot and the risk area; A morphological feature assessment mechanism that dynamically adjusts the risk score by calculating changes in the volume, surface area, and shape of the target organ; The historical risk accumulation assessment mechanism comprehensively considers the cumulative risk effects during the operation and generates a comprehensive risk score.

3. The construction method according to claim 1, characterized in that: The risk attention module dynamically adjusts the attention weight using the risk score, and the specific implementation steps include: The initial feature map F is extracted from the input 3D medical image, and the query matrix (Q), key matrix (K) and value matrix (V) are generated through a first-order convolutional layer: Q=W q ·F,K=W k ·F,V=W v ·F; Among them, W q ,W k ,W v is a learnable weight matrix; The risk attention module dynamically adjusts the attention weights of the feature maps according to the risk score. Features in high-risk areas are given higher weights to ensure that the segmentation model extracts richer features in these areas. The attention weight matrix is ​​calculated by the following formula: in: A(Q,K): attention weight matrix; R(t): comprehensive risk function; d k : Feature dimension.

4. The construction method according to claim 1, characterized in that: Before the image segmentation step, the input individual patient CT image sequence is preprocessed to specifically enhance the display of the target tissue structure in the individualized case; after the image segmentation step, a three-dimensional reconstruction algorithm is executed to construct a three-dimensional model through a marching cubes algorithm to achieve accurate visualization of the target anatomical structure.

5. The construction method according to claim 2, characterized in that: The morphological characteristics were evaluated by the following formula: in: M(t) represents the morphological feature evaluation function; t represents time; V(t): time-varying volume change characteristics. Time-varying volume refers to the volume of an organ that changes over time. When there is a collision or compression, the volume of the organ will change; w1: volume feature weight of risk area; w2: surface area characteristic weight of the risk area; A(t): is the surface complexity of the risk area, which is used to reflect the irregularity of the organ surface and indicate the possible lesion area; w3 is the shape change rate weight, the integral term It is used to capture the dynamic changes of organ morphology over time, where the shape complexity function is defined as w4 is the shape complexity weight, The geometric features describing the shape of the organ, where P(t) is the projected perimeter and r(t) is the equivalent radius; The distance dynamic evaluation mechanism is calculated by the following formula: in: D(t) is the distance dynamic evaluation function; L(t) is the actual physical distance between the robot arm and the risk area at the current moment t; ε=0.001 is the conventional coefficient; μ is the speed impact factor, which reflects the impact of the current dynamic speed of the robot on the risk; v is the acceleration impact factor, which reflects the impact of acceleration on risk; It is the basic item, representing the inverse of the instantaneous distance between the surgical instrument and the risk area. The shorter the distance, the higher the risk score, which increases nonlinearly. It is the first-order derivative of distance with respect to time, representing the instantaneous speed of the robot relative to the risk area; is the second-order derivative of distance with respect to time, indicating the acceleration of the manipulator relative to the risk area; The historical risk accumulation assessment is calculated using the following formula: in: H(t) is the historical risk accumulation assessment function; t is time, τ represents the time at a specific moment during the operation; e -λ(t-τ) is an exponential decay kernel function, which represents the decay effect of historical risk over time, where t-τ is the time difference and λ is the decay rate parameter; R(τ)=R d (τ)+R m (τ)+R v (τ)+R a (τ) is the comprehensive risk function, which is used to record the instantaneous risk value at time τ, where R d (τ) is the distance-related risk, R m (τ) is the shape-related risk, R v (τ) is the speed-related risk, R a (τ) is the acceleration related risk.

6. A feedback method for a risk feedback model in an individualized virtual laparoscopic operation, characterized in that: The following steps are involved: (1) Setting an adaptive threshold mechanism to dynamically adjust the risk level according to the risk changes during the surgery and generate corresponding visual feedback based on the risk level; (2) Visual feedback maps different risk levels to corresponding visual attributes, including color, transparency, brightness, and flicker; (3) According to the change of risk level, the visual feedback intensity is adjusted dynamically in real time, including transparency change, color switching or flashing prompt, to provide accurate risk warning information to assist surgical operation; Wherein, the transparency adjustment method preferably includes the following steps: (1) Construct a risk assessment model to calculate a real-time comprehensive risk score based on the distance between the virtual robotic arm and the anatomical structure, organ importance, and morphological characteristics; (2) Setting a transparency processing threshold. When the distance between the surgical instrument and the high-risk area is less than the preset threshold, dynamic transparency adjustment is triggered; (3) Dynamically adjust the transparency level based on risk scores and adaptively adjust the scope and intensity of transparency changes; (4) Set up a gradual mechanism for transparency adjustment to achieve a smooth transition of transparency through gradual changes and avoid visual jumps.

7. The feedback method according to claim 6, characterized in that: The adaptive threshold function calculation formula is as follows: in: T i (t) is the adaptive threshold function; t is time, τ represents the time at a specific moment during the operation; R(t) and R(τ) represent the comprehensive risk function at time t and τ; T i 0 is the basic threshold, the second term in the above formula represents the cumulative effect of risk during a fixed time period Δt, the third term in the above formula reflects the changing trend of risk, and u1 and u2 are the weight coefficients of the cumulative effect and the rate of change, respectively.

8. An individualized virtual laparoscopic intraoperative risk feedback system, characterized in that: include: An input module, used to input individual patient characteristic parameters, including medical imaging data, organ morphological characteristics, robotic arm distance parameters, and historical risk parameters accumulated during the operation; An output module is used to output the dynamically adjusted risk score and visual feedback parameters, and control the visual display module to dynamically adjust the transparency, color, brightness and flickering frequency of high-risk areas; The control module is used to generate a comprehensive risk score based on the input individual characteristics and real-time surgical parameters, control the RU-Net segmentation model to dynamically segment the patient's key anatomical structures, and dynamically adjust the transparency and warning effect of the visual prompt according to the risk level.

9. A processor executable program, which can be directly executed by a processor or can be executed by a processor after compilation, and is used to execute the construction method and feedback method of the personalized virtual laparoscopic intraoperative risk feedback model as described in any one of claims 1-7.

10. A storage medium, characterized in that: The storage medium stores non-volatile program code that is executable by a processor or executable after compilation, and the program code is used to execute the construction method and feedback method of the personalized virtual laparoscopic intraoperative risk feedback model as described in any one of claims 1-7.

Images