Navigation method for nursing cooperation of endoscopic submucosal dissection based on augmented reality

Abstract

The application discloses an endoscopic submucosal dissection nursing cooperation navigation method based on augmented reality, relates to the field of augmented reality display technology, and takes impedance change rate of an electric knife as an independent calibration benchmark, converts the impedance change rate into a theoretical chroma offset value through ex vivo experiment calibration mapping relationship, and drives graded correction after cross verification of the measured chroma offset amount of an endoscope image, so that the observed chroma offset amount is close to the theoretical value; boundary step changes are eliminated through a hysteresis interval, and continuous deviation intervals are combined with continuous monitoring to improve and accumulate compensation to deal with continuous deviation; a fusion weight is dynamically allocated according to an interval where impedance is located to generate a thermal injury comprehensive index, and finally, the thermal injury comprehensive index is presented in the form of a dynamic color coating in the field of vision of a nurse through an augmented reality device; the application solves the problem that navigation information is easy to lose in the prior art under visual interference such as bleeding and smoke, and reduces the cognitive burden of nurses through intuitive visual coding.

Description

Technical Field

[0001] This invention relates to the field of augmented reality display technology, specifically to a navigation method for endoscopic submucosal dissection nursing care based on augmented reality. Background Technology

[0002] Endoscopic submucosal dissection (ESD) is a standard minimally invasive procedure for treating early-stage gastrointestinal cancers and precancerous lesions. This procedure involves inserting an endoscope through the natural orifices of the digestive tract and completely removing the diseased tissue from the submucosal layer, which is only 1 to 3 millimeters thick. It offers advantages such as minimal trauma, rapid recovery, and preservation of organ function. However, ESD is a complex and high-risk procedure. Close collaboration between the surgeon and the assisting nurse is crucial. The nurse must continuously monitor the endoscopic images, assess the surgical status based on changes in wound color and bleeding, and promptly perform auxiliary procedures such as instrument delivery and insufflation / inflation. This collaborative approach heavily relies on the nurse's subjective judgment of the endoscopic images and their clinical experience, resulting in significant uncertainty and individual variations.

[0003] Computer-aided techniques have been introduced to improve the accuracy of nursing care, such as marking the lesion area using preoperative imaging data and displaying it on the intraoperative endoscopic screen to help nurses understand the lesion location. Building on this, augmented reality technology is being explored for application in ESD surgery, merging and overlaying preoperative imaging data with intraoperative endoscopic images to display virtual anatomical structures or lesion boundaries on a real-time screen. Simultaneously, research has been conducted on generating risk indication information reflecting tissue status based on real-time intraoperative image data, such as assessing wound condition by analyzing the visual characteristics of tissues in endoscopic images. Furthermore, techniques for jointly analyzing intraoperative sensor data and image information are being explored to provide more comprehensive intraoperative decision-making references.

[0004] However, the aforementioned existing technologies share a common deficiency: the lack of a calibration mechanism independent of visual signals. In these schemes, the registration and overlay of preoperative images and endoscopic images, and the generation of risk indicators based on image features, all rely on endoscopic images as the core data source. Their accuracy depends on the premise that the endoscopic images accurately reflect the tissue state. However, intraoperative bleeding can cover a large area of ​​the wound, masking color signals; smoke from electrocautery cutting can cause image blurring or color temperature shifts; and blood or mucus adhering to the lens tip can severely degrade image quality. When these visual interferences occur, the color information of the endoscopic image becomes unreliable, and any navigation or instruction information generated based on this information becomes distorted. Existing technologies lack a reference mechanism independent of visual signals to determine whether the current information is still reliable, and there is no solution to use non-visual signals to correct or compensate for potentially distorted visual information. Summary of the Invention

[0005] To address the shortcomings of existing technologies, this invention provides a navigation-based nursing care method for endoscopic submucosal dissection based on augmented reality.

[0006] To achieve the above objectives, the technical solution of the present invention is as follows: an augmented reality-based endoscopic submucosal dissection nursing and navigation method, comprising:

[0007] Acquire endoscopic image data and extract the observed chromaticity shift of the wound area; acquire electrosurgical impedance data and calculate the rate of impedance change per unit time.

[0008] Based on the preset mapping relationship, the theoretical chromaticity offset corresponding to the impedance change rate is determined, and the chromaticity difference between the observed chromaticity offset and the theoretical chromaticity offset is calculated.

[0009] The compensation direction is determined based on the sign of the chromaticity difference value. The deviation range into which the chromaticity difference value falls is determined based on the comparison result between the absolute value of the chromaticity difference value and the preset deviation threshold. Different deviation ranges correspond to different compensation values, and the compensation value increases as the deviation range increases. After limiting the compensation value with a preset maximum step size, the observed chromaticity offset is corrected to obtain the corrected chromaticity.

[0010] Based on the numerical range of the impedance change rate, determine the first weight corresponding to the corrected chromaticity and the second weight corresponding to the impedance change rate. After normalizing the corrected chromaticity and the impedance change rate, perform a weighted calculation according to the first weight and the second weight to generate a comprehensive thermal damage index.

[0011] Based on the range of the thermal damage comprehensive index, the corresponding color and flashing frequency are determined, a dynamic color coating is generated, and the result is output to the nurse's augmented reality display device.

[0012] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0013] 1. This invention introduces the impedance change rate of an electrosurgical unit as a physical reference signal independent of the endoscopic image, and converts the impedance change rate into a theoretical chromaticity offset value through a mapping relationship calibrated by an in vitro experiment. This value is then compared with the chromaticity offset actually observed in the endoscopic image to drive correction, thereby ensuring the reliability of augmented reality navigation information under visual interference conditions. This solves the problem of AR information distortion under visual interference caused by the lack of a calibration mechanism independent of visual signals in the prior art.

[0014] 2. This invention processes the sign and absolute value of the chroma difference value separately. The sign determines the correction direction, and the absolute value determines the correction intensity through segmented threshold comparison. This achieves hierarchical correction with separation of direction and intensity, so that the corrected chroma can gradually approach the theoretical value while maintaining inter-frame stability. At the same time, the compensation value is limited by a preset maximum step size to prevent abnormal deviations in a single frame from causing correction overshoot, thus ensuring the safety of the correction process.

[0015] 3. This invention forms a hysteresis band by configuring a hysteresis amount for the deviation threshold, separating the entry and exit boundaries, eliminating the frequent step changes in the compensation deviation range caused by the fluctuation of chromaticity difference values ​​at the threshold boundary, and ensuring the stability of the AR display image; at the same time, through continuous monitoring and deviation range improvement mechanism and cumulative compensation value mechanism, it realizes the qualitative leap of the correction strategy and the quantitative accumulation of correction strength within the same deviation range, respectively, solving the problem of insufficient response of single frame correction to continuous deviation.

[0016] 4. This invention uses the impedance change rate as both a control variable for weight allocation and a fusion input data, achieving weight allocation of chroma and impedance change rate during fusion after correction in different impedance change rate ranges. At low impedance, visual dominance prevails; at medium impedance, balance prevails; and at high impedance, impedance dominance prevails, ensuring that the comprehensive thermal damage index is always dominated by the most reliable data source. At the same time, by unifying the three core parameters—the mapping relationship, the deviation threshold, and the impedance range boundary—from in vitro experiments, a complete parameter system is established, avoiding the problem of mismatch between different parameter sources.

[0017] 5. This invention maps the thermal injury comprehensive index to a dynamic color coating superimposed on the corresponding position of the wound area on the endoscopic image. By using the color gradient from green to red and the flashing frequency gradually increasing from no flashing to high frequency, intuitive visual encoding of risk information is achieved. Combined with the active guidance of dynamic directional arrows and the indispensable presentation of mandatory operation instruction boxes in high-risk situations, nurses can directly complete the conversion from visual information to operational decisions without interpreting any numerical values, reducing cognitive burden and shortening reaction time. The semi-transparent coating design allows nurses to see the real texture of the wound through the coating while observing risk information, without affecting normal surgical operation judgment. Attached Figure Description

[0018] The disclosure of this invention is illustrated with reference to the accompanying drawings. It should be understood that the drawings are for illustrative purposes only and are not intended to limit the scope of protection of this invention. In the drawings, the same reference numerals are used to refer to the same parts. Wherein:

[0019] Figure 1 This is a flowchart of the method of the present invention;

[0020] Figure 2 This is a data processing flowchart of the present invention. Detailed Implementation

[0021] It is readily understood that, based on the technical solution of this invention, those skilled in the art can propose various interchangeable structural methods and implementations without altering the essential spirit of the invention. Therefore, the following detailed embodiments and accompanying drawings are merely illustrative examples of the technical solution of this invention and should not be considered as the entirety of the invention or as limitations or restrictions on the technical solution of this invention.

[0022] like Figure 1 and Figure 2 As shown, the specific implementation steps of the present invention include the following steps:

[0023] Acquire endoscopic image data and extract the observed chromaticity shift of the wound area; acquire electrosurgical impedance data and calculate the rate of impedance change per unit time.

[0024] This step is the data acquisition and preprocessing stage of the entire navigation method. First, real-time image data of the surgical area is acquired using an endoscopic camera, and the dissected wound area is identified from the image. By calculating the chromaticity difference of this area relative to normal tissue, the observed chromaticity shift is output. This data will subsequently serve as the input signal for the correction step. Simultaneously, the tissue impedance value is read in real time by the electrosurgical unit, and its rate of change per unit time is calculated, outputting the impedance change rate. This data serves two purposes: firstly, as input to the preset mapping relationship to calculate the theoretical chromaticity shift value; secondly, its numerical range is used to dynamically adjust the weight allocation during weighted fusion.

[0025] Specifically, endoscopic image data is continuously received according to a preset frame rate. For each frame, the set of pixels corresponding to the dissected wound area is extracted, and the chromaticity value of the pixels in that area is calculated. The preset frame rate ranges from 25 to 30 frames per second. This range covers the conventional output frame rate of clinical endoscopy systems. 25 frames per second corresponds to the PAL standard, and 30 frames per second corresponds to the NTSC standard. Both are standard frame rates in the field of medical imaging and can meet the needs of the human eye for continuous visual perception.

[0026] The chromaticity values ​​are calculated in the Lab color space, which represents color as three channels: lightness channel, red-green axis channel, and yellow-blue axis channel. The Lab color space is chosen because the Euclidean distance between two points in this space is basically consistent with the human eye's perception of color differences. Furthermore, the red-green axis and yellow-blue axis channels are less sensitive to changes in lighting than the RGB space, making them more suitable for scenarios with unstable lighting conditions during surgery.

[0027] For each pixel belonging to the wound area, its red-green axis channel value and yellow-blue axis channel value are extracted, and the average value of the red-green axis channel and the average value of the yellow-blue axis channel of all pixels in the wound area are calculated respectively. Then, these two average values ​​are compared with the chromaticity reference value obtained before surgery to calculate the chromaticity offset of the wound area in the current frame relative to the normal mucosa, that is, the observed chromaticity offset.

[0028] Simultaneously, tissue impedance values ​​are read from the electrosurgical unit at the same sampling frequency as the image frame rate. For each set of consecutive impedance sampling points, the impedance value of the current sampling point is subtracted from the impedance value of the previous sampling point, and then divided by the time interval between the two samplings. The unit of impedance change rate is ohms / second. A positive number indicates that the impedance is rising and the tissue is dehydrating or coagulating. A negative number indicates that the impedance is falling and the tissue moisture is increasing or the electrosurgical contact state is changing. A value close to zero indicates that the tissue state is stable.

[0029] The specific method for obtaining the observed chromaticity shift is as follows:

[0030] Obtain the colorimetric baseline values ​​of the patient's normal mucosal tissue before surgery;

[0031] The endoscopic image data is segmented to extract the area of ​​the dissected wound;

[0032] Obtain the chromaticity value of the pixels within the stripped wound area, and calculate the difference between the chromaticity value and the chromaticity reference value as the observed chromaticity offset.

[0033] This step specifies the acquisition of the observed chromaticity offset, breaking down the extraction process into three logically progressive sub-steps: first, establishing a baseline (chromaticity baseline value); second, determining the range (segmenting and extracting the wound area); and finally, calculating the difference (the difference between the wound chromaticity value and the chromaticity baseline value).

[0034] First, before the surgery begins and before the endoscopic probe touches the lesion area, an image of the normal mucosal area in the current field of view is captured by the endoscopic camera. This normal mucosal area is manually selected by the physician on a touchscreen, with selection criteria including no lesions, no bleeding, no electrocautery marks, and no significant reflections within the endoscopic field of view. The selected area must contain at least 1000 pixels. For all pixels within the selected area, the red-green axis channel values ​​and yellow-blue axis channel values ​​in the Lab color space are extracted, and their arithmetic mean is calculated as the patient's chromaticity reference value. This chromaticity reference value comprises two components: the reference value for the red-green axis channel and the reference value for the yellow-blue axis channel.

[0035] If no usable normal mucosal area is available in the endoscopic field of view, the physician can manually load previously archived normal mucosal images of the patient for chromaticity reference value calibration. The chromaticity reference value remains valid throughout the procedure. If the physician determines that the reference value has become invalid due to endoscope replacement or significant changes in the field of view, recalibration can be triggered through the user interface.

[0036] Then, semantic segmentation is performed on each frame of the endoscopic image. The segmentation model adopts the U-Net architecture, which is pre-trained using a labeled endoscopic image dataset and deployed to the system processor after training. The U-Net model classifies each pixel in the input image and outputs a classification mask map of the same size as the input image. The set of pixels marked as wound regions in the mask map is the stripped wound region of the current frame. If multiple discrete connected components appear in the segmentation result, the system selects the connected component with the largest area as the main wound region and excludes isolated small regions with an area of ​​less than 500 pixels.

[0037] Finally, the red-green axis channel values ​​and yellow-blue axis channel values ​​of each pixel within the wound area are extracted. The arithmetic mean of the red-green axis channels and the arithmetic mean of the yellow-blue axis channels for all pixels within the area are calculated. The difference between the measured average value of the red-green axis channels and the reference value of the red-green axis channels is calculated to obtain the red-green axis direction difference; the difference between the measured average value of the yellow-blue axis channels and the reference value of the yellow-blue axis channels is calculated to obtain the yellow-blue axis direction difference; the squares of the red-green axis direction difference and the yellow-blue axis direction difference are then added together and the square root is taken. The result is the observed chromaticity shift.

[0038] This step objectively quantifies the chromaticity shift of the wound by extracting the chromaticity value of the dissected area from the endoscopic image and comparing it with the preoperative chromaticity reference value. This provides quantifiable input data for subsequent correction steps, avoiding the uncertainty caused by relying on the doctor's subjective judgment. By synchronously acquiring the electrosurgical impedance change rate, a physical reference dimension independent of the visual signal is established. This impedance change rate is used in subsequent steps to both calculate the theoretical chromaticity shift value and dynamically adjust the weight allocation during weighted fusion, making the entire method's decision-making dependent on information from both visual and physical dimensions. Synchronizing the data from the image acquisition and impedance acquisition channels in time ensures that each frame of image data has corresponding impedance data as paired input, providing a time-aligned data foundation for subsequent inter-frame comparison and temporal analysis. By calculating the chromaticity shift of the red-green axis and yellow-blue axis channels in the Lab color space, the impact of intraoperative lighting conditions on chromaticity measurement is effectively reduced, improving the stability and reliability of the chromaticity shift in the surgical environment. By setting a minimum area threshold for the wound area and excluding isolated areas, the contamination of chromaticity statistics by local interference sources during surgery is effectively eliminated, improving the data reliability of the observed chromaticity shift.

[0039] In practical implementation, the color space can be replaced with the HSV color space, where the offsets of the hue and saturation channels can be calculated, which can also characterize tissue color changes. The wound area segmentation method can be replaced with segmentation based on color thresholds, edge detection, or region growth. The quantization method for chromaticity offset can be replaced with using only the offsets of the yellow and blue axis channels, or using the standard color difference calculation formula. The calculation method for impedance change rate can be replaced with the sliding window average slope method, i.e., taking impedance values ​​from multiple past frames and performing linear fitting, using the slope of the fitted line as the impedance change rate at the current moment. The image frame sampling method can be replaced with frame extraction, i.e., performing a complete extraction process every preset number of frames, with intermediate frames using the most recent valid observation value. The method for obtaining the chromaticity reference value can be replaced with extraction from previously archived endoscopic images of the patient, or retrieving the reference chromaticity value for the anatomical site from a standard chromaticity database.

[0040] Based on the preset mapping relationship, the theoretical chromaticity offset corresponding to the impedance change rate is determined, and the chromaticity difference between the observed chromaticity offset and the theoretical chromaticity offset is calculated.

[0041] This step first uses the impedance change rate, through a pre-established mapping relationship, to calculate the theoretical chromaticity shift that should be observed under the current impedance change rate. This theoretical value is then compared with the actually observed chromaticity shift, and the difference is calculated to obtain the chromaticity difference value. This chromaticity difference value represents the degree of inconsistency between the chromaticity deviation actually observed by the endoscope and the chromaticity deviation predicted by the impedance data. This difference value is directly input into the correction step, driving the correction logic to correct the direction and magnitude of the observed chromaticity shift.

[0042] Specifically, during the procedure, for each image frame acquired, the corresponding impedance change rate is also acquired. Using the current impedance change rate as an index, a preset mapping relationship is consulted to obtain the corresponding chroma offset, which is the theoretical chroma offset for the current frame. If the current impedance change rate falls between two adjacent index values ​​in the mapping relationship lookup table, the system uses linear interpolation to calculate the corresponding theoretical chroma offset. The specific interpolation method is as follows: determine the two adjacent index values ​​of the current impedance change rate in the lookup table, take the two chroma offsets corresponding to these two index values, and calculate the corresponding chroma offset proportionally according to the relative position of the current impedance change rate between the two index values. If the current impedance change rate exceeds the coverage range of the preset mapping relationship, the system uses the chroma offset corresponding to the boundary value of the coverage range as the theoretical chroma offset and generates a prompt message.

[0043] The chromaticity difference value is obtained by subtracting the theoretical chromaticity shift from the observed chromaticity shift in the same frame. This difference can be positive or negative. If the difference is positive, it means that the observed chromaticity shift is greater than the shift predicted by the impedance, that is, the yellowness of the image is greater than expected; if it is negative, it means that the observed chromaticity shift is less than the shift predicted by the impedance, that is, the yellowness of the image is less than expected.

[0044] The specific method for establishing the preset mapping relationship is as follows:

[0045] Multiple sets of sample data were acquired during the electrosurgical cutting process of ex vivo tissue samples. Each set of sample data included the synchronously acquired impedance change rate and sample chromaticity offset.

[0046] Using the impedance change rate as the independent variable and the sample chromaticity shift as the dependent variable, data fitting is performed to obtain the correspondence between the impedance change rate and the theoretical chromaticity shift.

[0047] Store the correspondence as the preset mapping relationship;

[0048] The theoretical chromaticity offset value is the offset value obtained during the intraoperative application stage based on the current impedance change rate through the mapping relationship. The sample chromaticity offset value is the offset value obtained during the calibration stage through actual measurement of ex vivo tissue samples. Both are manifestations of the same physical quantity at different stages.

[0049] This step is a specific limitation on the preset mapping relationship establishment method, breaking down the mapping relationship establishment process into four stages: obtaining synchronous data of ex vivo tissue samples, fitting with impedance change rate as independent variable and chromaticity offset as dependent variable, storing the fitting results as mapping relationship, and explaining the relationship between the calibration stage and the intraoperative application stage.

[0050] The preset mapping relationship is established as follows: Through ex vivo tissue experiments, electrocautery is applied to the ex vivo tissue sample for cutting, and impedance change rate data is collected during the cutting process. Simultaneously, the chromaticity shift of the sample in the corresponding cutting area is measured, obtaining multiple sets of synchronously corresponding data points. The ex vivo tissue experiment employs conventional ex vivo tissue electrocautery cutting methods in the art, and the electrocautery model used is consistent with the model used during the surgical application phase to ensure the validity of the calibration data during the application phase. The synchronous acquisition of impedance change rate and sample chromaticity shift uses conventional data acquisition methods in the art; specific experimental parameters will not be elaborated here.

[0051] Using the rate of change of impedance as the independent variable and the synchronously acquired chromaticity shift as the dependent variable, all experimental data points were plotted on a two-dimensional coordinate system. Curve fitting was performed on the scatter distribution to obtain the correspondence between the rate of change of impedance and the sample chromaticity shift. This correspondence was stored in the system memory as a lookup table, indexed by the rate of change of impedance, storing the corresponding theoretical chromaticity shift.

[0052] The sample chromaticity shift is the shift value obtained through actual measurement of ex vivo tissue samples during the calibration phase, while the theoretical chromaticity shift is the shift value obtained during the intraoperative application phase based on the current rate of change of impedance through a mapping relationship. Both represent the same physical quantity in different stages; the calibration phase measures the actual observed value of this physical quantity, while the intraoperative application phase obtains the predicted value. The two are correlated through a mapping relationship.

[0053] This step establishes a mapping relationship between impedance change rate and chromaticity shift in advance through ex vivo tissue experiments. This allows the intraoperative impedance change rate to be converted into a chromaticity reference value independent of the endoscopic image, providing a non-visual benchmark for subsequent correction steps. The observed chromaticity shift is compared with the theoretical chromaticity shift, generating a unified difference signal that simultaneously reflects visual and physical information, serving as the sole driving input for subsequent correction steps. When the impedance change rate falls between adjacent index values ​​in the lookup table, linear interpolation is used to ensure the continuity of the theoretical chromaticity shift; when it exceeds the coverage range, the boundary value is used as the theoretical value and a prompt is generated to ensure the system's usability in extreme cases. Through the correspondence between the calibration stage and the intraoperative application stage, the ex vivo experimental data is transformed into a mapping relationship that is usable in real-time during surgery.

[0054] In practice, the fitting method for the mapping relationship can be replaced by linear regression, multinomial regression, or piecewise linear interpolation. The storage format of the mapping relationship can be replaced by storing the function expression of the fitted curve, with the theoretical chromaticity shift calculated directly by substituting the impedance change rate during surgery. The establishment method of the mapping relationship can be replaced by real-time intraoperative calibration, with synchronous data collected during the initial safe cutting phase of surgery to establish the mapping relationship online. The calculation method for chromaticity difference values ​​can be replaced by normalization processing, mapping the difference to a fixed interval for subsequent unified threshold calibration.

[0055] The compensation direction is determined based on the sign of the chromaticity difference value. The deviation range into which the chromaticity difference value falls is determined based on the comparison result between the absolute value of the chromaticity difference value and the preset deviation threshold. Different deviation ranges correspond to different compensation values, and the compensation value increases as the deviation range increases. After limiting the compensation value with a preset maximum step size, the observed chromaticity offset is corrected to obtain the corrected chromaticity.

[0056] This step is the core correction stage of the entire navigation method, responsible for correcting the observed chromaticity offset. The chromaticity difference value represents the degree of deviation between visual observation and impedance expectation. This step determines how to correct the observed chromaticity offset based on the sign and magnitude of the difference value.

[0057] Specifically, the compensation direction is determined by obtaining the chromaticity difference value and judging whether its sign is positive or negative. If the chromaticity difference value is positive, it means that the observed chromaticity offset is greater than the theoretical chromaticity offset, and the observed chromaticity offset needs to be corrected in a decreasing direction, i.e., the compensation direction is negative. If the chromaticity difference value is negative, it means that the observed chromaticity offset is less than the theoretical chromaticity offset, and the observed chromaticity offset needs to be corrected in an increasing direction, i.e., the compensation direction is positive.

[0058] The deviation range is determined by obtaining the absolute value of the chromaticity difference value and comparing it with a preset deviation threshold. The preset deviation threshold contains multiple values, dividing the possible range of chromaticity difference values ​​into several continuous intervals. The larger the absolute value of the chromaticity difference value, the higher the interval number it falls into, indicating a more severe degree of deviation.

[0059] The rules for applying compensation values ​​are based on different compensation values ​​corresponding to different deviation ranges. The higher the range number, the larger the corresponding compensation value, that is, the more serious the deviation and the greater the correction force. The corresponding compensation value is selected according to the range to which the current chromaticity difference value belongs.

[0060] The amplitude limiting process limits the compensation value by setting a preset maximum step size. If the absolute value of the compensation value is less than or equal to the preset maximum step size, the original value remains unchanged; if the absolute value of the compensation value is greater than the preset maximum step size, the absolute value of the compensation value is truncated to the preset maximum step size.

[0061] The preset maximum step size is set during system initialization. Its dimensions are the same as the chromaticity offset, and its value ranges from 5 to 30 chromaticity units, with 15 chromaticity units being preferred. The specific setting of the preset maximum step size is based on clinical experience. The core principle is that the maximum step size should be less than the first deviation threshold to ensure that the amount of correction in a single instance does not exceed the upper limit for judging mild deviation, and to avoid a single frame correction spanning multiple deviation intervals. If the maximum step size is set too large and exceeds the first deviation threshold, even if it is within the mild deviation interval, a single correction may push the observed value beyond the theoretical value.

[0062] Finally, the compensation value after limiting is applied to the observed chromaticity offset according to the determined compensation direction. If the compensation direction is negative, the compensation value is subtracted from the observed chromaticity offset; if the compensation direction is positive, the compensation value is added to the observed chromaticity offset. After the correction is performed, the corrected chromaticity is obtained.

[0063] The specific steps for determining the compensation direction based on the sign of the chromaticity difference value are as follows:

[0064] When the chromaticity difference value is positive, the compensation direction is determined to be negative, and the negative direction is used to indicate that the observed chromaticity offset is reduced and corrected.

[0065] When the chromaticity difference value is negative, the compensation direction is determined to be positive, and the positive direction is used to indicate that the observed chromaticity offset is increased and corrected.

[0066] This step specifies how to determine the compensation direction based on the sign of the chromaticity difference value, clarifying the one-to-one mapping relationship between positive signs and negative corrections, and vice versa. The chromaticity difference value itself is a signed numerical value, but its sign has a completely different function from its absolute value. The absolute value determines the intensity of the correction, while the sign determines the direction of the correction.

[0067] Specifically, when the chromaticity difference value is positive, the observed chromaticity shift is greater than the theoretical chromaticity shift, indicating that the yellowish tint seen by the endoscope is higher than the level predicted by the impedance data. In this case, the observed value needs to be corrected in the direction of reduction so that it falls back to the theoretical value. Therefore, the compensation direction is determined to be negative.

[0068] When the chromaticity difference value is negative, the observed chromaticity shift is less than the theoretical chromaticity shift, indicating that the yellowish tint seen by the endoscope is lower than the level predicted by the impedance data. In this case, the observed value needs to be corrected in the direction of increase to raise it to the theoretical value; therefore, the compensation direction is determined to be positive.

[0069] When the chromaticity difference is zero, the observed value is consistent with the theoretical value, there is no deviation between the two, and no correction is required.

[0070] Specifically, the deviation range into which the chromaticity difference value falls is determined as follows:

[0071] The preset deviation threshold includes a first deviation threshold and a second deviation threshold, and the first deviation threshold is less than the second deviation threshold;

[0072] When the absolute value of the chromaticity difference value is less than or equal to the first deviation threshold, it is determined that the chromaticity difference value falls into the first deviation range;

[0073] When the absolute value of the chromaticity difference value is greater than the first deviation threshold and less than or equal to the second deviation threshold, it is determined that the chromaticity difference value falls into the second deviation range.

[0074] When the absolute value of the chromaticity difference is greater than the second deviation threshold, the chromaticity difference is determined to fall into the third deviation range.

[0075] This step specifies the method for dividing the deviation intervals, sets two deviation thresholds, divides the degree of deviation into three intervals, and clarifies the judgment conditions for each interval.

[0076] The absolute value of the chromaticity difference is a continuously changing numerical value, theoretically allowing for an infinitely refined continuous compensation scheme. However, continuous schemes have problems in practical applications. Intraoperative endoscopic images contain random noise, and the chromaticity difference value fluctuates slightly in each frame. If the compensation value continuously changes with each frame's fluctuation, the corrected chromaticity will jitter slightly in each frame, causing continuous color shifts in the image, severely affecting the nurse's judgment of the true color of the wound. Therefore, this scheme discretizes the degree of continuous deviation into three fixed deviation intervals, each corresponding to a fixed compensation value. As long as the interval boundaries are not crossed, the compensation value remains constant, avoiding the image shift problem in continuous schemes. The design of the three intervals corresponds to three levels of correction for mild, moderate, and severe deviations.

[0077] The preset deviation threshold was obtained through in vitro tissue experiments. The core logic of the calibration is as follows: the in vitro tissue sample is cut with an electrocautery knife, and after the cutting is completed, the vertical distance from the bottom of the wound to the upper edge of the muscle layer at each test point is measured, that is, the residual thickness of the submucosal layer. The degree of thermal damage is characterized by the ratio of the residual thickness to the original thickness.

[0078] All test points were divided into three groups according to the proportion of residual thickness: the first group consisted of test points with a residual thickness greater than or equal to two-thirds of the original thickness; the second group consisted of test points with a residual thickness greater than or equal to one-third but less than two-thirds of the original thickness; and the third group consisted of test points with a residual thickness less than one-third of the original thickness.

[0079] The distribution of chromaticity difference values ​​for each group is statistically analyzed. The maximum value of chromaticity difference values ​​of all test points in the first group is taken as the first deviation threshold, and the maximum value of chromaticity difference values ​​of all test points in the second group is taken as the second deviation threshold. The method of taking the upper limit value instead of the average value is to ensure that the interval boundary of the subsequent segmented correction can cover all cases of the group, and to avoid the deviation state that should be classified into the group being classified into a lower group due to the boundary being too low.

[0080] The first deviation threshold is 10 to 20 chromaticity units, preferably 15 chromaticity units, and the second deviation threshold is 25 to 45 chromaticity units, preferably 35 chromaticity units. The specific values ​​vary depending on the tissue type and electrosurgical parameters, and are set based on the preoperative calibration results during system initialization.

[0081] The above grading method maps the histopathological indicator of thermal damage depth to a color difference value, an indicator that can be obtained in real time during surgery, through in vitro experiments. The grading is based on the residual tissue thickness; the greater the residual thickness, the shallower the thermal damage, and the smaller the corresponding color difference value; the smaller the residual thickness, the deeper the thermal damage, and the greater the corresponding color difference value.

[0082] The three groups correspond to three clinical degrees of thermal injury: mild (only surface protein denaturation), moderate (coagulative necrosis of tissue but without penetration of the submucosa), and severe (full-thickness coagulative necrosis). Two thresholds divide the possible range of chromaticity differences into three consecutive intervals: the first interval corresponds to mild deviation, using a small compensation value for conservative correction; the second interval corresponds to moderate deviation, using a moderate compensation value for routine correction; and the third interval corresponds to severe deviation, using a large compensation value for strong correction.

[0083] The calibration process described above is completed offline before surgery, without taking up intraoperative time, and the calibration results are pre-loaded into the system as parameters. During the intraoperative application phase, the system only performs threshold comparison operations and does not involve any calibration steps.

[0084] Determining the deviation range into which the chromaticity difference value falls also includes:

[0085] The first deviation threshold corresponds to the first lag amount, and the second deviation threshold corresponds to the second lag amount;

[0086] When the absolute value of the chromaticity difference increases from less than or equal to the first deviation threshold to greater than the sum of the first deviation threshold and the first hysteresis, it is determined that the chromaticity difference value has entered the second deviation interval from the first deviation interval.

[0087] When the absolute value of the chromaticity difference value decreases from greater than the first deviation threshold to less than or equal to the difference between the first deviation threshold and the first hysteresis, it is determined that the chromaticity difference value returns from the second deviation interval to the first deviation interval;

[0088] When the absolute value of the chromaticity difference increases from less than or equal to the second deviation threshold to greater than the sum of the second deviation threshold and the second hysteresis, it is determined that the chromaticity difference value has entered the third deviation interval from the second deviation interval.

[0089] When the absolute value of the chromaticity difference value decreases from greater than the second deviation threshold to less than or equal to the difference between the second deviation threshold and the second hysteresis, it is determined that the chromaticity difference value returns from the third deviation interval to the second deviation interval.

[0090] This step, based on determining the deviation range into which the chromaticity difference value falls, further introduces a hysteresis interval mechanism. In actual surgery, endoscopic images contain random noise and slight fluctuations. The absolute value of the chromaticity difference value may oscillate around the threshold, causing the interval determination to change abruptly each frame. The compensation value then switches back and forth between two deviation ranges, resulting in abrupt changes in chromaticity between the two states after correction. This abrupt change is more visually disruptive to nurses than the continuous jitter in a continuous approach. By configuring a hysteresis amount for each threshold, the boundaries of entry and exit are separated. Entry requires exceeding the threshold plus the hysteresis amount, while exit requires falling below the threshold minus the hysteresis amount. The area between these two boundaries is the hysteresis band. When the absolute value fluctuates within this band, the interval determination remains unchanged, avoiding the abrupt boundary changes caused by a single-point threshold.

[0091] Specifically, during initialization, a first hysteresis amount is configured for the first deviation threshold and a second hysteresis amount is configured for the second deviation threshold. The hysteresis amounts are all positive values ​​and are less than the corresponding deviation thresholds.

[0092] The specific value of the hysteresis is determined by analyzing the noise characteristics of the chromaticity signal in the intraoperative endoscopic image. In the ex vivo experiment, the inter-frame fluctuation amplitude of the chromaticity difference value under non-electrosurgical cutting conditions is measured and used as a reference value for the system noise level. The hysteresis is set as a preset multiple of the noise amplitude, with a value ranging from 1.5 to 2.5 times, preferably 2.0 times.

[0093] Meanwhile, the percentage of the corresponding deviation threshold is used as a reference for verification, with the percentage of hysteresis relative to the corresponding deviation threshold ranging from 10% to 25%. During calibration, the initial value of the hysteresis is first calculated using the noise amplitude multiple method, and then the percentage of this value relative to the corresponding deviation threshold is calculated to confirm whether it falls within the percentage range.

[0094] When the results calculated by the two methods are inconsistent, the noise amplitude ratio method, which is based on the actual noise level measurement, should be used as the primary basis for determination. This is because this method is directly derived from noise measurements in the actual working environment of the system and is highly targeted. The deviation threshold percentage is only used as a reference range for reasonableness verification, to confirm that the hysteresis is not too small to cover the noise or too large to excessively compress the hysteresis band and affect the response speed.

[0095] When determining the interval, the system needs to obtain the absolute value of the current frame and the absolute value of the previous frame, and determine the direction of change of the current value by comparing the two.

[0096] When the absolute value changes in the direction of increase: the condition for entering the second interval from the first interval is that the absolute value is greater than the first deviation threshold plus the first hysteresis; the condition for entering the third interval from the second interval is that the absolute value is greater than the second deviation threshold plus the second hysteresis.

[0097] When the absolute value changes in a decreasing direction: the condition for returning from the second interval to the first interval is that the absolute value is less than or equal to the first deviation threshold minus the first lag amount; the condition for returning from the third interval to the second interval is that the absolute value is less than or equal to the second deviation threshold minus the second lag amount.

[0098] This comparison is only used to determine the direction of change of the current frame relative to the previous frame. After the direction is determined, the interval determination result and subsequent correction are still only for the current frame.

[0099] Before determining the compensation value, the deviation range is also increased, specifically:

[0100] Obtain the deviation interval determination results of multiple consecutive frames. When the deviation interval determination results of a preset number of consecutive frames all belong to the first deviation interval, upgrade the current deviation interval to the second deviation interval.

[0101] When the deviation interval determination results of a consecutive preset number of frames all belong to the second deviation interval, the current deviation interval is upgraded to the third deviation interval;

[0102] If the deviation interval determination results of a consecutive preset number of frames all belong to the third deviation interval, no boosting will be performed.

[0103] This step introduces a timing monitoring mechanism. The preceding direction determination and interval division were performed on a single-frame basis, with each frame independently determined and corrected, and no correlation between frames. This single-frame independent processing method has shortcomings when dealing with persistent deviations. If the deviation remains within a certain interval for multiple consecutive frames, it indicates that the compensation strength of the current deviation interval is insufficient to eliminate the deviation. However, the single-frame correction mechanism starts correcting from the same deviation interval each frame and does not automatically switch to a higher deviation interval, causing system response lag. By monitoring the interval determination results of multiple consecutive frames, when the deviation persists, the compensation value is proactively increased to an adjacent higher deviation interval, enabling the system to adaptively upgrade the correction strategy when the compensation strength is insufficient.

[0104] Specifically, a sliding buffer is maintained in memory to store the deviation interval determination results of the most recent N frames, where N is a preset number, ranging from 3 to 10 frames, preferably 5 frames.

[0105] After each frame completes the interval determination, the determination result of the current frame is stored in the buffer, while the oldest frame is removed, always keeping the buffer storing the results of the most recent N frames. Before performing correction in each frame, the system checks whether the determination results of the N frames in the buffer are completely identical. If they are completely identical, the deviation interval is increased; otherwise, the current compensation value remains unchanged.

[0106] The specific rules for increasing the deviation range are as follows:

[0107] When all N frames in the buffer are determined to be in the first deviation range, it means that the slight deviation has persisted but has not disappeared on its own. The compensation value corresponding to the first deviation range is increased to the compensation value corresponding to the second deviation range, that is, intervention is carried out with a greater correction force.

[0108] When all N frames in the buffer are determined to be in the second deviation interval, it means that the moderate deviation has persisted and the compensation value corresponding to the second deviation interval has failed to effectively eliminate the deviation. The compensation value corresponding to the second deviation interval is then increased to the compensation value corresponding to the third deviation interval.

[0109] When all N frames in the buffer are determined to be in the third deviation interval, since the compensation value corresponding to the third deviation interval is already the highest correction level, there is no higher compensation value available for improvement, so the compensation value corresponding to the third deviation interval is used for correction.

[0110] The deviation range boost does not permanently change the compensation value corresponding to the deviation range to which the current frame belongs. Instead, when it is determined that the current frame falls into a certain deviation range and the deviation state of that range persists for a sufficiently long time (i.e., the condition of N consecutive frames is met), the compensation value of that range is temporarily boosted to the compensation value corresponding to the adjacent higher range. After the deviation range boost, if subsequent frames no longer meet the deviation range boost condition, the compensation value immediately reverts to the normal compensation value corresponding to the deviation range of the current frame.

[0111] The determination of the compensation value is specifically as follows:

[0112] The first deviation interval corresponds to the first compensation value, the second deviation interval corresponds to the second compensation value, and the third deviation interval corresponds to the third compensation value;

[0113] Obtain a preset benchmark value, wherein the preset benchmark value is a dimensionless proportionality coefficient;

[0114] Determine the first median value of the first deviation interval, the second median value of the second deviation interval, and the third median value of the third deviation interval. The first median value is half of the first deviation threshold, the second median value is half of the sum of the first deviation threshold and the second deviation threshold, and the third median value is half of the sum of the second deviation threshold and the preset maximum range of chromaticity difference value.

[0115] The first compensation value is the product of the first median and the preset benchmark value; the second compensation value is the product of the second median and the preset benchmark value; and the third compensation value is the product of the third median and the preset benchmark value.

[0116] The second compensation value is greater than the first compensation value, and the third compensation value is greater than the second compensation value.

[0117] This step specifies the method for determining the compensation value corresponding to each deviation interval. The compensation value for each level is obtained by multiplying the midpoint of the interval by a preset benchmark value. The midpoint of the interval represents the typical deviation level of that deviation interval, and multiplying it by the benchmark value yields the compensation intensity corresponding to that typical level. Since the midpoints of the three intervals naturally increase, the compensation values ​​for the three levels naturally increase after multiplying by the same benchmark value. Only one parameter needs to be preset for the benchmark value, and the compensation value will be automatically recalculated when the threshold is adjusted.

[0118] Specifically, the preset benchmark value is a dimensionless proportionality coefficient used to characterize the compensation ratio corresponding to a unit deviation, that is, the proportion of compensation required for each unit of chromaticity deviation to the deviation value. The specific value of the benchmark is obtained through closed-loop calibration in in vitro tissue experiments: in the in vitro experiment, the tissue sample is cut with an electrocautery knife of known intensity, and the synchronous chromaticity difference value is recorded. Then, compensation values ​​are calculated using different benchmark values, ranging from 0.1 to 0.8, with a step size of 0.05. After applying compensation, the degree of closeness between the corrected chromaticity and the theoretical chromaticity offset is observed, and the benchmark value that minimizes the deviation between the two is selected as the preferred value.

[0119] Calculate the median values ​​for the three deviation intervals: the first median is half the first deviation threshold, representing the typical deviation level of the mild deviation interval; the second median is half the sum of the first and second deviation thresholds, representing the typical deviation level of the moderate deviation interval; and the third median is half the sum of the second deviation threshold and the preset maximum range of chromaticity difference values, representing the typical deviation level of the severe deviation interval. Multiply each median value by a preset benchmark value to obtain the compensation value for each interval.

[0120] After determining the compensation value, the process also includes determining the correction amount through the cumulative compensation value, specifically:

[0121] Get the cumulative compensation value of the previous frame;

[0122] The compensation value generated in the current frame is added to the cumulative compensation value of the previous frame to obtain the correction amount for the current frame;

[0123] The observed chromaticity offset is corrected using the correction amount of the current frame to obtain the corrected chromaticity;

[0124] The correction amount of the current frame is stored as the cumulative compensation value of the current frame for use in the next frame;

[0125] When the absolute value of the chromaticity difference is less than or equal to the first deviation threshold for a consecutive preset number of frames, the cumulative compensation value is reset to the initial value.

[0126] This step introduces a cross-frame timing control mechanism. The compensation value generated in the current frame is superimposed with the cumulative compensation value carried over from the previous frame to form the cumulative compensation value for the current frame. This cumulative compensation value is then used to correct the observed chromaticity shift. Because the cumulative compensation value is continuously propagated between frames, the correction effect has inertia. As the deviation persists, the cumulative correction amount increases frame by frame, and the correction strength becomes increasingly stronger. After the deviation is eliminated, the cumulative compensation value is reset to its initial value when it is within a safe range for several consecutive frames, and the correction strength returns to zero. This upgrades the correction process from independent for each frame to cross-frame correlation, making the change in correction strength smoother and more continuous.

[0127] The difference between this step and the continuous monitoring and deviation range improvement step is that the continuous monitoring and deviation range improvement step switches at the deviation range level, jumping from the first deviation range to the second deviation range, while this step accumulates at the numerical level, increasing the compensation value frame by frame within the same deviation range. Their mechanisms differ; the former changes the deviation range to which the compensation value belongs, while the latter changes the magnitude of the compensation value, and they do not conflict.

[0128] Specifically, the cumulative compensation value is a variable maintained by the system to store the total correction amount after frame-by-frame superposition up to the current frame. The initial value of the cumulative compensation value is zero when the system starts.

[0129] After calculating the compensation value for each frame, the cumulative compensation value stored in the previous frame is simultaneously read from memory. The two values ​​are then superimposed to obtain the cumulative compensation value for the current frame. The superposition operation is algebraic addition, with the sign determined by the compensation direction. If the compensation value generated in the current frame is negative, it indicates that the observed chromaticity offset needs to be corrected in the decreasing direction. In this case, the cumulative compensation value for the current frame is the cumulative compensation value of the previous frame minus the absolute value of the current frame's compensation value. If the compensation value is positive, the cumulative compensation value of the previous frame is added to the absolute value of the current frame's compensation value. Regardless of the direction, algebraic addition is used, with the sign automatically handled. When the deviation direction remains constant, the cumulative compensation value accumulates frame by frame in that direction, gradually increasing the correction strength. When the deviation direction changes, the cumulative compensation value regresses in the opposite direction, and the correction strength weakens accordingly, enabling the system to adaptively follow changes in the deviation direction.

[0130] Due to random noise in intraoperative endoscopic images, the chromaticity difference value of a single frame may occasionally fall into the safe range, but then return to the deviation range in the next frame. If a single-frame reset is used, the cumulative compensation value will frequently be cleared to zero due to noise interference, causing the correction process to be interrupted. Therefore, a preset number of consecutive frames must meet the condition to ensure that the deviation has been truly eliminated before resetting the cumulative compensation value, avoiding false triggering caused by noise. By monitoring the absolute value of the chromaticity difference value in real time, if a preset number of consecutive frames are less than or equal to the first deviation threshold (ranging from 3 to 10 frames, preferably 5 frames), it indicates that the deviation has been consistently and stably within the safe range, and the system resets the cumulative compensation value to its initial value of zero. After resetting, the correction intensity is zero, and the system restarts accumulation from the initial state. If the chromaticity difference value in the next frame after the reset falls into the deviation range again, the cumulative compensation value starts accumulating again from zero.

[0131] To prevent the cumulative compensation value from increasing indefinitely during prolonged correction and causing uncontrolled correction, the system sets an upper limit on the absolute value of the cumulative compensation value. This upper limit is related to the magnitude of the current chromaticity difference value. The upper limit of the cumulative compensation value is a preset proportional coefficient of the absolute value of the current chromaticity difference value, with a value ranging from 1.5 to 3, preferably 2. That is, the absolute value of the cumulative compensation value must not exceed twice the absolute value of the current chromaticity difference value. When the absolute value of the cumulative compensation value exceeds this upper limit, the system truncates the cumulative compensation value to this upper limit value.

[0132] This step achieves multi-dimensional, hierarchical correction of chromaticity deviation. The correction direction is determined by the sign of the chromaticity difference value, and the deviation interval is determined by the absolute value of the chromaticity difference value. Different intervals correspond to different compensation values. The direction and intensity dimensions are determined independently yet work synergistically, ensuring the accuracy and stability of the correction. Hysteresis intervals eliminate boundary abrupt changes. A hysteresis amount is configured for each deviation threshold, separating the entry and exit boundaries and forming a hysteresis band near the threshold. When the chromaticity difference value fluctuates near the threshold due to noise, the interval determination will not frequently change abruptly, ensuring the stability and continuity of the corrected image and avoiding flickering in the AR display. Continuous monitoring and a deviation interval enhancement mechanism achieve a progressive response to persistent deviations. When multiple consecutive frames are in the same deviation interval, the system actively increases the compensation deviation interval, so that the correction intensity gradually increases with the duration of the deviation, solving the problem of delayed response to persistent deviations in single-frame independent correction. The compensation value is determined by multiplying the median of the interval by a preset benchmark value, ensuring a natural increasing relationship between each compensation level. This calculation method guarantees a positive correlation between the compensation value and the typical deviation level of each interval, and only one benchmark parameter needs to be preset to automatically adapt the compensation value to each interval, eliminating the need for manual setting. Two levels of safety protection—single-frame limiting and cumulative compensation value upper limit—prevent overshoot. Single-frame limiting restricts the maximum amplitude of a single correction, while the cumulative compensation value upper limit restricts the maximum absolute value of the total correction. These two layers of protection jointly ensure the safety of the correction process, avoiding correction loss of control due to single-frame anomalies or long-term accumulation. The cross-frame transmission of the cumulative compensation value achieves a smooth continuation of the correction effect. The cumulative compensation value is continuously transmitted between frames; when the deviation persists, the correction intensity accumulates frame by frame. After the deviation is eliminated, the cumulative compensation value is reset to zero after several consecutive safe frames, giving the correction process inertia while avoiding interference from single-frame noise.

[0133] In practical implementation, the compensation direction can be determined by combining the sign of the chromaticity difference value with its absolute value. For example, additional bias correction can be applied when the absolute value exceeds a certain threshold, or no correction can be applied when the absolute value is close to zero to avoid oversensitivity. The number of deviation thresholds and interval divisions can also be achieved by using one threshold to divide into two intervals, or by using three or more thresholds to divide into four or more intervals, or by using dynamic thresholds that adaptively adjust according to the progress of the surgery or changes in tissue condition. The compensation value can be determined by directly presetting fixed compensation values ​​for each interval, multiplying the interval boundary value by the reference value, using the weighted average of the intervals instead of the median, or replacing the reference value with a dynamic coefficient related to the rate of impedance change. The implementation of the hysteresis interval can also be replaced by a hysteresis method based on historical states, requiring that the switching conditions be met for multiple consecutive frames before switching can occur. The implementation of continuous monitoring and deviation range improvement can be replaced by triggering deviation range improvement when a certain proportion of consecutive frames belong to the same range, or by triggering based on the cumulative frame count rather than the consecutive frame count, or by introducing a hysteresis mechanism, requiring more frames after deviation range improvement before downgrading, to avoid frequent switching near the boundary after deviation range improvement. The limiting method can also restrict the inter-frame change rate of the compensation value, that is, limit the change amplitude of the compensation value between adjacent frames, making the correction process smoother, or allow a larger limiting value when the deviation is large, and a smaller limiting value when the deviation is small. The cumulative compensation method can also be replaced by an exponential decay cumulative method, that is, the cumulative compensation value of each frame is multiplied by a decay factor based on the cumulative compensation value of the previous frame and then added to the compensation value of the current frame, so that the influence of historical corrections gradually weakens, or a window moving average method, only accumulating the average of the compensation values ​​of the most recent M frames. The protection method for the cumulative compensation value can be replaced by a fixed absolute upper limit, or by using a percentage of the maximum range of the chroma difference value as the upper limit, or by using a cumulative compensation value freezing method, freezing the accumulation when the cumulative compensation value exceeds a certain multiple of the current chroma difference value, until the deviation shrinks and the relationship between the two is restored. The interaction method between the cumulative compensation value and single-frame correction can also be replaced by a weighted combination of single-frame compensation value and cumulative compensation value, so that single-frame response and cumulative effect participate in the correction with different proportions, and the contribution of the two can be adjusted according to actual needs. The implementation method of the reset condition can also be replaced by the cumulative compensation value gradually decaying to zero at a fixed rate within the safe range, or by using a dynamic reset threshold related to the size of the cumulative compensation value. The larger the cumulative compensation value, the fewer consecutive frames are required, thus accelerating the speed at which the cumulative compensation value reaches zero.

[0134] Based on the numerical range of the impedance change rate, a first weight corresponding to the corrected chromaticity and a second weight corresponding to the impedance change rate are determined. After normalizing the corrected chromaticity and the impedance change rate, a weighted calculation is performed according to the first weight and the second weight to generate a comprehensive thermal damage index.

[0135] This step is the data fusion stage of the entire navigation method, integrating data from two different sources into a unified index. The corrected chromaticity, obtained after visual-impedance cross-calibration, represents the current color state of the wound tissue. Simultaneously, the impedance change rate represents the change in the tissue's electrophysical properties. This step weights and sums these two signals according to a certain proportion to generate a comprehensive thermal damage index. This index will serve as the direct driving data for augmented reality navigation information; its value directly determines the color of the color coating and the flicker frequency in the AR image.

[0136] Before weighted calculation, both the corrected chromaticity and the rate of change of impedance must be normalized. The normalization of the corrected chromaticity is based on its maximum possible range of variation, obtained through in vitro tissue experiments, with a value range of 0 to 100 chromaticity units. The corrected chromaticity is divided by this maximum range value and multiplied by 100 to convert it into a dimensionless value in the 0-100 range. The normalization of the rate of change of impedance is based on its maximum possible range of variation, obtained through in vitro tissue experiments, with a value range of -20 to 200 ohms / second. The current rate of change of impedance is subtracted from the minimum value, divided by the range, and multiplied by 100 to convert it into a dimensionless value in the 0-100 range. After normalization, both fall within the same numerical range, have consistent dimensions, and the weighted sum is dimensionally valid.

[0137] Both the corrected chromaticity and impedance change rate have their limitations. Although the corrected chromaticity has been calibrated, it is still essentially a visual signal and may be distorted when the wound is extensively covered by blood or the lens is heavily contaminated. While the impedance change rate is not affected by visual interference, its value may fluctuate due to changes in the contact state between the electrosurgical unit and the tissue. Through weighted fusion, the system can dynamically adjust the dependence on the two signals under different conditions, ensuring that the final output thermal damage index is always dominated by the more reliable signal at the moment.

[0138] The rate of change of impedance is a numerical value, but its magnitude reflects the stage of thermal damage in the tissue. Specifically, a low rate of change of impedance typically corresponds to a safe period where the tissue is well-hydrated and has not yet undergone significant thermal denaturation; a medium rate of change of impedance corresponds to the stage of thermal damage accumulation where the tissue begins to dehydrate and denature; and a high rate of change of impedance corresponds to a high-risk stage where the tissue is deeply carbonized and about to perforate. The reliability of the corrected chromaticity and the rate of change of impedance varies at different stages. This step utilizes the tissue stage information reflected by the rate of change of impedance to determine the weighting of the two signals during fusion.

[0139] Specifically, the range of impedance change rate is divided into multiple numerical intervals. Based on the interval into which the impedance change rate of the current frame falls, a first weight (weight of the corrected chroma) and a second weight (weight of the impedance change rate) are determined. Both weights are values ​​between 0 and 1, and their sum is 1. The corrected chroma and impedance change rate of the current frame are obtained, multiplied by the corresponding first and second weights respectively, and then added together to obtain the comprehensive thermal damage index.

[0140] The method for determining the first weight and the second weight is as follows:

[0141] Multiple sets of calibration data were obtained from the ex vivo tissue sample during the electrosurgical cutting process. Each set of calibration data included the synchronously acquired impedance change rate, corrected chromaticity, and thermal damage depth.

[0142] The calibration data is divided into a first numerical range, a second numerical range, and a third numerical range according to the rate of change of impedance.

[0143] For each numerical interval, the comprehensive thermal damage index is calculated with different weight combinations, and the correlation coefficient between the comprehensive thermal damage index and the thermal damage depth under each weight combination is calculated. The weight combination with the highest correlation coefficient is selected as the weight value for that interval.

[0144] Specifically, when the impedance change rate is in the first numerical range, the first weight in the weight combination with the highest correlation coefficient is greater than the second weight; when the impedance change rate is in the second numerical range, the difference between the first weight and the second weight in the weight combination with the highest correlation coefficient decreases; when the impedance change rate is in the third numerical range, the second weight in the weight combination with the highest correlation coefficient is greater than the first weight.

[0145] This step defines the specific method for determining the weights, clarifying the weight relationships among the three impedance intervals. The design logic is to correspond the weight allocation to the physical meaning of the impedance change rate: when the impedance change rate is low, the visual signal is reliable, allowing the corrected chroma to dominate; when the impedance change rate is moderate, both have reference value and are treated in a balanced manner; when the impedance change rate is high, the visual signal may be distorted, allowing the impedance change rate to dominate. The weight allocation across the three intervals transitions sequentially from chroma dominance to balance and then to impedance dominance, forming a gradual fusion strategy.

[0146] The range of impedance change rate was divided into three numerical intervals: the first interval was the low impedance change rate interval, corresponding to the tissue safety period; the second interval was the medium impedance change rate interval, corresponding to the thermal damage accumulation period; and the third interval was the high impedance change rate interval, corresponding to the tissue high-risk period. The boundaries of the three intervals were calibrated through in vitro tissue experiments.

[0147] The specific calibration method is as follows: In the in vitro experiment, isolated porcine gastric tissue samples were cut using high-frequency electrosurgical units with different powers ranging from 20 watts to 50 watts and a step size of 10 watts. Impedance change rate data were collected simultaneously during the cutting process at a sampling rate of 30 times / second. After cutting, the tissue samples were fixed, embedded, and sectioned. The thermal damage depth at each cutting point was measured under a microscope, characterized as the percentage of the residual submucosal layer thickness to the original thickness, and divided into three groups according to the damage depth. The impedance change rate data corresponding to each group were statistically analyzed, and the average and standard deviation of the impedance change rate within each group were calculated. The boundary between the first and second numerical intervals was taken as the maximum value of all measured values ​​in the mild damage group; the boundary between the second and third numerical intervals was taken as the maximum value of all measured values ​​in the moderate damage group. The reason for using the maximum value instead of the average value is to ensure that the boundary covers all cases in the group, avoiding the situation where samples that should belong to the first group are classified into adjacent groups due to a low boundary.

[0148] Typical calibration values ​​are as follows: the first range is less than 20 ohms / second, the second range is 20 to 80 ohms / second, and the third range is greater than 80 ohms / second. Specific values ​​vary depending on the tissue type and electrosurgical parameters, and are set during system initialization based on the preoperative calibration results.

[0149] When the impedance change rate is within the first range, the tissue is still in a safe period, and the visual signal is not severely interfered with by smoke, bleeding, etc., so the corrected chromaticity has high reliability. However, the impedance change rate itself has a low value, the signal-to-noise ratio is relatively poor, and it is easily interfered with by equipment noise. Therefore, the first weight is determined to be greater than the second weight, and the comprehensive thermal damage index is mainly determined by the corrected chromaticity.

[0150] When the rate of change of impedance is in the second range, the tissue has entered the stage of thermal damage accumulation, and the wound begins to show slight charring and bleeding. The reliability of the corrected color begins to decrease. At the same time, the rate of change of impedance has risen to a moderate level, and the signal intensity is sufficient to be comparable to the visual signal. Therefore, the difference between the first and second weights is determined to be smaller, and their contributions to the comprehensive thermal damage index tend to be balanced.

[0151] When the rate of change of impedance is in the third range, the tissue has entered a high-risk stage, and deep carbonization may have occurred. However, the wound surface is often accompanied by bleeding and fumes, and the corrected color may be severely distorted. Conversely, when the rate of change of impedance is high, its value directly reflects the degree of tissue moisture vaporization and structural damage, and is not affected by visual interference. Therefore, the second weight is determined to be greater than the first weight, and the comprehensive thermal damage index is mainly determined by the rate of change of impedance.

[0152] The specific values ​​of the first and second weights were obtained through closed-loop calibration in in vitro tissue experiments. The calibration process is as follows: In the in vitro experiments, tissue samples were cut using electrosurgical units of different powers, and three sets of data were collected simultaneously: impedance change rate, corrected chromaticity, and thermal damage depth. All data points were divided into three intervals according to the impedance change rate. For each data point, a comprehensive thermal damage index was calculated using different weight combinations. The first weight ranged from 0.1 to 0.9 with a step size of 0.05, and the second weight was equal to 1 minus the first weight. Then, the correlation coefficient between the comprehensive thermal damage index and the actual thermal damage depth under each weight combination was calculated. The weight combination with the highest correlation coefficient was selected as the optimal weight value for that interval.

[0153] Based on the above calibration method, the range and preferred values ​​of the weights for each interval are as follows:

[0154] The first numerical range is as follows: the first weight ranges from 0.6 to 0.8, preferably 0.7; the second weight ranges from 0.2 to 0.4, preferably 0.3. In this range, the first weight is greater than the second weight, and the difference between the two is relatively large, reflecting the absolute dominance of the visual signal during the safe period.

[0155] The second numerical range is as follows: the first weight ranges from 0.4 to 0.6, preferably 0.5; the second weight ranges from 0.4 to 0.6, preferably 0.5. At this point, the first and second weights are equal or nearly equal, reflecting the balanced reference value of the two signals during the thermal damage accumulation period.

[0156] The third numerical range: the first weight ranges from 0.2 to 0.4, preferably 0.3; the second weight ranges from 0.6 to 0.8, preferably 0.7. In this range, the second weight is greater than the first weight, and the difference between the two is about 0.4, reflecting the dominance of the impedance signal during the high-risk period.

[0157] The selection of optimal weight values ​​for each interval is related to the calibration results of the mapping relationship. Regions with steep slopes in the mapping relationship curve correspond to the impedance change rate sensitive area, and the weight allocation in this region tends to favor impedance signals; regions with gentle slopes tend to favor visual signals. Both are based on the same set of in vitro experimental data and together constitute a complete calibration system.

[0158] This step divides the impedance change rate into three intervals and dynamically allocates weights, automatically adjusting the composition of the thermal damage index under different tissue states: visual signals dominate at low impedance, both signals are balanced at medium impedance, and physical signals dominate at high impedance. The final output thermal damage index is always dominated by the most reliable data source, improving the accuracy and robustness of the index across all scenarios. The weight allocation strategy directly corresponds to the physical meaning of the impedance change rate, giving the fusion logic a clear physical basis. The three intervals correspond to three weight allocation modes. This correspondence is based on an objective analysis of the credibility of each signal at different thermal damage stages, rather than arbitrary settings, enhancing the interpretability and persuasiveness of the solution. The gradual allocation of weights avoids abrupt changes in the thermal damage index when the impedance change rate crosses intervals, making the index change smoother and more continuous, which is beneficial for the stable presentation of subsequent AR navigation information. The calibration of weight values ​​and mapping relationships uses the same set of in vitro experimental data, forming a complete calibration system. The mapping relationship focuses on the corresponding curve between the rate of change of impedance and the chromaticity shift, while the weight calibration focuses on the reliability differences of various signals under different degrees of thermal damage. The two complement and corroborate each other, jointly ensuring that the parameter calibration of the entire technical solution has a unified experimental basis. By fusing the corrected chromaticity and the rate of change of impedance, the fused comprehensive thermal damage index has higher anti-interference ability and is less susceptible to the influence of a single signal source anomaly. As the direct data driver for the AR navigation information in step five, the accuracy and stability of the comprehensive thermal damage index directly affect the reliability of the AR display content.

[0159] In practical implementation, the weight determination method can also be replaced by a continuous weight function, where the weights change continuously with the impedance change rate; fuzzy logic weight allocation, which fuzzifies the impedance change rate into multiple membership functions, with each weight weighted according to its membership degree; or dynamic weights based on Bayesian estimation, which automatically adjust the weights based on the real-time reliability estimate of the two signals. The calibration of interval boundaries can also be performed in real-time during the operation, collecting impedance change rate data during the initial safe cutting phase of the surgery, and dividing intervals online based on the distribution characteristics of the actual data to achieve individualized adaptive calibration; or directly presetting based on empirical values ​​from literature, without relying on experimental calibration, suitable for scenarios where in vitro experiments cannot be performed, but with lower accuracy. The number of intervals can also be replaced with two intervals, distinguishing only between the safe and high-risk periods, suitable for scenarios where fusion accuracy requirements are not high, or four or more intervals, providing a more granular weight adjustment gradient, suitable for scenarios requiring more precise weight control. The fusion method can also be replaced by weighted product fusion, suitable for scenarios where the relationship between the two signals is nonlinear; or Kalman filter fusion, which treats the two signals as independent observations for optimal estimation, suitable for scenarios where noise characteristics are known and optimal estimation is required; or confidence-based selection fusion, where the confidence of one signal is below a threshold and the other signal is directly used as the output, suitable for extreme scenarios where one signal is completely ineffective. The determination of weight values ​​can also be replaced by minimizing the mean square error between the comprehensive thermal damage index and the actual thermal damage depth, minimizing the deviation between the index and the damage depth; or by setting weights directly based on clinical expert experience, suitable for scenarios where it is difficult to obtain a large amount of experimental data; or by adjusting weights online through intraoperative feedback, fine-tuning the weights in real time according to the surgical outcome to achieve continuous optimization.

[0160] Based on the range of the thermal damage comprehensive index, the corresponding color and flashing frequency are determined, a dynamic color coating is generated, and the result is output to the nurse's augmented reality display device.

[0161] This step is the output presentation stage of the entire navigation method, and it is responsible for converting the generated thermal injury comprehensive index into visual information that nurses can directly perceive.

[0162] After processing, a quantified thermal damage index has been obtained, which integrates visual signals (corrected chromaticity) and physical signals (impedance change rate) to represent the current risk level of thermal damage to the wound tissue. However, a single numerical value cannot be directly used by nurses for surgical coordination decisions. This step maps the index to augmented reality navigation information, presenting it in a visual form on the augmented reality display device worn by the nurse, enabling the nurse to intuitively identify the current risk status of the wound and areas requiring attention.

[0163] This step involves two sequential operations: generating augmented reality navigation information and outputting the generated navigation information to the nurse's augmented reality display device.

[0164] Specifically, the system generates a corresponding dynamic color coating based on the comprehensive thermal injury index generated in step four. The dynamic color coating is a semi-transparent, colored graphic layer whose shape matches the outline of the wound area segmented in step one, generated with the wound area mask as its boundary. The coating is superimposed on the corresponding position of the wound area in the endoscopic image, allowing nurses to visually see the current thermal injury risk distribution of the wound.

[0165] The display properties of the coating are determined by the value of the thermal damage index, which includes both color and flicker frequency.

[0166] First, the comprehensive thermal damage index is divided into four consecutive risk ranges. The comprehensive thermal damage index ranges from 0 to 100, and the ranges are divided into low-risk, medium-risk, relatively high-risk, and high-risk ranges in ascending order of value.

[0167] The low-risk zone corresponds to a value range of 0 to 30, indicated by a green coating, signifying a safe tissue condition requiring no special intervention. The medium-risk zone corresponds to a value range of 30 to 60, indicated by a yellow coating, signifying the need for monitoring. The higher-risk zone corresponds to a value range of 60 to 80, indicated by an orange coating, signifying the need to prepare for intervention. The high-risk zone corresponds to a value range of 80 to 100, indicated by a red coating, signifying the need for immediate action.

[0168] To avoid abrupt changes in coating color at the boundaries of adjacent intervals, the system employs a continuous gradient approach during color mapping: when the thermal damage index changes from low to high and crosses the interval boundary, the color does not switch instantaneously, but rather transitions smoothly from the current color to the next color in a gradual manner. The transition bandwidth is 2 units on each side of the interval boundary, making the color change more natural and smooth, and avoiding unnecessary visual impact on nurses.

[0169] While determining the coating color, the flicker frequency of the coating is determined based on the risk range of the comprehensive thermal damage index.

[0170] When the overall risk index is in the low-risk range, the coating remains constantly lit without flickering to avoid unnecessary visual interference. When the overall risk index is in the medium-risk range, the coating flickers at a low frequency of 0.5 to 1 Hz, completing a bright-dark cycle every 1 to 2 seconds, alerting nurses to the current situation without requiring immediate action. When the overall risk index is in the high-risk range, the coating flickers at a medium frequency of 1 to 2 Hz. When the overall risk index is in the high-risk range, the coating flickers at a high frequency of 3 to 5 Hz, flashing 3 to 5 times per second, creating a sense of visual urgency and prompting nurses to prepare for immediate procedures. The coating uses an RGBA color model with a fixed transparency of 40% to 60%, preferably 50%, allowing nurses to see the true texture and details of the wound while observing the coating color, without affecting observation of the surgical procedure.

[0171] In addition to the dynamic color coating, auxiliary visual prompts are generated based on the risk range of the comprehensive thermal damage index: when the comprehensive index is in the medium-risk or higher range, the system generates a dynamic directional arrow at the position with the highest comprehensive thermal damage index in the wound area; when the comprehensive index is in the high-risk range, the system generates a forced operation instruction box in the central area of ​​the AR display device's field of view, displaying text prompts such as "prepare to inject water immediately", and at the same time triggers the AR device's border to flash red.

[0172] Finally, the generated augmented reality navigation information is combined with the endoscopic image to form a complete augmented reality frame. The compositing method is as follows: a dynamic color coating and its auxiliary elements are superimposed on the endoscopic image according to the pixel coordinates of the wound area. The coating and the endoscopic image share the same coordinate system to ensure that the virtual information and the real wound are accurately corresponded in space.

[0173] The synthesized augmented reality (AR) image is transmitted via wired or wireless means to the nurse's AR display device for rendering and presentation. The AR display device is a head-mounted AR glasses device with optical see-through capabilities. The nurse sees a coated endoscope image through this device, and the coating position corresponds to the actual position of the wound in the nurse's field of vision.

[0174] The transmission and display of the entire screen are synchronized with the frame rate of the endoscopic image, continuously updating at a frequency of 30 frames per second to ensure that the guidance information seen by the nurses is kept in real time synchronized with the surgical screen.

[0175] This step maps the thermal injury composite index to a dynamic color coating, transforming abstract numerical values ​​into visual information directly perceptible to nurses. This reduces cognitive burden; nurses do not need to understand the calculation principles and numerical meanings of the thermal injury composite index, but can judge the current thermal injury risk status simply by observing the overlaid color on the wound surface, significantly shortening the reaction time from information acquisition to operational decision-making. Through dual encoding of color and flashing frequency, risk information can be effectively perceived under different attentional states. Color encoding provides clear risk classification information when the nurse is focused on the wound area, while flashing encoding still attracts attention through dynamic changes when the nurse's attention is diverted or in the peripheral vision. The two complement each other, ensuring the reliability of information transmission. By overlaying the dynamic color coating on the wound area rather than displaying it at the screen edge, spatial correspondence between navigation information and the actual wound surface is achieved. Nurses do not need to switch their gaze between the surgical screen and the numerical display area; they can directly determine the specific wound area requiring attention based on the position of the color coating, reducing gaze shift time and lowering the risk of operational interruption due to gaze switching. With a real-time update frequency of 30 frames per second, the navigation information is synchronized with the surgical progress. The coating color and flashing frequency update synchronously with the change of the thermal damage index in each frame, with a lag time of less than one frame interval. Nurses always see the current risk status, eliminating judgment errors caused by information delays. Multi-level color differentiation and flashing frequency grading clearly categorize the risk levels: a constant green indicates a safe period, a low-frequency yellow flash indicates attention but no immediate action is needed, a medium-frequency orange flash indicates preparation for intervention, and a high-frequency red flash indicates immediate action. This four-level gradient design allows nurses to quickly determine the priority of operations based on visual information. Dynamic directional arrows and mandatory operation instruction boxes proactively guide nurses' attention and actions at critical moments. Arrows indicate specific locations of concentrated thermal damage, preventing nurses from searching the wound area; mandatory instruction boxes are presented in an unobtrusive manner at the highest risk, preventing nurses from missing crucial warnings due to visual fatigue or distraction. This combination of proactive guidance and passive warnings ensures the effectiveness of information transmission in high-risk situations. By overlaying the dynamic color coating in a semi-transparent manner rather than completely covering it, nurses can still see the real texture and details of the wound through the coating while observing its color. When the transparency is 50%, the coating color and the real color of the wound each account for half of the visual weight. Nurses can obtain risk information without affecting the observation of key surgical features such as wound shape, bleeding points, and blood vessels, ensuring that navigation information does not interfere with normal surgical operation judgment.

[0176] In practical implementation, the form of navigation information can be replaced with wound boundary outlines, heat maps, or numerical labels. Color mapping schemes can be replaced with continuous gradient color bands, two-color gradients, or grayscale mapping. Flashing frequency schemes can be replaced with continuous frequency modulation, constant frequency with varying duty cycles, or pulsed flashing. Warning prompts can be replaced with voice prompts, vibration prompts, or line-of-sight guidance prompts. Augmented reality display device types can be replaced with medical head-mounted displays, medical augmented reality monitors, or tablet / mobile devices. The method of combining virtual images with real-world images can also transmit endoscopic images and color coatings as independent data streams to the AR device, which then performs the synthesis and overlay itself. This method reduces the burden on the system processing unit but requires the AR device to have image synthesis capabilities.

[0177] The present invention also includes an augmented reality-based endoscopic submucosal dissection nursing care navigation system, comprising:

[0178] The image acquisition and preprocessing module is used to acquire endoscopic image data and extract the observed chromaticity shift of the wound area;

[0179] The mapping and difference calculation module is used to acquire electrosurgical impedance data and calculate the impedance change rate per unit time. Based on a preset mapping relationship, it determines the theoretical chromaticity offset corresponding to the impedance change rate and calculates the chromaticity difference value between the observed chromaticity offset and the theoretical chromaticity offset.

[0180] The correction module is used to determine the compensation direction based on the sign of the chromaticity difference value, determine the deviation range into which the chromaticity difference value falls based on the comparison result between the absolute value of the chromaticity difference value and the preset deviation threshold, different deviation ranges correspond to different compensation values, and the compensation value increases as the deviation range increases, and after limiting the compensation value with a preset maximum step size, the observed chromaticity offset is corrected to obtain the corrected chromaticity.

[0181] The weighted fusion module is used to determine the first weight corresponding to the corrected chromaticity and the second weight corresponding to the impedance change rate based on the numerical range of the impedance change rate, and to perform weighted calculation on the corrected chromaticity and the impedance change rate after normalization according to the first weight and the second weight to generate a comprehensive thermal damage index.

[0182] The augmented reality output module is used to determine the corresponding color and flashing frequency according to the range of the thermal damage comprehensive index, generate a dynamic color coating, and output it to the nurse's augmented reality display device.

[0183] The technical scope of this invention is not limited to the content described above. Those skilled in the art can make various modifications and variations to the above embodiments without departing from the technical concept of this invention, and all such modifications and variations should fall within the protection scope of this invention.

Claims

1. A navigation-based nursing care method for endoscopic submucosal dissection based on augmented reality, characterized in that: Specifically, it includes: Acquire endoscopic image data and extract the observed chromaticity shift of the wound area; acquire electrosurgical impedance data and calculate the rate of impedance change per unit time. Based on the preset mapping relationship, the theoretical chromaticity offset corresponding to the impedance change rate is determined, and the chromaticity difference between the observed chromaticity offset and the theoretical chromaticity offset is calculated. The compensation direction is determined based on the sign of the chromaticity difference value. The deviation range into which the chromaticity difference value falls is determined based on the comparison result between the absolute value of the chromaticity difference value and the preset deviation threshold. Different deviation ranges correspond to different compensation values, and the compensation value increases as the deviation range increases. After limiting the compensation value with a preset maximum step size, the observed chromaticity offset is corrected to obtain the corrected chromaticity. Based on the numerical range of the impedance change rate, determine the first weight corresponding to the corrected chromaticity and the second weight corresponding to the impedance change rate. After normalizing the corrected chromaticity and the impedance change rate, perform a weighted calculation according to the first weight and the second weight to generate a comprehensive thermal damage index. Based on the range of the thermal damage comprehensive index, the corresponding color and flashing frequency are determined, a dynamic color coating is generated, and the result is output to the nurse's augmented reality display device.

2. The augmented reality-based endoscopic submucosal dissection nursing care and navigation method according to claim 1, characterized in that: The specific method for obtaining the observed chromaticity shift is as follows: Obtain the colorimetric baseline values ​​of the patient's normal mucosal tissue before surgery; The endoscopic image data is segmented to extract the area of ​​the dissected wound; Obtain the chromaticity value of the pixels within the stripped wound area, and calculate the difference between the chromaticity value and the chromaticity reference value as the observed chromaticity offset.

3. The augmented reality-based endoscopic submucosal dissection nursing care and navigation method according to claim 1, characterized in that: The specific method for establishing the preset mapping relationship is as follows: Multiple sets of sample data were acquired during the electrosurgical cutting process of ex vivo tissue samples. Each set of sample data included the synchronously acquired impedance change rate and sample chromaticity offset. Using the impedance change rate as the independent variable and the sample chromaticity shift as the dependent variable, data fitting is performed to obtain the correspondence between the impedance change rate and the theoretical chromaticity shift. Store the correspondence as the preset mapping relationship; The theoretical chromaticity offset value is the offset value obtained during the intraoperative application stage based on the current rate of change of impedance through the mapping relationship. The sample chromaticity offset value is the offset value obtained during the calibration stage through actual measurement of ex vivo tissue samples. Both are manifestations of the same physical quantity at different stages.

4. The augmented reality-based endoscopic submucosal dissection nursing care and navigation method according to claim 1, characterized in that: The specific steps for determining the compensation direction based on the sign of the chromaticity difference value are as follows: When the chromaticity difference value is positive, the compensation direction is determined to be negative, and the negative direction is used to indicate that the observed chromaticity offset is reduced and corrected. When the chromaticity difference value is negative, the compensation direction is determined to be positive, and the positive direction is used to indicate that the observed chromaticity offset is increased and corrected.

5. The augmented reality-based endoscopic submucosal dissection nursing care and navigation method according to claim 1, characterized in that: Specifically, the deviation range into which the chromaticity difference value falls is determined as follows: The preset deviation threshold includes a first deviation threshold and a second deviation threshold, and the first deviation threshold is less than the second deviation threshold; When the absolute value of the chromaticity difference value is less than or equal to the first deviation threshold, it is determined that the chromaticity difference value falls into the first deviation range; When the absolute value of the chromaticity difference value is greater than the first deviation threshold and less than or equal to the second deviation threshold, it is determined that the chromaticity difference value falls into the second deviation range. When the absolute value of the chromaticity difference is greater than the second deviation threshold, the chromaticity difference is determined to fall into the third deviation range.

6. The augmented reality-based endoscopic submucosal dissection nursing care and navigation method according to claim 5, characterized in that: Determining the deviation range into which the chromaticity difference value falls also includes: The first deviation threshold corresponds to the first lag amount, and the second deviation threshold corresponds to the second lag amount; When the absolute value of the chromaticity difference increases from less than or equal to the first deviation threshold to greater than the sum of the first deviation threshold and the first hysteresis, it is determined that the chromaticity difference value has entered the second deviation interval from the first deviation interval. When the absolute value of the chromaticity difference value decreases from greater than the first deviation threshold to less than or equal to the difference between the first deviation threshold and the first hysteresis, it is determined that the chromaticity difference value returns from the second deviation interval to the first deviation interval; When the absolute value of the chromaticity difference increases from less than or equal to the second deviation threshold to greater than the sum of the second deviation threshold and the second hysteresis, it is determined that the chromaticity difference value has entered the third deviation interval from the second deviation interval. When the absolute value of the chromaticity difference value decreases from greater than the second deviation threshold to less than or equal to the difference between the second deviation threshold and the second hysteresis, it is determined that the chromaticity difference value returns from the third deviation interval to the second deviation interval.

7. The augmented reality-based endoscopic submucosal dissection nursing care and navigation method according to claim 5, characterized in that: Before determining the compensation value, the deviation range is also increased, specifically: Obtain the deviation interval determination results of multiple consecutive frames. When the deviation interval determination results of a preset number of consecutive frames all belong to the first deviation interval, upgrade the current deviation interval to the second deviation interval. When the deviation interval determination results of a consecutive preset number of frames all belong to the second deviation interval, the current deviation interval is upgraded to the third deviation interval; If the deviation interval determination results of a consecutive preset number of frames all belong to the third deviation interval, no boosting will be performed.

8. The augmented reality-based endoscopic submucosal dissection nursing care and navigation method according to claim 5, characterized in that: The determination of the compensation value is specifically as follows: The first deviation interval corresponds to the first compensation value, the second deviation interval corresponds to the second compensation value, and the third deviation interval corresponds to the third compensation value; Obtain a preset benchmark value, wherein the preset benchmark value is a dimensionless proportionality coefficient; Determine the first median value of the first deviation interval, the second median value of the second deviation interval, and the third median value of the third deviation interval. The first median value is half of the first deviation threshold, the second median value is half of the sum of the first deviation threshold and the second deviation threshold, and the third median value is half of the sum of the second deviation threshold and the preset maximum range of chromaticity difference value. The first compensation value is the product of the first median and the preset benchmark value; the second compensation value is the product of the second median and the preset benchmark value; and the third compensation value is the product of the third median and the preset benchmark value. The second compensation value is greater than the first compensation value, and the third compensation value is greater than the second compensation value.

9. The augmented reality-based endoscopic submucosal dissection nursing care and navigation method according to claim 8, characterized in that: After determining the compensation value, the process also includes determining the correction amount through the cumulative compensation value, specifically: Get the cumulative compensation value of the previous frame; The compensation value generated in the current frame is added to the cumulative compensation value of the previous frame to obtain the correction amount for the current frame; The observed chromaticity offset is corrected using the correction amount of the current frame to obtain the corrected chromaticity; The correction amount of the current frame is stored as the cumulative compensation value of the current frame for use in the next frame; When the absolute value of the chromaticity difference is less than or equal to the first deviation threshold for a consecutive preset number of frames, the cumulative compensation value is reset to the initial value.

10. The augmented reality-based endoscopic submucosal dissection nursing care and navigation method according to claim 1, characterized in that: The method for determining the first weight and the second weight is as follows: Multiple sets of calibration data were obtained from the ex vivo tissue sample during the electrosurgical cutting process. Each set of calibration data included the synchronously acquired impedance change rate, corrected chromaticity, and thermal damage depth. The calibration data is divided into a first numerical range, a second numerical range, and a third numerical range according to the rate of change of impedance. For each numerical interval, the comprehensive thermal damage index is calculated with different weight combinations, and the correlation coefficient between the comprehensive thermal damage index and the thermal damage depth under each weight combination is calculated. The weight combination with the highest correlation coefficient is selected as the weight value for that interval. Specifically, when the impedance change rate is in the first numerical range, the first weight in the weight combination with the highest correlation coefficient is greater than the second weight; when the impedance change rate is in the second numerical range, the difference between the first weight and the second weight in the weight combination with the highest correlation coefficient decreases; when the impedance change rate is in the third numerical range, the second weight in the weight combination with the highest correlation coefficient is greater than the first weight.

Images