A method and system for evaluating microscopic endoscopic procedures

Abstract

This invention provides a method and system for evaluating microendoscopic operations, comprising: preprocessing and extracting features from a microendoscopic operation data stream to generate a raw motion primitive sequence and a device event sequence; obtaining a comprehensive objective performance score within an operation analysis window based on preset microendoscopic operation meta-indicators; simultaneously generating a conditional Pareto front for each anatomical site by combining a dynamic high-efficiency operation library; acquiring environmental state data; analyzing the operation to be evaluated using a preset strategy performance evaluation mechanism to obtain strategy performance evaluation results; performing causal analysis on the operation to be evaluated based on a Bayesian structural equation model and counterfactual attribution using the dynamic high-efficiency operation library to generate a causal analysis dataset; and performing a comprehensive operation evaluation based on the comprehensive objective performance score, the conditional Pareto front, the strategy performance evaluation results, and the causal analysis dataset to generate a comprehensive operation evaluation result, thereby improving the accuracy of microendoscopic operation evaluation.

Description

Technical Field

[0001] This invention relates to the field of image analysis technology, and more specifically, to a method and system for evaluating microscopic endoscopy operations. Background Technology

[0002] Quantitative assessment and effective training of microendoscopic skills are important components of clinical medical education; however, current traditional assessment methods have certain limitations.

[0003] Specifically, traditional assessment methods mostly remain at the descriptive statistics level, often neglecting to model the strategic logic contained in the operator's behavior; at the same time, they often lack modeling of the interpretable causal chain between raw data and clinical errors, resulting in insufficient accuracy in the assessment of microendoscopic operations. Summary of the Invention

[0004] To address the shortcomings of existing technologies, embodiments of the present invention provide a method and system for evaluating microscopic endoscopy operations, thereby solving at least one of the aforementioned technical problems in the prior art.

[0005] To solve the above-mentioned technical problems, the technical solution adopted by the present invention is as follows:

[0006] In a first aspect, the present invention provides a method for evaluating microendoscopic operation, comprising:

[0007] The acquired microendoscopic operation data stream is preprocessed, and features are extracted from the preprocessed data stream to generate the original motion primitive sequence and the device event sequence.

[0008] Based on preset microendoscopic operation meta-indicators, a comprehensive objective performance score is obtained in the operation analysis window. Simultaneously, combined with a dynamic high-efficiency operation library, a conditional Pareto front for each anatomical site is generated.

[0009] The environmental status data corresponding to the microendoscopic operation data stream is acquired, and the operation to be evaluated is analyzed in combination with the preset strategy effectiveness evaluation mechanism to obtain the strategy effectiveness evaluation results.

[0010] Causal analysis of the operation to be evaluated is performed based on the Bayesian structural equation model, and counterfactual attribution is performed in combination with the dynamic and efficient operation library to generate a causal analysis dataset.

[0011] A comprehensive operational evaluation is performed based on the comprehensive objective performance score, conditional Pareto front, strategy performance evaluation results, and causal analysis dataset to generate a comprehensive operational evaluation result.

[0012] In one possible implementation of the first aspect, the acquired microendoscopic operation data stream is preprocessed, and features are extracted from the preprocessed data stream to generate an original motion primitive sequence and a device event sequence, including:

[0013] The microendoscopic operation data stream includes at least the raw motion signal sequence, the device operation log, and the corresponding timestamp;

[0014] The original motion signal sequence is divided into multiple motion segments of fixed size, and features are extracted from each motion segment to obtain a motion segment feature set;

[0015] A Gaussian mixture model is used to cluster the feature set of the motion segments to generate multiple original motion pattern clusters;

[0016] The original motion signal sequence is processed based on the original motion pattern cluster to generate the original motion primitive sequence;

[0017] Extract device events from the device operation log that are time-aligned with the original motion primitive sequence to generate a device event sequence.

[0018] In one possible implementation of the first aspect, a comprehensive objective performance score within the operation analysis window is obtained based on preset microendoscopic operation meta-indicators. Simultaneously, a dynamic high-efficiency operation library is combined to generate a conditional Pareto front for each anatomical site, including:

[0019] The preset microendoscopic operation meta-indicators include at least the information integrity index, target acquisition efficiency, and physiological disturbance index.

[0020] A combination of visual SLAM and image sharpness analysis was used to analyze the current video stream of microscopic endoscopy operations and obtain the information integrity index.

[0021] By combining a pre-trained lesion detection model, the proportion of lesions effectively captured by the operator during the current microendoscopic operation is obtained, and the target capture efficiency is generated based on the proportion of effectively captured lesions.

[0022] Physiological disturbance index of current microendoscopic operation is obtained by using optical flow and tissue deformation analysis;

[0023] The current microscopic endoscopy operation is divided into multiple operation analysis windows according to the preset operation analysis window capacity;

[0024] Based on the preset microscopic endoscopy operation meta-indicators in the operation analysis window, a comprehensive objective performance score is obtained.

[0025] By combining a dynamic and efficient operation library, a conditional Pareto front is generated for the corresponding anatomical site of the current microendoscopic operation.

[0026] In one possible implementation of the first aspect, a conditional Pareto front is generated for the anatomical site corresponding to the current microendoscopic operation, in conjunction with a dynamic and efficient operation library, including:

[0027] The dynamic high-efficiency operation library is represented as a high-efficiency operation database constructed based on expert demonstration operation data, and the dynamic high-efficiency operation library is updated regularly based on the comprehensive objective performance score.

[0028] The dynamic and efficient operation library is used to store efficient microendoscopic operations, and each efficient microendoscopic operation includes at least the corresponding original operation segment, comprehensive objective performance score and descriptive label.

[0029] Within the dynamic and efficient operation library, the data in the library is grouped based on preset anatomical site labels to generate multiple anatomical site groups;

[0030] For the anatomical site group, the corresponding conditional Pareto front is obtained based on the preset microscopic endoscopy operation meta-indicators in the operation analysis window.

[0031] In one possible implementation of the first aspect, environmental state data corresponding to the microendoscopic operation data stream is acquired, and the operation to be evaluated is analyzed in conjunction with a preset strategy effectiveness evaluation mechanism to obtain the strategy effectiveness evaluation results, including:

[0032] Environmental state features are extracted from the video frames corresponding to the microendoscopic operation data stream to obtain environmental state data;

[0033] The environmental status data includes at least the exploration completeness and the local task complexity;

[0034] The operator's policy network is trained based on the microendoscopic operation data stream, and the operator's policy network is used to simulate the operator's decision-making mechanism.

[0035] Based on the operator policy network, policy effectiveness evaluation and policy environment stability analysis are performed to obtain policy effectiveness evaluation results.

[0036] In one possible implementation of the first aspect, policy effectiveness evaluation and policy environment stability analysis are performed based on the operator policy network to obtain policy effectiveness evaluation results, including:

[0037] The strategy effectiveness is evaluated based on a preset set of strategy effectiveness evaluation indicators to obtain the strategy effectiveness index;

[0038] The preset strategy effectiveness evaluation index set includes at least strategy entropy, environmental action mutual information, and goal orientation index;

[0039] The operator policy network performs simulation prediction based on the environmental state data and generates a simulation action sequence.

[0040] The environmental state data is injected with a disturbance to generate disturbed environmental state data;

[0041] The operator policy network performs simulation prediction based on the disturbed environment state data and generates a sequence of disturbance simulation actions.

[0042] Based on a preset set of strategy environment stability indicators, the simulated action sequence and the disturbance simulated action sequence are analyzed to obtain the strategy stability index.

[0043] Based on the strategy effectiveness index and strategy stability index, a strategy effectiveness evaluation result is generated.

[0044] In one possible implementation of the first aspect, causal analysis is performed on the operation to be evaluated based on a Bayesian structural equation model, and counterfactual attribution is performed in conjunction with the aforementioned dynamic and efficient operation library to generate a causal analysis dataset, including:

[0045] Intent inference is performed on the original motion primitive sequence based on a hidden Markov model to obtain intent variables;

[0046] Extract action and response variables from the microendoscopic video frame sequence corresponding to the operation to be evaluated;

[0047] The action variables are used to characterize the actions performed by the operator;

[0048] The response variable is used to characterize the quality of the microendoscopic video frame;

[0049] Perception variables are obtained according to a preset perception analysis mechanism, and the perception variables are used to characterize the operator's perception state.

[0050] A Bayesian structural equation model is constructed based on the aforementioned intention variables, extracted action variables, response variables, and perception variables.

[0051] The Bayesian structural equation model is estimated using the Markov chain Monte Carlo method to obtain a set of causal estimation parameters.

[0052] In one possible implementation of the first aspect, the method further includes:

[0053] Based on a comprehensive objective performance score, low-efficiency periods are identified in the current microendoscopic operation process, and a set of low-efficiency periods is obtained.

[0054] For each inefficient time period within the inefficient time period set, select an ideal reference segment from the dynamic efficient operation library;

[0055] Based on the ideal reference segment, counterfactual simulations are performed in the corresponding low-efficiency period to obtain counterfactual attribution results;

[0056] The causal estimation parameter set and the counterfactual attribution results are encapsulated into a causal analysis dataset.

[0057] In one possible implementation of the first aspect, a comprehensive operational evaluation is performed based on the comprehensive objective performance score, the conditional Pareto front, the policy performance evaluation results, and the causal analysis dataset to generate a comprehensive operational evaluation result, including:

[0058] Based on the conditional Pareto front, an operational efficiency score is obtained;

[0059] Based on the strategy effectiveness evaluation results, obtain the strategy effectiveness score;

[0060] Based on the aforementioned causal analysis dataset, a causal association score is generated;

[0061] Based on the comprehensive objective performance score, operational efficiency score, strategy performance score, and causal relationship score, a comprehensive operational score and operational improvement suggestions are generated.

[0062] The comprehensive operational score and operational improvement suggestions are packaged into a comprehensive operational evaluation result.

[0063] Secondly, embodiments of the present invention also provide a microendoscopic operation evaluation system, comprising:

[0064] The feature extraction module is used to preprocess and extract features from the microendoscopic operation data stream to obtain the original motion primitive sequence and device event sequence.

[0065] The operation evaluation module is used to obtain the comprehensive objective performance score in the operation analysis window, and generate the conditional Pareto front for each anatomical site by combining the dynamic high-efficiency operation library.

[0066] The strategy evaluation module is used to acquire environmental status data and analyze the operation to be evaluated in conjunction with a preset strategy effectiveness evaluation mechanism to obtain the strategy effectiveness evaluation result.

[0067] The causal analysis module is used to perform causal analysis on the operation to be evaluated, and to perform counterfactual attribution in conjunction with the dynamic and efficient operation library to generate a causal analysis dataset.

[0068] The comprehensive evaluation module is used to perform a comprehensive operational evaluation based on the comprehensive objective performance score, the conditional Pareto front, the strategy performance evaluation results, and the causal analysis dataset, and generate a comprehensive operational evaluation result.

[0069] Compared with the prior art, the present invention has the following beneficial effects:

[0070] The acquired microendoscopic operation data stream is preprocessed, and features are extracted from the preprocessed data stream to generate original motion primitive sequences and device event sequences. This step transforms the original, heterogeneous multimodal data stream into standardized original motion primitive sequences and device event sequences, providing a unified and structured data foundation for subsequent steps.

[0071] Based on the preset microendoscopic operation meta-indicators, the comprehensive objective performance score in the operation analysis window is obtained. Simultaneously, combined with the dynamic high-efficiency operation library, the conditional Pareto front for each anatomical site is generated. This step uses the preset microendoscopic operation meta-indicators, a set of performance indicators independent of subjective evaluation, to ensure the objectivity and fairness of the evaluation.

[0072] The environmental state data corresponding to the microendoscopic operation data stream is acquired, and the operation to be evaluated is analyzed in combination with the preset strategy effectiveness evaluation mechanism to obtain the strategy effectiveness evaluation results. This step achieves a quantitative evaluation of the intrinsic effectiveness and robustness of the strategy adopted by the operator by inversely fitting the operator strategy network used to simulate the operator's decision-making mechanism and conducting simulation perturbation tests on it.

[0073] Causal analysis is performed on the operation to be evaluated based on the Bayesian structural equation model, and counterfactual attribution is performed in combination with the dynamic and efficient operation library to generate a causal analysis dataset. This step improves the accuracy of subsequent operation improvement suggestions by applying the Bayesian structural equation model and counterfactual reasoning and quantifying the contribution of different error sources to specific error events.

[0074] Based on the comprehensive objective performance score, conditional Pareto front, strategy performance evaluation results, and causal analysis dataset, a comprehensive operational evaluation is performed to generate a comprehensive operational evaluation result. This step integrates the quantitative indicators and diagnostic results generated in the previous steps to provide operators with accurate operational evaluation results. Attached Figure Description

[0075] Figure 1 This is a flowchart of the steps of a microendoscopic operation evaluation method according to the present invention;

[0076] Figure 2 This is a flowchart illustrating step S4 in a microendoscopic operation evaluation method of the present invention.

[0077] Figure 3 This is a schematic diagram of a microendoscopic operation evaluation system according to the present invention. Detailed Implementation

[0078] To make the technical solution of the present invention clearer and its technical advantages more apparent, the technical solution of the present invention will be clearly and completely described below in conjunction with specific embodiments. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of the present invention.

[0079] It should be noted that, in this document, the reference to "embodiment" means that a specific feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of the present invention. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. Those skilled in the art will understand, explicitly and implicitly, that the embodiments described herein can be combined with other embodiments.

[0080] The present invention will now be described in detail with reference to the accompanying drawings. Figure 1 This is a flowchart of the steps of a microendoscopic operation evaluation method according to the present invention. Figure 2 This is a flowchart illustrating step S4 in a microendoscopic operation evaluation method of the present invention. The following is a detailed description of this microendoscopic operation evaluation method.

[0081] Step S1: Preprocess the acquired microendoscopic operation data stream and extract features from the preprocessed data stream to generate the original motion primitive sequence and the device event sequence.

[0082] Specifically, the microendoscopic operation data stream includes at least the original motion signal sequence, the device operation log, and the corresponding timestamp.

[0083] The original motion signal sequence is divided into multiple motion segments of fixed size. Features are extracted from each motion segment to obtain a motion segment feature set. The motion segment feature set is clustered using a Gaussian mixture model to generate multiple original motion pattern clusters. The original motion signal sequence is then processed based on the original motion pattern clusters to generate an original motion primitive sequence.

[0084] Extract device events from the device operation log that are time-aligned with the original motion primitive sequence to generate a device event sequence.

[0085] In one possible embodiment, the raw motion signal sequence corresponding to the microendoscopic operation is acquired. This sequence is represented as a multi-degree-of-freedom kinematic time series, which includes at least the following dimension: the instantaneous linear velocity vector of the probe tip in three-dimensional space. For example, a data point in this dimension might be... , representing the velocity components of the probe in the X, Y, and Z directions; the corresponding instantaneous linear acceleration vector, this dimension component can be obtained from the velocity difference or directly from the accelerometer; the instantaneous angular velocity vector of the probe about its axis, used to reflect the rotational motion of the probe.

[0086] Obtain the equipment operation log corresponding to the operation of the microendoscopy. The log is represented as a discrete event sequence actively triggered by the operator and recorded by the endoscope control system. Each log contains the event type and the corresponding timestamp. All the event types constitute a finite set. For example, a partial set of event types is {optical or digital zoom adjustment, focal length adjustment, still image or video clip acquisition, light source brightness adjustment, water and air perfusion control}.

[0087] The original motion signal in the original motion signal sequence is preprocessed to eliminate noise and outliers, generating a continuous motion signal stream. For example, a low-pass filter such as a Butterworth filter with a cutoff frequency of 5Hz is used to smooth the velocity and acceleration signals to retain the intended motion components and filter out high-frequency jitter.

[0088] The preprocessed continuous motion signal stream is cut into a series of fixed-duration, non-overlapping time segments along the time axis. The segment duration can be set according to the corresponding time resolution and feature stability. For example, if the segment duration is set to 300 milliseconds, an operation lasting 5 minutes will generate 600 consecutive segments.

[0089] For each time segment From the smoothed motion signal subsequence it contains, a set of predefined multi-dimensional features are extracted and transformed into a numerical vector representing the motion pattern of the segment. Specifically, the multi-dimensional features include at least three dimensions: time domain, frequency domain, and trajectory.

[0090] For time-domain features, the arithmetic mean of the resultant velocities within the segment is obtained, whereby the resultant velocity is expressed as... The average intensity of motion is reflected by the mean of the resultant velocity; the standard deviation of the resultant velocity reflects the degree of velocity fluctuation, with high values ​​indicating frequent acceleration / deceleration; the mean of the resultant acceleration reflects the urgency of the motion; the standard deviation of the resultant acceleration; and the Shannon entropy of the resultant velocity sequence quantifies the randomness or unpredictability of motion speed.

[0091] For frequency domain features, the proportion of spectral energy in several preset key physiological frequency bands such as 0-0.5 Hz, 0.5-2 Hz, and 2-5 Hz is calculated by performing a Fast Fourier Transform (FFT) on the velocity sequence within the segment. For example, a steady scanning motion may have a high concentration of regular energy in the 0.5-2 Hz frequency band.

[0092] For trajectory features, obtain the total trajectory length of the probe end within the segment in three-dimensional space, and the ratio of the straight-line distance between the start and end points of the segment to the total trajectory length;

[0093] Ultimately, each segment It is transformed into a multidimensional feature vector, and the feature vectors of all segments constitute the feature matrix.

[0094] A Gaussian mixture model is used to cluster the feature matrix. Specifically, the Gaussian mixture model assumes that the data is generated by a mixture of multiple Gaussian distributions, outputs the probability that each data point belongs to each cluster, and automatically fits the clusters using the expectation-maximization algorithm. Then it is automatically determined by minimizing the Bayesian information criterion (BIC).

[0095] Right now ;

[0096] in, Is the model in Likelihood values ​​for each component The number of parameters is used to select the one that minimizes the BIC. As the final cluster number After clustering, each fragment Assigned to a primary cluster label ;

[0097] After clustering is completed, for each cluster The central trend and distribution of the feature vectors of all segments are analyzed and semantically mapped, and semantic labels are defined for each cluster.

[0098] For example, clusters The feature centers show that the mean and standard deviation of the resultant velocity within the segment are extremely high, and the standard deviation of the resultant acceleration is also very high. Furthermore, the frequency domain energy is relatively dispersed. Combined with random sampling, this indicates that... By observing the original video corresponding to the segment, it was found that the operator was rapidly moving the probe over a large area to find the target region. Therefore, the cluster was... The semantic tag is defined as "fast search"; cluster The feature center shows a moderate average resultant velocity, and a very low standard deviation of resultant velocity and resultant acceleration. It also has significant spectral peaks in the low frequency range of 0.5-2Hz, which corresponds to the video showing the probe moving at a constant speed, back and forth or spirally on the tissue surface. Therefore, its semantic label is defined as "smooth scan".

[0099] Through the semantic mapping described above, each time segment in the entire operation process is represented. Original cluster label Replace with its corresponding semantic tag At this point, the original continuous motion signal is transformed into a symbolic and semantic motion state sequence synchronized with the time segment. This sequence is the original motion primitive sequence MS(t). For example, within a certain time window, MS(t) = ["Quick Search", "Quick Search", "Smooth Scan", "Fine Observation", "Judgment Pause", "Smooth Scan", ...]. Each element in this sequence represents a motion behavior pattern with clear clinical meaning led by the operator within the corresponding time window.

[0100] Parse the device operation logs and sort all events by global timestamp to form a device event sequence DE(t). Each event is represented as a tuple, i.e. (time point, event type).

[0101] By using a global clock signal, the original motion primitive sequence MS(t) is timestamped with the device event sequence DE(t), thereby ensuring that for any time t, we can accurately know what motion state we are in and what device operation events have occurred around that time.

[0102] Step S2: Based on the preset microendoscopic operation meta-indicators, obtain the comprehensive objective performance score in the operation analysis window, and simultaneously combine it with the dynamic high-efficiency operation library to generate the conditional Pareto front for each anatomical site.

[0103] Understandably, the dynamic high-efficiency operation library is a high-efficiency operation database constructed based on expert demonstration operation data. This library is periodically updated based on the comprehensive objective performance score. Specifically, it periodically collects operation analysis window data from all anonymous operations within past periods. Assuming this set contains multiple windows, they are sorted in descending order of the comprehensive objective performance score. The windows ranking in the top P percentile are selected as candidate high-efficiency windows. Only window data within the candidate windows that have a physiological disturbance index below a preset threshold or a target capture efficiency exceeding a preset threshold are retained. The retained window data is then updated into the library according to a first-in, first-out principle. The dynamic high-efficiency operation library stores high-efficiency microendoscopic operations, and each high-efficiency microendoscopic operation includes at least the corresponding original operation segment, comprehensive objective performance score, and descriptive label. The descriptive label includes at least the semantic label and device event label of the original motion primitive sequence MS(t) and the device event sequence DE(t) corresponding to the operation sequence.

[0104] In this embodiment, step S2 includes:

[0105] Step S2-1: Obtain the comprehensive objective performance score.

[0106] Specifically, the preset microendoscopic operation meta-indicators include at least the information integrity index, target acquisition efficiency, and physiological disturbance index.

[0107] A combination of visual SLAM and image sharpness analysis was used to analyze the current video stream of microscopic endoscopy operations and obtain the information integrity index.

[0108] By combining a pre-trained lesion detection model, the proportion of lesions effectively captured by the operator during the current microendoscopic operation is obtained, and the target capture efficiency is generated based on the proportion of effectively captured lesions.

[0109] The physiological disturbance index of current microendoscopic procedures was obtained by using optical flow and tissue deformation analysis.

[0110] Furthermore, the current microendoscopic operation is divided into multiple operation analysis windows according to the preset operation analysis window capacity, and a comprehensive objective performance score is obtained based on the preset microendoscopic operation meta-indicators within the operation analysis window.

[0111] In one possible embodiment, by Obtaining information integrity index in this way ,in Deadline Coverage integrity, Specifically, this refers to the average image quality of the covered area:

[0112] For coverage integrity Its physical definition is the effective observed mucosal surface area divided by the estimated total surface area of ​​the target region. Real-time sparse 3D scene reconstruction is performed using feature-point-based visual SLAM such as ORB-SLAM3, meaning that for each frame... ORB feature points are extracted, matched, and tracked, and a 3D map is generated through triangulation. Combined with a global 3D point cloud map Each point Associate its three-dimensional coordinates with the timestamp of its most recent observation;

[0113] Assume a map point In the time window If an area is clearly observed at least once, then its corresponding coverage integrity is:

[0114] Coverage integrity ;

[0115] in, The cardinality of the set is represented by the number of map points that are effectively observed. It is the estimated total surface area of ​​the target anatomical site, which can be estimated from the convex hull area of ​​the point cloud established in the early exploration phase of this operation;

[0116] For example, assuming the 3D point cloud reconstructed by the SLAM system is uniformly distributed on the mucosal surface, meaning the number of points per unit area is roughly the same, then under the assumption of uniform point cloud density, the area ratio is approximately equal to the point-to-number ratio. Let the total surface area... Corresponding total points Let the observed surface area be... Corresponding valid observation points Due to density Uniformity, thus yielding Therefore, coverage integrity .

[0117] For average image quality For each video frame Calculate its overall image quality score And only if If the value exceeds a preset threshold, the current observation is determined to be a valid or clear observation.

[0118] Overall image quality score ;

[0119] in, For sharpness, it is obtained by calculating the variance of the Laplacian operator of the image, i.e. High variance indicates clear edges and a sharp image;

[0120] For illumination uniformity, the entropy of image luminance channels, such as the V channel in HSV, is calculated;

[0121] Motion blur estimation can be obtained by comparing the radial decay rate of high-frequency components in the Fourier spectrum of the image, or more simply, by estimating the correlation between the magnitude of the optical flow vector between two frames and the image gradient direction. Normalize to [0,1], where 1 indicates severe blur;

[0122] , , Preset weights for the corresponding items;

[0123] When a frame Observe a certain map point At that time, if , If the current map point record is of the best quality, then update. , It refers to all map points currently marked as "valid observations". Best quality score The arithmetic mean,

[0124] Right now ;

[0125] in, This represents the number of valid observation points.

[0126] use Methods to obtain target capture performance ,in, It is the number of unique high-confidence suspected targets detected within window T. It is the target number of valid responses from the operated parties, specifically:

[0127] Each frame is processed using a lesion detection model such as YOLOv7, which is pre-trained based on a large dataset of lesions. The model outputs a set of detection results that include at least bounding boxes, confidence scores, and categories. When the confidence score exceeds a preset threshold, the location is... Marked as a candidate target;

[0128] Assign a unique ID to each newly detected candidate target. The lifecycle of a device is tracked using simple online real-time tracking algorithms such as SORT, and all unique devices appearing within window T are tracked. The quantity as ;

[0129] For a goal At its first appearance time After Within the window, check if any of the following valid response conditions are met:

[0130] There is a point in time This causes the operator's gaze to fall on the target. Within the extended area of ​​the bounding box in the current frame, for example, by extending each of the four sides of the bounding box by 20 pixels to cover the surrounding observation, and for a duration exceeding a preset threshold, a focus response is determined to have occurred.

[0131] exist Within the context, there exists a device event DE(t), and when this event occurs, the screen coordinates of the cursor or the logical region where the event is triggered are relative to the target. If the bounding boxes overlap, it is determined that an operation response has occurred.

[0132] If the target If any of the above valid response conditions are met, then Add 1.

[0133] use Methods for obtaining physiological disturbance index ,in, It is the excessive pressure of normalization on scores. It is the normalized potential damage density. and For the preset weighting coefficients, specifically:

[0134] For excessive pressure scores Dense optical flow methods, such as the Farneback method, are used to calculate consecutive frames. arrive pixel-level displacement field By calculating the divergence of the displacement field To estimate the compressive / expansion strain of local tissues;

[0135] By fitting an affine transformation model using RANSAC, the global rigid motion caused by probe translation and rotation is separated from the local non-rigid deformation, thus obtaining a residual displacement field. ;

[0136] For each spatial location Calculate its residual strain amplitude:

[0137] ;

[0138] When the residual strain amplitude When the value exceeds a preset threshold, the position is determined to be at time [time value missing]. Excessive pressure occurs;

[0139] The sum of all (pixel, time) pairs experiencing overcompression within the current window T is calculated, and then divided by the total number of pixels and the total number of frames to obtain the spatiotemporal proportion of overcompression. ,use Normalization is performed in a way that yields the normalized over-suppression score. ,in It is a preset reference threshold, for example, set A value of 0.05 is used to indicate that a 5% spatiotemporal proportion is considered significantly too high.

[0140] For potential damage density In the HSV color space, the proportion of pixels in the monitoring image that meet the preset bleed condition. For example, the preset bleed condition can be set as follows: in the HSV color space, H∈[0,30]∪[330,360], which means it belongs to the red hue, and S>the preset saturation threshold. ;

[0141] Obtain the pixel ratio that meets the preset bleed condition. Time difference

[0142] ;

[0143] when If the bleeding exceeds a preset threshold and continues for several frames, it is marked as a "suspected bleeding event".

[0144] The local binary mode LBP histogram of the image is obtained, and the chi-square distance between the LBP histograms of consecutive frames is compared. When the chi-square distance exceeds a preset threshold, it indicates that a sudden change in local texture has occurred, which may indicate mucosal damage, and is marked as a "texture mutation event".

[0145] The number of all unique suspected hemorrhage or texture mutation events within a statistical window T. ,according to Obtain event density in this way and combined Normalization is performed in the manner described above. To obtain the potential damage density by setting a reference density. .

[0146] To obtain a stable overall objective performance score, a sliding window analysis was performed on the entire microendoscopic operation flow, with the window length set as . For the first window Calculate its overall objective effectiveness score. ,

[0147] Right now Perform calculations;

[0148] in, , , To preset positive weights, satisfying ;

[0149] The , , windows respectively The average value of the corresponding terms within the term.

[0150] Step S2-2: Generate the conditional Pareto front.

[0151] Specifically, within the dynamic and efficient operation library, the data in the library is grouped based on preset anatomical site labels to generate multiple anatomical site groups;

[0152] For the anatomical site group, the corresponding conditional Pareto front is obtained based on the preset microscopic endoscopy operation meta-indicators in the operation analysis window.

[0153] In one possible embodiment, within a dynamic high-efficiency operation library, the data is grouped according to anatomical location labels to generate multiple anatomical location groups. For each high-efficiency window h within a group, the average velocity is calculated by weighting the MS(t) sequence within that window and combining it with the feature velocity center value corresponding to each semantic label. For example, if "Quick Search" accounts for 30% of the window with a speed center of 20mm / s; "Smooth Scan" accounts for 60% with a speed center of 8mm / s; and "Fine Observation" accounts for 10% with a speed center of 1mm / s, then... ≈0.3*20+0.6*8+0.1*1=10.9 mm / s;

[0154] Within this high-efficiency window h, within each anatomical site group, each high-efficiency window is considered as a two-dimensional efficiency space. , A is a point in space A that dominates point B if and only if point A has a velocity not less than that of point B. Points at or above B, and strictly superior to B in at least one dimension, that are not dominated by any other point, constitute the Pareto front of this group. This front depicts the highest information integrity achievable at a given velocity in that particular anatomical location. For example, the front of the antrum might show that, in When the speed is 10 mm / s, the optimal value is... It can reach 0.85; if you want to increase the speed to =15mm / s, then the leading edge indicates the optimal... It will drop to around 0.78.

[0155] Step S3: Obtain the environmental status data corresponding to the microendoscopic operation data stream, analyze the operation to be evaluated in conjunction with the preset strategy effectiveness evaluation mechanism, and obtain the strategy effectiveness evaluation results.

[0156] Specifically, environmental state features are extracted from the video frames corresponding to the microendoscopic operation data stream to obtain environmental state data, which includes at least the exploration completeness and local task complexity.

[0157] The operator policy network is trained based on the microendoscopic operation data stream, and the operator policy network is used to simulate the operator's decision-making mechanism.

[0158] Furthermore, strategy effectiveness is evaluated based on a preset strategy effectiveness evaluation index set to obtain a strategy effectiveness index. The preset strategy effectiveness evaluation index set includes at least strategy entropy, environmental action mutual information, and goal orientation index.

[0159] The operator policy network performs simulation prediction based on the environmental state data and generates a simulation action sequence.

[0160] The environmental state data is injected with perturbations to generate perturbed environmental state data. The operator policy network performs simulation prediction based on the perturbed environmental state data to generate a perturbed simulation action sequence.

[0161] Furthermore, based on a preset set of strategy environment stability indicators, the simulated action sequence and the disturbance simulated action sequence are analyzed to obtain the strategy stability index.

[0162] Based on the strategy effectiveness index and strategy stability index, a strategy effectiveness evaluation result is generated.

[0163] In one possible embodiment, environmental state data is acquired; specifically, the completeness of the data is investigated. Directly reuse the coverage integrity in step S2 ,Right now ;

[0164] Local task complexity ;

[0165] in, For the local entropy of the image, ,in This represents the gray-level histogram distribution of a local region of the image.

[0166] This represents the sum of the pixel percentages of various suspected lesions in the current frame;

[0167] The average value of the blood vessel enhancement filter response amplitude, for example, for the input video frame. The image was converted to a color space more suitable for vascular analysis, the green channel was extracted, and multi-scale vascular enhancement filtering was performed using a Frangi filter based on the Hessian matrix to obtain a multi-scale vascular enhancement response map. The multiscale vascular enhancement response map The mean as ;

[0168] , , These are the weighting coefficients for the corresponding items. For the Sigmoid function; A value close to 1 indicates a complex tissue situation, numerous suspected targets, and dense blood vessels within the field of view, making the operation difficult; a value close to 0 indicates a flat and simple field of view.

[0169] This allows us to obtain the current environmental state time series. It is then timestamped and aligned with the original motion primitive sequence MS(t) and the device event sequence DE(t). For example, a point in time might correspond to... =[0.45,0.8] indicates that at the 120th second of the operation, the overall exploration was 45% complete, but the current local field of view complexity is very high.

[0170] Time-aligned sequences Transform the sample pairs required for supervised learning, given the length of the historical context. If we assume the sampling rate is 2Hz, then =10 represents data from the past 5 seconds;

[0171] Operator policy network constructed using Long Short-Term Memory (LSTM) network The historical context is encoded and then followed by a fully connected layer to output action probabilities. Specifically, this involves encoding the environmental state sequence corresponding to the historical context. Input the LSTM layer of this policy network and output the hidden state at the last time step. Then, it is mapped through a fully connected layer, and the probability distribution of each possible action is output through the Softmax function, that is, the probability distribution of (motion state, device event combination);

[0172] Operator Policy Network The objective is to maximize the log-likelihood of the observed action sequence under the model, using negative log-likelihood loss.

[0173] Right now ;

[0174] in, To adjust the size of the analysis window, For the operator policy network in input Next, assign to real actions The probability; parameters are optimized using backpropagation and gradient descent algorithms. , minimize ;

[0175] A trained operator policy network is a differentiable function that, for any given environmental state history, outputs an action probability distribution, thereby simulating the operator's decision-making tendencies.

[0176] Based on the operator policy network, each time point in the operator's entire operation... Calculation model output distribution Shannon entropy ,in, It is the set of all possible actions;

[0177] The average policy entropy corresponding to the entire operation sequence For example, a highly automated, pattern-fixed expert, whose It might be low, indicating that similar actions are always chosen in similar situations; while a novice might show a higher level. This indicates that decision-making is more hesitant and exploratory.

[0178] Use the KSG estimator to obtain the environment state With action Mutual information between , This refers to mutual information between the environment and actions. For example, discretizing the continuous environmental state space and then calculating the discretized environmental state variables. With action mutual information of experience ,

[0179] Right now ;

[0180] in yes and The joint empirical probability, and It is the marginal probability.

[0181] At multiple randomly selected time points Construct the environmental state history of two perturbations. and It should be noted that, and With true history The only difference is that the most recent moment Replace with and , It is a small positive number, such as 0.05;

[0182] Will and Input Operator Policy Network This yields two action probability distributions. and , obtain and In a predefined set of promotional actions Total probability difference of the middle action The set of promotional actions It is represented as a set of actions that can expand the detection coverage, such as "smooth scan" and "fast search";

[0183] Target-oriented index This is expressed as the average of all these differences.

[0184] Right now ;

[0185] in, The number of sampling points. A value greater than 0 indicates that the strategy tends to choose actions that can advance the exploration when the exploration completeness is slightly insufficient, which reflects a strong goal orientation.

[0186] Will , and The three indicators are normalized to the [0,1] interval, and then weighted and merged according to the preset weights. Finally, the result is multiplied by 100 to obtain a score of [0,100], which is the strategy effectiveness index.

[0187] It should be noted that, , and The normalization of all three indicators adopts the percentile mapping method based on the statistical distribution of the dynamic efficient operation library. Specifically, for Obtain the operator to be evaluated The dynamic and efficient operation library consists of each operation corresponding to... In the constructed distribution, the 10th percentile P10 and the 90th percentile P90 are obtained, where P10 represents the excellent level and P90 represents the passing level. These are then combined with a pre-defined piecewise function to... Perform a mapping to generate a normalized value; similarly, for and Obtain the operator to be evaluated and The dynamic and efficient operation library consists of each operation corresponding to... and In the constructed distribution, the 90th percentile P90 and the 10th percentile P10 are extracted respectively. P90 is regarded as the excellent level and P10 as the passing level, and combined with the preset piecewise function, and Perform a mapping to generate a normalized value.

[0188] At each time step, the operator policy network Based on the historical output action probability distribution, an action is sampled according to probability. Generate simulated behavior sequences The dynamic time warp distance (DTW) between the simulated sequence and the real sequence was calculated. A small DTW distance indicates that the behavior pattern generated by the model is highly similar to the operator's real behavior, thus proving the effectiveness of the model.

[0189] In a real-world environment state sequence, the following perturbation is injected at multiple randomly selected time points: a complexity spike perturbation, at the selected time points. ,Will The value was changed to ,in It is a relatively large increase, such as 0.6, and continues. Duration, such as 3 time steps, or 1.5 seconds, simulates the sudden appearance of hemorrhage or complex structures within the field of vision; field of vision reset perturbation, at a selected time point. ,Will The value was changed to ,in This is the reset value, such as 0.2, simulating the loss of some explored areas due to the probe accidentally slipping off.

[0190] Generate a sequence of states of the perturbed environment after the perturbation is injected. The disturbed environment state sequence Input Operator Policy Network The perturbation simulation action sequence is obtained;

[0191] For each perturbation event in the perturbation simulation action sequence, obtain the average recovery steps. With policy entropy perturbation change Specifically:

[0192] For average recovery steps Calculate the histogram of the primitive distribution of simulated actions within the time window before the disturbance; after the disturbance occurs, calculate the action primitive distribution within a window of the same length as the time window starting at each subsequent time point using a sliding window method; calculate the Jaccard distance between this action primitive distribution and the primitive distribution histogram; when the Jaccard distance first decreases and stabilizes below a preset threshold, the behavior pattern is considered to have recovered; count the number of steps from the start of the disturbance to the recovery point, which is the recovery step count for this disturbance event; and calculate the average recovery step count. This is the average number of recovery steps for all perturbation events;

[0193] Obtain the average policy entropy during the period of disturbance, such as the average policy entropy from the start of the disturbance to 10 seconds thereafter, and the difference between this and the average policy entropy before the disturbance. This difference represents the policy entropy change during the disturbance. ;

[0194] The average recovery steps With policy entropy perturbation change The two indicators are normalized and weighted, then multiplied by 100 to obtain a score in the range [0, 100], which is defined as the strategy stability index.

[0195] It should be noted that the average number of recovery steps With policy entropy perturbation change Both methods employ percentile mapping based on the statistical distribution of a dynamic, efficient operation library. Specifically, for the average number of recovery steps... Obtain the operator to be evaluated The dynamic and efficient operation library consists of each operation corresponding to... In the constructed distribution, the 10th percentile P10 and the 90th percentile P90 are obtained, where P10 represents the excellent level and P90 represents the passing level. These are then combined with a pre-defined piecewise function to... Perform a mapping to generate a normalized value; similarly, for policy entropy perturbation changes... Obtain the operator to be evaluated The dynamic and efficient operation library consists of each operation corresponding to... From the constructed distribution, the 50th percentile P50 and the 90th percentile P90 are extracted and combined with a preset piecewise function. Perform a mapping to generate a normalized value.

[0196] The strategy effectiveness index, strategy stability index, and related indicator parameters are encapsulated into a strategy effectiveness evaluation result and output.

[0197] Step S4: Perform causal analysis on the operation to be evaluated based on the Bayesian structural equation model, and combine the dynamic and efficient operation library to perform counterfactual attribution to generate a causal analysis dataset.

[0198] In this embodiment, step S4 includes:

[0199] Step S4-1: Perform causal analysis on the operation to be evaluated.

[0200] Specifically, intention inference is performed on the original motion primitive sequence based on a hidden Markov model to obtain intention variables;

[0201] Action variables and response variables are extracted from the sequence of microendoscopic video frames corresponding to the operation to be evaluated. The action variables are used to characterize the actions performed by the operator, and the response variables are used to characterize the quality of the microendoscopic video frames.

[0202] Perception variables are obtained according to a preset perception analysis mechanism, and these perception variables are used to characterize the operator's perception state.

[0203] Furthermore, a Bayesian structural equation model is constructed based on the intent variable, extracted action variable, response variable, and perception variable. The Markov chain Monte Carlo method is used to estimate the Bayesian structural equation model to obtain a causal estimation parameter set.

[0204] In one possible embodiment, supervised learning of the Hidden Markov Model is performed based on a dynamic and efficient operation library. Specifically, the model parameters include the initial state distribution, i.e., the probability that each macroscopic intention is the starting point of an operation; the state transition matrix, i.e., the probability of transitions between intentions over time; and the observation probability matrix, i.e., the likelihood of observing each low-level semantic motion state given a certain macroscopic intention.

[0205] The training data for the Hidden Markov Model uses segments labeled by domain experts from a dynamic and efficient operation library. When watching the efficient operation videos, experts assign macro-intention labels such as "systematic exploration", "local fine examination", and "target confirmation / biopsy" to consecutive MS(t) sequence segments based on clinical procedures and context.

[0206] Supervised learning is employed, and by minimizing the negative log-likelihood loss, the model learns the optimal probability mapping from macro-intention to micro-behavior contained in the training data, as well as the dynamic pattern of intention evolution.

[0207] After training, for the MS(t) sequence to be evaluated, the Viterbi algorithm is used for decoding. Dynamic programming is used to find the hidden state sequence that can generate a given observation sequence with the maximum posterior probability, thereby outputting the most likely macroscopic intention sequence. , i.e., intention variables.

[0208] Action variables were extracted from the microendoscopic video frame sequence corresponding to the operation to be evaluated. The action variable comprises a discrete part and a continuous part, the discrete part being derived from the device event sequence. Extract from, Converted into a one-hot encoded vector If there are no events, it is a zero vector; the continuous part is obtained by extracting features from the original motion primitive sequence, i.e. ,in and These are the resultant velocity and resultant acceleration of the probe at the current moment, respectively; This can be represented as the concatenation of the above vectors, i.e. .

[0209] Physical response indicators are extracted in real time from the sequence of microendoscopic video frames corresponding to the operation to be evaluated, and response variables are generated based on the extracted indicators. The physical response metrics include at least image sharpness, i.e., the current frame. The variance of the Laplacian operator; field-of-view stability, i.e., calculating the frame using the dense optical flow method. arrive The average displacement amplitude; the spatial average of the absolute value of the divergence of the optical flow field, i.e., the organizational deformation; the response variable. =[Image sharpness, visual stability, tissue deformation].

[0210] Based on the eye-tracking features, MS(t) state features, DE(t) event features, and R(t) vector corresponding to the current operation, a perceptual feature vector is generated. This perceptual feature vector is then input into a classifier, which outputs a […]. , , , The probability vector is used to determine the perceived state estimate at that moment, and the category with the highest probability is selected. Specifically:

[0211] For eye movement features, calculate the coordinates of the current gaze point. A radius of The average image sharpness within a circular region of a pixel; if this average value exceeds a preset high-resolution threshold, a matching index is assigned. =1 indicates that the gaze point falls within the information-rich area; if this average value is lower than the preset low-resolution threshold, a matching score is assigned. =-1 indicates that the gaze may be on a blurry or uninformative region; otherwise, it is 0.

[0212] If a rapid saccade with both amplitude and peak velocity exceeding a corresponding preset threshold is detected within the time window, and the saccade endpoint falls within the image edge region, such as the outer 20% of the image width and height, then a corrective saccade flag is assigned to it. =1, otherwise 0. This flag is used to indicate that the operator is actively searching for a lost field of vision or adjusting the gaze target.

[0213] For the MS(t) state characteristic and the DE(t) event characteristic, if the current MS(t) = "judgment paused", then assign a value. =1, otherwise 0. This indicator is used to characterize the operator's cognitive assessment.

[0214] If DE(t) contains any event belonging to the "adjustment" category within the time window, then assign a... Otherwise, it is 0. This value is used to indicate that the operator has recently taken proactive corrective or optimization actions.

[0215] For the response variable R(t), the values ​​of its normalized dimensions can be directly used. Standard normalization can be performed based on the global mean and standard deviation of each indicator of the response variable R(t) in the dynamic efficient operation library.

[0216] The generated feature labels are directly encapsulated into vector form to generate perceptual feature vectors.

[0217] It should be noted that the classifier is constructed using a gradient boosting decision tree, and its training data is data contained in a dynamic and efficient operation library. For example, senior endoscopy experts are invited to replay videos of efficient operation segments and simultaneously view the eye-tracking overlay map and behavioral event markers of the segments. Experts label each frame with the following perceptual state labels: clear, blurry, suspicious discovery, and loss of vision. The training process uses the cross-entropy loss function to minimize the loss until the loss value is lower than a preset threshold or the preset maximum number of training steps is reached.

[0218] The MS(t) sequence of the subject to be evaluated is compared with each prototype in the efficient policy prototype set. By comparing typical state transition sequences, the distance between sequences is calculated using the Dynamic Time Warping (DTW) algorithm. The prototype with the smallest DTW distance is selected as the dominant matching prototype. The efficient strategy prototype set is represented by a strategy set obtained by periodically performing sequence clustering on the dynamic efficient operation library. Each prototype includes: typical semantic state transition probability matrices such as prototype center sequences, average motion feature vectors, and statistics of device operation triggering conditions.

[0219] For each time point t, the following linear structure equations are established:

[0220] ;

[0221] ;

[0222] ;

[0223] ;

[0224] in, , , , , The coefficient matrix or vector to be estimated;

[0225] , , , The residuals follow a multivariate normal distribution.

[0226] , , , Let be the covariance matrix of the corresponding terms.

[0227] A Bayesian approach is employed, using Markov chain Monte Carlo sampling to estimate the posterior distribution of the aforementioned parameters. Specifically, weakly informative priors are set for all parameters to be estimated, and it is assumed that the data is generated by the aforementioned linear Gaussian model. A large number of samples are drawn from the posterior distribution of the parameters using the NUTS sampler. Based on the sampling results, the posterior mean, standard deviation, and 95% highest density interval for each parameter are calculated. For example, it is possible to obtain... The posterior mean is [0.85, 0.12, ...], indicating that the mapping strength from the intention of "systematic exploration" to the action of "rapid search" is 0.85, and the mapping strength to the event of "focusing" is 0.12.

[0228] By respectively and The effective elements of the vector are compared with the posterior mean distribution of the parameters of the same strategy in the dynamic and efficient operation library, their percentiles are calculated, and then the average is taken and mapped to [0,100] points to obtain the execution accuracy index.

[0229] Based on parameters The posterior mean was compared with the distribution of the efficient database and then standardized to obtain the perceptual accuracy index;

[0230] Based on parameters The decision quality index is obtained by standardizing the posterior mean of the data after comparing it with the distribution of the efficient database.

[0231] The execution accuracy index, perception accuracy index, and decision quality index together constitute the causal estimation parameter set.

[0232] For example, suppose we are currently evaluating an operator's actions in the antrum of the stomach, and the dominant matching prototype is a "precision" strategy. Using the Bayesian structural equation model (BSEM) mentioned above, we estimate the transition from the intention of "fine observation" to the action of "low-speed, steady movement." The mean value is 0.82, indicating the effect of "slow, steady motion" on "image sharpness". The mean is 0.75; the probability that the "image sharpness" signal is correctly perceived. The mean value is 0.90, representing the intensity of the "adjustment" intention triggered after perceiving a "fuzzy" state. The mean is 0.60;

[0233] From the dynamic high-efficiency operation library, all high-efficiency operation segments marked as "precise" strategies and located in the gastric antrum are extracted. BSEM estimation is performed independently on each segment to obtain the posterior mean of the parameters for each segment, thereby forming a high-efficiency population distribution for each parameter. The parameter values ​​of the operators to be evaluated are then compared with the corresponding high-efficiency population distribution.

[0234] calculate =0.82 and =0.75 percentile in their respective efficient distributions, assuming 0.82 in... It is located at the 30th percentile in the distribution, with a value of 0.75. Located at the 4th percentile in the distribution, the arithmetic mean of the two percentiles is taken to obtain the composite percentile of (30+40) / 2=35. The calculated percentile is directly linearly mapped to a score in the range [0,100], which is the execution precision index.

[0235] Similarly, the calculation yields =0.90 is the 20th percentile in the efficient distribution, and =0.60 is the 50th percentile in the efficient distribution, thus the perception accuracy index and decision quality index are 20 and 50, respectively.

[0236] Step S4-2: Perform counterfactual attribution on the operation to be evaluated.

[0237] Specifically, based on a comprehensive objective performance score, low-efficiency periods are identified in the current microendoscopic operation process to obtain a set of low-efficiency periods;

[0238] For each inefficient time period within the inefficient time period set, an ideal reference segment is selected from the dynamic efficient operation library. Based on the ideal reference segment, counterfactual simulation is performed in the corresponding inefficient time period to obtain counterfactual attribution results.

[0239] In one possible embodiment, based on the sliding window curve of the comprehensive objective performance score, a continuous period of time in which all window scores are lower than a preset threshold is found and defined as a low-efficiency period. For each low-efficiency period, a segment with the same anatomical location, the same dominant matching prototype type, a comprehensive objective performance score corresponding to the segment exceeding the 80th percentile, and the smallest DTW distance on the environmental state sequence corresponding to the current low-efficiency period is selected from the dynamic high-efficiency operation library as an ideal reference segment.

[0240] The above BSEM estimation is performed independently on the ideal reference segment to obtain the mean of its posterior parameter distribution. , , , Based on the estimated parameters, we perform perfect counterfactual and decision-making simulations.

[0241] The execution of a perfect counterfactual simulation is expressed as follows: for and The corresponding linear structural equation will evaluate the parameters of the entity to be evaluated. , Replace with , At the same time, fix its , , The sequence represents the actual values, and the counterfactual actions and responses are recursively calculated based on the corresponding equations.

[0242] ;

[0243] in This is expressed as the sum of the L2 norms of the response vector differences over the entire time period. For actual response, In order to execute a counterfactual response, The actual response is for the ideal reference segment;

[0244] The perfect counterfactual simulation of the decision is expressed as follows: For and The corresponding linear structural equation will evaluate the parameters of the entity to be evaluated. , Replace with , At the same time, fix its )sequence, , For the actual values, the counterfactual perception and counterfactual intent of the decision are recursively calculated according to the corresponding equations, and then the counterfactual intent of the decision is substituted into... and The corresponding linear structure equation yields the counterfactual response of the decision.

[0245] Similarly, ;

[0246] in This is expressed as calculating the difference in response vectors over the entire time period. The sum of norms For actual response, Counterfactual response to decision-making The actual response is for the ideal reference segment;

[0247] Counterfactual attribution is performed based on the aforementioned error contribution and decision error contribution to obtain counterfactual attribution results. Specifically:

[0248] If the contribution of execution error is much greater than that of decision error, it indicates that the inefficiency mainly stems from the execution stage; conversely, it indicates that it stems from the decision-making or perception stage; if both are high, it indicates that both have significant problems.

[0249] If the strategy efficiency index is lower than the lower limit of the preset threshold range when a period of low efficiency is identified, it indicates that the strategy used in this operation is generally low in information utilization efficiency.

[0250] If the strategy stability index is lower than the lower limit of the preset threshold range, it indicates that the current strategy is weak in responding to environmental disturbances.

[0251] Finally, the causal estimation parameter set and the counterfactual attribution results are encapsulated into a causal analysis dataset.

[0252] Step S5: Perform a comprehensive operational evaluation based on the comprehensive objective performance score, conditional Pareto front, strategy performance evaluation results, and causal analysis dataset to generate a comprehensive operational evaluation result.

[0253] Specifically, an operational efficiency score is obtained based on the conditional Pareto front; a strategy effectiveness score is obtained based on the strategy effectiveness evaluation results; and a causal association score is generated based on the causal analysis dataset.

[0254] Based on the comprehensive objective performance score, operational efficiency score, strategy performance score, and causal relationship score, a comprehensive operational score and operational improvement suggestions are generated, and the comprehensive operational score and operational improvement suggestions are packaged into a comprehensive operational evaluation result.

[0255] In one possible embodiment, the coordinates of the projection point of the operation to be evaluated on the conditional Pareto front of the corresponding anatomical site are obtained. , ),in, This represents the average probe movement speed during the exploration phase of a specific anatomical site in the procedure being evaluated. The information integrity index of the operation to be evaluated at the corresponding anatomical site, and its shortest Euclidean distance to the frontier. And convert it into an operational efficiency score of [0, 100]. ,

[0256] Right now ;

[0257] in, This represents the maximum distance from the Pareto front to each point of the current anatomical location in the dynamic, efficient manipulation library;

[0258] The comprehensive objective performance score is also linearly mapped to [0,100]. The strategy performance index and strategy stability index in the strategy performance evaluation results are defined as the first and second strategy performance scores, respectively. Similarly, the execution accuracy index, perception accuracy index and decision quality index contained in the causal estimation parameter set in the causal analysis dataset are defined as the first, second and third causal association scores, respectively.

[0259] Based on the preset weights, the comprehensive objective performance score, operational efficiency score, first and second strategy performance scores, and first, second, and third causal relationship scores are weighted and integrated to generate a comprehensive operational score.

[0260] Based on the counterfactual attribution results contained in the causal analysis dataset, corresponding operational improvement suggestions are generated. For example, "For the critical event of a 20% decrease in image sharpness around the 180th second, causal attribution analysis shows that the error stems from the failure to detect the gradual defocusing of the image in time," the operational improvement suggestion is generated as "It is recommended to conduct two weeks of modular basic training, focusing on strengthening hand stability and fine pressure control."

[0261] Figure 3 This is a schematic diagram of a microendoscopic operation evaluation system according to the present invention.

[0262] Specifically, a microendoscopic operation evaluation system includes:

[0263] The feature extraction module is used to preprocess and extract features from the microendoscopic operation data stream to obtain the original motion primitive sequence and device event sequence.

[0264] The operation evaluation module is used to obtain the comprehensive objective performance score in the operation analysis window, and generate the conditional Pareto front for each anatomical site by combining the dynamic high-efficiency operation library.

[0265] The strategy evaluation module is used to acquire environmental status data and analyze the operation to be evaluated in conjunction with a preset strategy effectiveness evaluation mechanism to obtain the strategy effectiveness evaluation result.

[0266] The causal analysis module is used to perform causal analysis on the operation to be evaluated, and to perform counterfactual attribution in conjunction with the dynamic and efficient operation library to generate a causal analysis dataset.

[0267] The comprehensive evaluation module is used to perform a comprehensive operational evaluation based on the comprehensive objective performance score, the conditional Pareto front, the strategy performance evaluation results, and the causal analysis dataset, and generate a comprehensive operational evaluation result.

[0268] This embodiment provides an electronic device, which may include: at least one processor, at least one network interface, a user interface, a memory, and at least one communication bus.

[0269] It should be noted that the above formulas are all dimensionless calculations. The formulas are derived from software simulations based on a large amount of collected data to obtain the most recent real-world results. The preset parameters in the formulas are set by those skilled in the art according to the actual situation.

[0270] The above embodiments can be implemented, in whole or in part, through software, hardware (such as circuits), firmware, or any other combination thereof.

[0271] When implemented using software, the above embodiments can be implemented in whole or in part as a computer program product, which includes one or more computer instructions or computer programs; when the computer instructions or computer programs are loaded or executed on a computer, the processes or functions described in the embodiments of the present invention are generated in whole or in part.

[0272] It is understood that the computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device; the computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via transmission methods such as infrared, wireless, or microwave; the computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more sets of available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium. A semiconductor medium can be a solid-state drive.

[0273] It should be understood that the term "and / or" in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent three cases: A alone, A and B simultaneously, and B alone. A and B can be singular or plural. Additionally, the character " / " in this article generally indicates an "or" relationship between the preceding and following related objects, but it can also represent an "and / or" relationship. Please refer to the context for a more accurate understanding.

[0274] It should be understood that, in the embodiments of the present invention, the order of the above-mentioned process numbers does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0275] The above-described embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included within the protection scope of the present invention.

Claims

1. A method for evaluating microendoscopic operation, characterized in that, It includes the following steps: The acquired microendoscopic operation data stream is preprocessed, and features are extracted from the preprocessed data stream to generate the original motion primitive sequence and the device event sequence. Based on preset microendoscopic operation meta-indicators, a comprehensive objective performance score is obtained in the operation analysis window. Simultaneously, combined with a dynamic high-efficiency operation library, a conditional Pareto front for each anatomical site is generated. The dynamic and efficient operation library is used to store efficient microendoscopic operations, and each efficient microendoscopic operation includes at least the corresponding original operation segment, comprehensive objective performance score and descriptive label. The descriptive label includes at least the semantic label and device event label of the original motion primitive sequence MS(t) and the device event sequence DE(t) corresponding to the operation sequence. The environmental status data corresponding to the microendoscopic operation data stream is acquired, and the operation to be evaluated is analyzed in combination with the preset strategy effectiveness evaluation mechanism to obtain the strategy effectiveness evaluation results. Causal analysis of the operation to be evaluated is performed based on the Bayesian structural equation model, and counterfactual attribution is performed in combination with the dynamic and efficient operation library to generate a causal analysis dataset. A comprehensive operational evaluation is performed based on the comprehensive objective performance score, conditional Pareto front, strategy performance evaluation results, and causal analysis dataset to generate a comprehensive operational evaluation result.

2. The method for evaluating microendoscopic operation according to claim 1, characterized in that, The acquired microendoscopic operation data stream is preprocessed, and features are extracted from the preprocessed data stream to generate the original motion primitive sequence and device event sequence, including: The microendoscopic operation data stream includes at least the raw motion signal sequence, the device operation log, and the corresponding timestamp; The original motion signal sequence is divided into multiple motion segments of fixed size, and features are extracted from each motion segment to obtain a motion segment feature set; A Gaussian mixture model is used to cluster the feature set of the motion segments to generate multiple original motion pattern clusters; The original motion signal sequence is processed based on the original motion pattern cluster to generate the original motion primitive sequence; Extract device events from the device operation log that are time-aligned with the original motion primitive sequence to generate a device event sequence.

3. The method for evaluating microendoscopic operation according to claim 1, characterized in that, Based on preset microendoscopic operation meta-indicators, a comprehensive objective performance score is obtained within the operation analysis window. Simultaneously, combined with a dynamic high-efficiency operation library, a conditional Pareto front for each anatomical location is generated, including: The preset microendoscopic operation meta-indicators include at least the information integrity index, target acquisition efficiency, and physiological disturbance index. A combination of visual SLAM and image sharpness analysis was used to analyze the current video stream of microscopic endoscopy operations and obtain the information integrity index. By combining a pre-trained lesion detection model, the proportion of lesions effectively captured by the operator during the current microendoscopic operation is obtained, and the target capture efficiency is generated based on the proportion of effectively captured lesions. Physiological disturbance index of current microendoscopic operation is obtained by using optical flow and tissue deformation analysis; The current microscopic endoscopy operation is divided into multiple operation analysis windows according to the preset operation analysis window capacity; Based on the preset microscopic endoscopy operation meta-indicators in the operation analysis window, a comprehensive objective performance score is obtained. By combining a dynamic and efficient operation library, a conditional Pareto front is generated for the corresponding anatomical site of the current microendoscopic operation.

4. The method for evaluating microendoscopic operation according to claim 3, characterized in that, By combining a dynamic and efficient operation library, a conditional Pareto front is generated for the corresponding anatomical site of the current microendoscopic operation, including: The dynamic high-efficiency operation library is represented as a high-efficiency operation database constructed based on expert demonstration operation data, and the dynamic high-efficiency operation library is updated regularly based on the comprehensive objective performance score. The dynamic and efficient operation library is used to store efficient microendoscopic operations, and each efficient microendoscopic operation includes at least the corresponding original operation segment, comprehensive objective performance score and descriptive label. Within the dynamic and efficient operation library, the data in the library is grouped based on preset anatomical site labels to generate multiple anatomical site groups; For the anatomical site group, the corresponding conditional Pareto front is obtained based on the preset microscopic endoscopy operation meta-indicators in the operation analysis window.

5. The method for evaluating microendoscopic operation according to claim 1, characterized in that, Acquire environmental state data corresponding to the microendoscopic operation data stream, analyze the operation to be evaluated using a preset strategy effectiveness evaluation mechanism, and obtain the strategy effectiveness evaluation results, including: Environmental state features are extracted from the video frames corresponding to the microendoscopic operation data stream to obtain environmental state data; The environmental status data includes at least the exploration completeness and the local task complexity; The operator's policy network is trained based on the microendoscopic operation data stream, and the operator's policy network is used to simulate the operator's decision-making mechanism. Based on the operator policy network, policy effectiveness evaluation and policy environment stability analysis are performed to obtain policy effectiveness evaluation results.

6. The method for evaluating microendoscopic operation according to claim 5, characterized in that, Based on the operator policy network, policy effectiveness evaluation and policy environment stability analysis are performed to obtain policy effectiveness evaluation results, including: The strategy effectiveness is evaluated based on a preset set of strategy effectiveness evaluation indicators to obtain the strategy effectiveness index; The preset strategy effectiveness evaluation index set includes at least strategy entropy, environmental action mutual information, and goal orientation index; The operator policy network performs simulation prediction based on the environmental state data and generates a simulation action sequence. The environmental state data is injected with a disturbance to generate disturbed environmental state data; The operator policy network performs simulation prediction based on the disturbed environment state data and generates a sequence of disturbance simulation actions. Based on a preset set of strategy environment stability indicators, the simulated action sequence and the disturbance simulated action sequence are analyzed to obtain the strategy stability index. Based on the strategy effectiveness index and strategy stability index, a strategy effectiveness evaluation result is generated.

7. The method for evaluating microendoscopic operation according to claim 1, characterized in that, Causal analysis is performed on the operation to be evaluated based on the Bayesian structural equation model, and counterfactual attribution is performed in conjunction with the aforementioned dynamic and efficient operation library to generate a causal analysis dataset, including: Intent inference is performed on the original motion primitive sequence based on a hidden Markov model to obtain intent variables; Extract action and response variables from the microendoscopic video frame sequence corresponding to the operation to be evaluated; The action variables are used to characterize the actions performed by the operator; The response variable is used to characterize the quality of the microendoscopic video frame; Perception variables are obtained according to a preset perception analysis mechanism, and the perception variables are used to characterize the operator's perception state. A Bayesian structural equation model is constructed based on the aforementioned intention variables, extracted action variables, response variables, and perception variables. The Bayesian structural equation model is estimated using the Markov chain Monte Carlo method to obtain a set of causal estimation parameters.

8. The method for evaluating microendoscopic operation according to claim 7, characterized in that, The method further includes: Based on a comprehensive objective performance score, low-efficiency periods are identified in the current microendoscopic operation process, and a set of low-efficiency periods is obtained. For each inefficient time period within the inefficient time period set, select an ideal reference segment from the dynamic efficient operation library; Based on the ideal reference segment, counterfactual simulations are performed in the corresponding low-efficiency period to obtain counterfactual attribution results; The causal estimation parameter set and the counterfactual attribution results are encapsulated into a causal analysis dataset.

9. The method for evaluating microendoscopic operation according to claim 1, characterized in that, Based on the comprehensive objective performance score, conditional Pareto front, policy performance evaluation results, and causal analysis dataset, a comprehensive operational evaluation is performed to generate comprehensive operational evaluation results, including: Based on the conditional Pareto front, an operational efficiency score is obtained; Based on the strategy effectiveness evaluation results, obtain the strategy effectiveness score; Based on the aforementioned causal analysis dataset, a causal association score is generated; Based on the comprehensive objective performance score, operational efficiency score, strategy performance score, and causal relationship score, a comprehensive operational score and operational improvement suggestions are generated. The comprehensive operational score and operational improvement suggestions are packaged into a comprehensive operational evaluation result.

10. A microendoscopic operation evaluation system, used to implement the microendoscopic operation evaluation method according to any one of claims 1 to 9, characterized in that, include: The feature extraction module is used to preprocess and extract features from the microendoscopic operation data stream to obtain the original motion primitive sequence and device event sequence. The operation evaluation module is used to obtain the comprehensive objective performance score in the operation analysis window, and generate the conditional Pareto front for each anatomical site by combining the dynamic high-efficiency operation library. The strategy evaluation module is used to acquire environmental status data and analyze the operation to be evaluated in conjunction with a preset strategy effectiveness evaluation mechanism to obtain the strategy effectiveness evaluation result. The causal analysis module is used to perform causal analysis on the operation to be evaluated, and to perform counterfactual attribution in conjunction with the dynamic and efficient operation library to generate the causal analysis dataset. The comprehensive evaluation module is used to perform a comprehensive operational evaluation based on the comprehensive objective performance score, the conditional Pareto front, the strategy performance evaluation results, and the causal analysis dataset, and generate the comprehensive operational evaluation results.

Images