A minimally invasive surgery error prediction and skill evaluation network system based on expert eye movement data and a method thereof

Abstract

The application discloses a kind of based on expert eye movement data's minimally invasive surgery error pre-judgment and skill evaluation network system and method thereof, the method includes the following steps, S1, data acquisition and pre-processing step: video frame pre-processing to image device, obtain the eye movement signal when surgery expert analyzes video, generate and correspond to each video frame eye movement heat map;S2, network training step: first pre-training generator, then using the generator and discriminator of training generator and discriminant by antagonistic scheme;S3, surgery error pre-judgment and skill evaluation step: using the network of trained analysis surgery video, the frequency of occurrence of surgical error is counted and the skill evaluation of doctor is given accordingly;The method in the application does not need to position surgical instrument, by learning the eye movement mode when expert watches auxiliary image device video, combine artificial intelligence method, directly analyze the video output of auxiliary image device, complete the pre-judgment of surgical error, and give the evaluation of doctor surgery skill.

Description

Technical Field

[0001] This invention belongs to the fields of artificial intelligence and computer vision technology, and specifically relates to a network system and method for predicting errors and evaluating skills in minimally invasive surgery based on expert eye-tracking data. Background Technology

[0002] Compared to traditional surgery, minimally invasive surgery refers to a new technique that uses image-guided interventional devices to perform procedures within the body through natural cavities or vascular networks. It is characterized by less trauma, fewer complications, and faster postoperative recovery, representing a significant breakthrough in modern treatment. However, minimally invasive interventional surgery is more challenging than traditional surgery, facing clinical difficulties such as the small size of the interventional devices and the complexity of lesion localization. The clinical risks of minimally invasive interventional surgery are highly correlated with the surgeon's skill in operating the surgical instruments and their understanding of closed anatomical structures. Automated methods for predicting surgical errors and assessing skills can be used for surgical skills training and risk assessment, helping to reduce intraoperative harm and improve surgical safety, thus possessing significant research value and clinical significance.

[0003] Current methods for predicting surgical errors and evaluating surgical skills primarily focus on tracking the hand movements of surgeons and the movement of surgical instruments during specific surgical tasks. They utilize spatial positioning technology to determine the instrument's position, measure target distance errors, and compare these errors to pre-defined standards. Spatial positioning technology for surgical instruments can generally be divided into two categories: image-based positioning and electromagnetic positioning. Image-based positioning typically utilizes X-rays, intraoperative CT, and intraoperative MRI for instrument positioning. However, the use of X-rays generates repetitive radiation, posing safety concerns for both surgeons and patients. Intraoperative CT and MRI equipment are expensive, and MRI introduces magnetic compatibility issues, limiting the selection and use of surgical instruments. Electromagnetic positioning generally requires fixing a magnetic coil at the instrument's tip to generate and collect electromagnetic signals to track instrument movement. However, this method increases the size of the instruments, affecting the surgeon's operation and introducing magnetic compatibility problems.

[0004] Currently, there is no surgical skill evaluation method that does not introduce electromagnetic compatibility issues, ensures the safety of doctors and patients, does not interfere with the doctor's normal surgical operation, is inexpensive, and has a certain degree of accuracy that can be applied to minimally invasive surgery. Summary of the Invention

[0005] To address the problems in the prior art, this invention provides a network system and method for predicting errors and evaluating skills in minimally invasive surgery based on expert eye-tracking data. This method does not introduce electromagnetic compatibility issues, does not generate radiation that endangers the safety of doctors and patients, does not interfere with the normal surgical operation of doctors, and has a low cost, enabling it to be rapidly popularized and used.

[0006] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0007] A network system for predicting errors and evaluating skills in minimally invasive surgery based on expert eye-tracking data includes two CNN modules: a generator G and a discriminator D.

[0008] The generator G is a multi-task model, with the main task responsible for processing the input surgical video frames. The system categorizes whether errors occur during the process, and the auxiliary task is responsible for generating an attention map for experts when evaluating surgeries.

[0009] The discriminator D will generate the attention map from the generator G. and the real attention map The comparison is performed to classify whether the input eye-tracking heatmap is real or generated by the generator G.

[0010] Furthermore, the generator G employs a fully convolutional network. The first three convolutional blocks use 3x3 kernels in all convolutional layers. Specifically, the first convolutional block takes a 240x240 image input, is divided into two layers with 16 kernels each, and then performs 2x2 max pooling. The second convolutional block is also divided into two layers with 32 kernels each, followed by 2x2 max pooling. The third convolutional block consists of three layers with 64 kernels each. The feature map obtained from the third convolutional block... The attention maps are used for prediction in the auxiliary task and classification in the main task, respectively. The auxiliary and main tasks branch after the third convolutional block. The fourth convolutional block for both the auxiliary and main tasks uses the same structure: first, the feature maps are processed... Perform 2x2 max pooling, then pass through three convolutional layers with 128 3x3 kernels each to obtain 30x30 feature maps. and feature map The two maps are multiplied by a dot product and then input into the fifth convolutional block. They pass through three convolutional layers with 128 3x3 kernels each. After five convolutional blocks, the feature maps are fed into the adaptation layers for channel compression. The adaptation layers use 1x1 kernels. The first adaptation layer contains 256 kernels, and the second adaptation layer contains 2 kernels. After two adaptation layers, two 30x30 feature maps are obtained. Global average pooling is then performed on the two 30x30 feature maps, and softmax is used to obtain the classification score vector.

[0011] Furthermore, the discriminator D is a CNN network with three convolutional layers, having 64, 128, and 256 3x3 convolutional kernels respectively. Each layer has 2x2 max pooling and uses leaky ReLU activation. Three fully connected layers follow the convolutional layers; the first two are activated by tanh, and the last layer uses sigmoid activation. Since the expert's attention is related to the content of the video frames, the generated attention map... Or true attention map It is combined with the corresponding video frame I and used as the input of the discriminator D.

[0012] Furthermore, the generated attention map Or true attention map The pixel-level content loss between them is defined as Binary cross-entropy (BCE) is used as the metric; for classification tasks, cross-entropy loss is used and defined as follows: Take hyperparameters , , This is used to control the relative contributions of the three losses. Let the total number of sample video frames be m, and the th... Each video frame is The attention map generated based on it is The corresponding real attention map is Generator loss and discriminator loss Defined as:

[0013]

[0014]

[0015] in: This represents the probability that the discriminator successfully identifies the true attention map, while This represents the probability that the discriminator is fooled. This represents the weight of D.

[0016] A method for predicting errors and evaluating skills in minimally invasive surgery based on expert eye-tracking data includes the following steps:

[0017] S1. Data acquisition and preprocessing steps: preprocess the video frames of the imaging equipment, obtain the eye movement signals when the surgical expert analyzes the video, and generate an eye movement heatmap corresponding to each video frame.

[0018] S2. Network training steps: First, pre-train the generator, then train the generator and discriminator using an adversarial scheme;

[0019] S3. Surgical Error Prediction and Skill Evaluation Steps: Use a trained network to analyze surgical videos, count the frequency of surgical errors, and provide a skill evaluation for the doctor based on this.

[0020] Furthermore, step S1 includes the following steps:

[0021] S11. Preprocessing of surgical video dataset:

[0022] Frame data containing Doppler overlay is discarded, all frames are rescaled, and a 240x240 area is cropped, excluding the supplier logo and the device's GUI portion. Each frame of video is normalized by subtracting the average intensity value and dividing by the image pixel standard deviation.

[0023] S12. Eye movement signal acquisition:

[0024] The expert sits in front of a computer monitor and an eye tracker. After the eye tracker is calibrated, the pre-processed surgical video is played on the monitor. The expert is asked to watch the surgical video and analyze the surgeon's performance at key steps that reflect the surgeon's skill level. If the surgeon makes a mistake, the expert is asked to mark the video frame where the mistake occurred using the keyboard. During this process, the eye tracker is used to acquire and record the expert's eye movement signals.

[0025] As a preferred embodiment, the eye tracker is the Tobii Eye Tracker 4C, and the display is a 21-inch LCD screen.

[0026] S13. Generate eye-tracking heatmap:

[0027] Considering the high-frequency noise generated by eye tremors, a sliding window filter with a window size of 3 is applied to the obtained eye movement data. Eye movements with an angular velocity lower than 30 degrees per second are classified as fixations, while those with an angular velocity greater than or equal to 30 degrees per second are classified as saccades. Assuming the distance between the expert's eyes and the screen is 0.5m, the threshold for angular velocity is converted to a threshold in pixels per second. Fixations with a time interval of less than 7.5ms and a spatial angle change of less than 0.5 degrees are merged, while saccade data with a fixation time of less than 80ms are discarded. For ease of calculation, a truncated Gaussian kernel is used to generate an eye movement heatmap for each image, with the kernel size corresponding to a visual angle of 1.5 degrees.

[0028] Furthermore, step S2 includes the following steps:

[0029] S21, Pre-trained generator network:

[0030] The network was initialized using a zero-mean Gaussian distribution with a standard deviation of 0.01. Before the adaptation layer, each convolutional layer underwent batch normalization and dropout with a dropout rate of 0.2. Hyperparameters... Within an epoch, the hyperparameters are dynamically changed from 2 to 1 at a constant rate to allow the generator to focus on learning and generating attention maps before performing frame classification; It is set to 1 because classification is the network's primary task; hyperparameter Set to 0 to avoid adversarial training initially; pre-train the generator independently for 30 epochs.

[0031] S22, Adversarial Training of Discriminator and Generator:

[0032] The network is trained with 2000 steps, alternating between the discriminator and generator each step. Single-sided label smoothing is used during discriminator training, while the discriminator weights are not updated during generator training. The network is trained using Adam, with an initial learning rate set to... The batch size is 64, and 5-fold cross-validation is used.

[0033] Furthermore, step S3 includes the following steps:

[0034] S31. Surgical Error Prediction and Skill Evaluation:

[0035] The surgical video is preprocessed in the same way as in step S11 and input into the trained generator network. The generator predicts whether an error has occurred in the surgery frame by frame and gives a prediction score. A score threshold and a sliding window length and threshold are set. When the number of video frames that reach the prediction score threshold in the sliding window exceeds the set window threshold, it is determined that an error has occurred in the operator in that video segment.

[0036] Compared with the prior art, the present invention has the following beneficial effects:

[0037] 1. This invention directly predicts errors made by the surgical operator through video from auxiliary imaging equipment, without the need to install sensors on surgical instruments, which would affect the size of the surgical instruments, interfere with the doctor's operation, or introduce electromagnetic compatibility issues.

[0038] 2. This invention learns the eye movement patterns of experts and can imitate these patterns to generate eye movement heatmaps. It only requires inputting the expert's eye movement data during training, and has certain clinical usefulness.

[0039] 3. This invention has low practical cost and can be quickly promoted and used. Attached Figure Description

[0040] Figure 1 This is a schematic diagram of the network structure of the method of the present invention.

[0041] Figure 2 This is a flowchart illustrating the method of the present invention. Detailed Implementation

[0042] The present invention will be further described below with reference to embodiments.

[0043] Example 1

[0044] like Figure 1As shown, a network system for error prediction and skill evaluation in minimally invasive surgery based on expert eye-tracking data includes two CNN modules: a generator G and a discriminator D. The generator G is a multi-task model, with the main task responsible for evaluating the input surgical video frames. The system categorizes whether errors occur during the process, and the auxiliary task is responsible for generating the attention map of the expert during surgical evaluation; the discriminator D will then analyze the attention map generated by the generator G. and the real attention map The comparison is performed to classify whether the input eye-tracking heatmap is real or generated by the generator G.

[0045] As a preferred embodiment, the generator G employs a fully convolutional network. The first three convolutional blocks use 3x3 kernels in all convolutional layers. Specifically, the first convolutional block takes a 240x240 image input, is divided into two layers with 16 kernels each, and then performs 2x2 max pooling. The second convolutional block is divided into two layers with 32 kernels each, and then performs 2x2 max pooling. The third convolutional block is divided into three layers with 64 kernels each. The feature map obtained from the third convolutional block... The attention maps are used for prediction in the auxiliary task and classification in the main task, respectively. The auxiliary and main tasks branch after the third convolutional block. The fourth convolutional block for both the auxiliary and main tasks uses the same structure: first, the feature maps are processed... Perform 2x2 max pooling, then pass through three convolutional layers with 128 3x3 kernels each to obtain 30x30 feature maps. and feature map The two maps are multiplied by a dot product and then input into the fifth convolutional block. They pass through three convolutional layers with 128 3x3 kernels each. After five convolutional blocks, the feature maps are fed into the adaptation layers for channel compression. The adaptation layers use 1x1 kernels. The first adaptation layer contains 256 kernels, and the second adaptation layer contains 2 kernels. After two adaptation layers, two 30x30 feature maps are obtained. Global average pooling is then performed on the two 30x30 feature maps, and softmax is used to obtain the classification score vector.

[0046] As a preferred embodiment, the discriminator D is a CNN network with three convolutional layers, having 64, 128, and 256 3x3 convolutional kernels respectively. Each layer has 2x2 max pooling and uses leaky ReLU activation. The convolutional layers are followed by three fully connected layers; the first two are activated by tanh, and the last layer uses sigmoid activation. Since the expert's attention is related to the content of the video frames, the generated attention map... Or true attention map It is combined with the corresponding video frame I and used as the input of the discriminator D.

[0047] As a preferred approach, the generated attention map Or true attention map The pixel-level content loss between them is defined as Binary cross-entropy (BCE) is used as the metric; for classification tasks, cross-entropy loss is used and defined as follows: Take hyperparameters , , This is used to control the relative contributions of the three losses. Let the total number of sample video frames be m, and the th... Each video frame is The attention map generated based on it is The corresponding real attention map is Generator loss and discriminator loss Defined as:

[0048]

[0049]

[0050] in: This represents the probability that the discriminator successfully identifies the true attention map, while This represents the probability that the discriminator is fooled. This represents the weight of D.

[0051] Example 2

[0052] like Figure 2 As shown, a method for predicting errors and evaluating skills in minimally invasive surgery based on expert eye-tracking data includes the following steps:

[0053] S1. Data acquisition and preprocessing steps: preprocess the video frames of the imaging equipment, obtain the eye movement signals when the surgical expert analyzes the video, and generate an eye movement heatmap corresponding to each video frame.

[0054] S11. Preprocessing of surgical video dataset:

[0055] Frame data containing Doppler overlay is discarded, all frames are rescaled, and a 240x240 area is cropped, excluding the supplier logo and the device's GUI portion. Each frame of video is normalized by subtracting the average intensity value and dividing by the image pixel standard deviation.

[0056] S12. Eye movement signal acquisition:

[0057] The expert sits in front of a computer monitor and an eye tracker. After the eye tracker is calibrated, the pre-processed surgical video is played on the monitor. The expert is asked to watch the surgical video and analyze the surgeon's performance at key steps that reflect the surgeon's skill level. If the surgeon makes a mistake, the expert is asked to mark the video frame where the mistake occurred using the keyboard. During this process, the eye tracker is used to acquire and record the expert's eye movement signals.

[0058] As a preferred embodiment, the eye tracker is the Tobii Eye Tracker 4C, and the display is a 21-inch LCD screen.

[0059] S13. Generate eye-tracking heatmap:

[0060] Considering the high-frequency noise generated by eye tremors, a sliding window filter with a window size of 3 is applied to the obtained eye movement data (3 is the parameter of the sliding window filter, i.e., the window size of the sliding window). Eye movements with an angular velocity lower than 30 degrees per second are classified as fixation, and those with an angular velocity higher than or equal to 30 degrees per second are classified as saccades. Assuming the distance between the expert's eyes and the screen is 0.5m, the threshold of angular velocity is converted to a threshold of pixels per second. Fixation points with a time interval of less than 7.5ms and a spatial angle change of less than 0.5 degrees are merged, while saccade data with a fixation time of less than 80ms are discarded. For ease of calculation, a truncated Gaussian kernel is used to generate an eye movement heatmap for each image, with the kernel size corresponding to a visual angle of 1.5 degrees.

[0061] S2. Network training steps: First, pre-train the generator, then train the generator and discriminator using an adversarial scheme;

[0062] S21, Pre-trained generator network:

[0063] The network was initialized using a zero-mean Gaussian distribution with a standard deviation of 0.01. Before the adaptation layer, each convolutional layer underwent batch normalization and dropout with a dropout rate of 0.2. Hyperparameters... Within an epoch, the hyperparameters are dynamically changed from 2 to 1 at a constant rate to allow the generator to focus on learning and generating attention maps before performing frame classification; It is set to 1 because classification is the network's primary task; hyperparameter Set to 0 to avoid adversarial training initially; pre-train the generator independently for 30 epochs.

[0064] It is quite obvious from the formula that the hyperparameters When set to 0, this term is no longer present in the loss function, allowing adversarial training to be skipped initially; setting it to 1 enables the normal loss function.

[0065] S22, Adversarial Training of Discriminator and Generator:

[0066] The network is trained with 2000 steps, alternating between the discriminator and generator each step. Single-sided label smoothing is used during discriminator training, while the discriminator weights are not updated during generator training. The network is trained using Adam, with an initial learning rate set to... The batch size is 64, and 5-fold cross-validation is used.

[0067] S3. Surgical Error Prediction and Skill Evaluation Steps: Use a trained network to analyze surgical videos, statistically analyze the frequency of surgical errors, and provide a skill evaluation for the surgeon based on this.

[0068] The surgical video is preprocessed in the same way as in step S11 and input into the trained generator network. The generator predicts whether an error has occurred in the surgery frame by frame and gives a prediction score. A score threshold and a sliding window length and threshold are set. When the number of video frames that reach the prediction score threshold in the sliding window exceeds the set window threshold, it is determined that an error has occurred in the operator in that video segment.

[0069] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A network system for error prediction and skill evaluation in minimally invasive surgery based on expert eye-tracking data, characterized in that, It includes two CNN modules: a generator G and a discriminator D; The generator G is a multi-task model, with the main task responsible for processing the input surgical video frames. The system categorizes whether errors occur during the process, and the auxiliary task is responsible for generating an attention map for experts when evaluating surgeries. The discriminator D will generate the attention map from the generator G. and the real attention map The comparison is performed to classify whether the input eye-tracking heatmap is real or generated by the generator G; The generator G employs a fully convolutional network. The first three convolutional blocks use 3x3 kernels in all convolutional layers. Specifically, the first convolutional block takes a 240x240 image as input, is divided into two layers with 16 kernels each, and then performs 2x2 max pooling. The second convolutional block is also divided into two layers with 32 kernels each, followed by 2x2 max pooling. The third convolutional block consists of three layers with 64 kernels each. The feature map obtained from the third convolutional block... The attention maps are used for prediction in the auxiliary task and classification in the main task, respectively. The auxiliary and main tasks branch after the third convolutional block. The fourth convolutional block for both the auxiliary and main tasks uses the same structure: first, the feature maps are processed... Perform 2x2 max pooling, then pass through three convolutional layers with 128 3x3 kernels each to obtain 30x30 feature maps. and feature map The two are multiplied by a dot product and then input into the fifth convolutional block, passing through three convolutional layers with 128 3x3 kernels each. After a total of five convolutional blocks, the feature maps are fed into the adaptation layer for channel compression. The adaptation layer uses 1x1 kernels. The first adaptation layer contains 256 kernels, and the second adaptation layer contains 2 kernels. After two adaptation layers, two 30x30 feature maps are obtained. Global average pooling is then performed on the two 30x30 feature maps, and softmax is used to obtain the classification score vector. Since the expert's attention is related to the content of the video frames, the generated attention map Or true attention map It is combined with the corresponding video frame I and used as the input of the discriminator D.

2. The minimally invasive surgery error prediction and skill evaluation network system based on expert eye-tracking data according to claim 1, characterized in that, The discriminator D is a CNN network with three convolutional layers, having 64, 128, and 256 3x3 convolutional kernels respectively. Each layer has 2x2 max pooling and uses leaky ReLU activation. The convolutional layers are followed by three fully connected layers, the first two of which are activated by tanh and the last layer is activated by sigmoid.

3. The minimally invasive surgery error prediction and skill evaluation network system based on expert eye-tracking data according to claim 2, characterized in that, The generated attention map Or true attention map The pixel-level content loss between them is defined as Binary cross-entropy (BCE) is used as the metric; for classification tasks, cross-entropy loss is used and defined as follows: Take hyperparameters , , This is used to control the relative contributions of the three losses. Let the total number of sample video frames be m, and the th... Each video frame is The attention map generated based on it is The corresponding real attention map is Generator loss and discriminator loss Defined as: in: This represents the probability that the discriminator successfully identifies the true attention map, while This represents the probability that the discriminator is fooled. This represents the weight of D.

4. The method of the minimally invasive surgery error prediction and skill evaluation network system based on expert eye-tracking data according to any one of claims 1-3, characterized in that, Includes the following steps: S1. Data acquisition and preprocessing steps: preprocess the video frames of the imaging equipment, obtain the eye movement signals when the surgical expert analyzes the video, and generate an eye movement heatmap corresponding to each video frame. S2. Network training steps: First, pre-train the generator, then train the generator and discriminator using an adversarial scheme; S3. Surgical Error Prediction and Skill Evaluation Steps: Use a trained network to analyze surgical videos, count the frequency of surgical errors, and provide a skill evaluation for the doctor based on this.

5. The method of the minimally invasive surgery error prediction and skill evaluation network system based on expert eye-tracking data according to claim 4, characterized in that, Step S1 includes the following steps: Step S11: Preprocessing of the surgical video dataset: Frame data containing Doppler overlay is discarded, all frames are rescaled, and a 240x240 area is cropped, excluding the supplier logo and the device's GUI portion. Each frame of video is normalized by subtracting the average intensity value and dividing by the image pixel standard deviation. Step S12, Eye movement signal acquisition: The expert sits in front of a computer monitor and an eye tracker. After the eye tracker is calibrated, the pre-processed surgical video is played on the monitor. The expert is asked to watch the surgical video and analyze the surgeon's performance at key steps that reflect the surgeon's skill level. If the surgeon makes a mistake, the expert is asked to mark the video frame where the mistake occurred using the keyboard. During this process, the eye tracker is used to acquire and record the expert's eye movement signals. Step S13: Generate eye-tracking thermogram: Considering the high-frequency noise generated by eye tremors, a sliding window filter with a window size of 3 is applied to the obtained eye movement data. Eye movements with an angular velocity lower than 30 degrees per second are classified as fixations, while those with an angular velocity greater than or equal to 30 degrees per second are classified as saccades. Assuming the distance between the expert's eyes and the screen is 0.5m, the threshold for angular velocity is converted to a threshold in pixels per second. Fixations with a time interval of less than 7.5ms and a spatial angle change of less than 0.5 degrees are merged, while saccade data with a fixation time of less than 80ms are discarded. For ease of calculation, a truncated Gaussian kernel is used to generate an eye movement heatmap for each image, with the kernel size corresponding to a visual angle of 1.5 degrees.

6. The method of the minimally invasive surgery error prediction and skill evaluation network system based on expert eye-tracking data according to claim 5, characterized in that, The eye tracker is model Tobii Eye Tracker 4C, and the display is a 21-inch LCD screen.

7. The method of the minimally invasive surgery error prediction and skill evaluation network system based on expert eye-tracking data according to claim 4, characterized in that, Step S2 includes the following steps: Step S21, Pre-training the generator network: The network was initialized using a zero-mean Gaussian distribution with a standard deviation of 0.

01. Before the adaptation layer, each convolutional layer underwent batch normalization and dropout with a dropout rate of 0.

2. Hyperparameters... Within an epoch, the hyperparameters are dynamically changed from 2 to 1 at a constant rate to allow the generator to focus on learning and generating attention maps before performing frame classification; It is set to 1 because classification is the network's primary task; hyperparameter Set to 0 to avoid adversarial training initially; pre-train the generator independently for 30 epochs. Step S22, Adversarial Training of Discriminator and Generator: The network is trained with 2000 steps, alternating between the discriminator and generator each step. Single-sided label smoothing is used during discriminator training, while the discriminator weights are not updated during generator training. The network is trained using Adam, with an initial learning rate set to... The batch size is 64, and 5-fold cross-validation is used.

8. The method of the minimally invasive surgery error prediction and skill evaluation network system based on expert eye-tracking data according to claim 4, characterized in that, Step S3 includes the following steps: Step S31, Surgical Error Prediction and Skill Evaluation: The surgical video is preprocessed in the same way as in step S11 and input into the trained generator network. The generator predicts whether an error has occurred in the surgery frame by frame and gives a prediction score. A score threshold and a sliding window length and threshold are set. When the number of video frames that reach the prediction score threshold in the sliding window exceeds the set window threshold, it is determined that an error has occurred in the operator in that video segment.

Images