Intelligent detection method and device for enteroscopy video based on double-system cascade decision

Abstract

The application provides an enteroscope video intelligent detection method and device based on double-system cascade decision, and belongs to the technical field of computer vision. The device comprises: a rapid screening module, which extracts multi-dimensional feature information from an input enteroscope video stream, calculates a diagnosis value score of a current frame according to the multi-dimensional feature information, compares the diagnosis value score with a dynamic screening threshold, and screens out a high-value frame. An accurate detection module is used for target detection on the high-value frame, and outputs a detection result and a confidence. The accurate detection module at least comprises a reference frame dynamic selection mechanism based on screening decision and a time sequence consistency mechanism based on feature fusion constraint. A cooperation optimization module receives the target detection result and the detection confidence, generates a feedback control signal for dynamically correcting the dynamic screening threshold based on multi-dimensional indexes of the target detection result, and realizes dynamic adjustment of detection.

Description

Technical Field

[0001] This invention relates to the field of computer vision technology, and in particular to an intelligent detection method and apparatus for colonoscopy videos based on dual-system cascaded decision-making. Background Technology

[0002] Colonoscopy is the gold standard for colorectal cancer screening and diagnosis. Colonoscopy videos provide detailed information about each screening stage of the patient's intestines, greatly aiding in targeted detection during colonoscopy. However, traditional methods for targeted detection based on manual screening of colonoscopy videos are time-consuming and labor-intensive, highly dependent on physician experience for accuracy, and prone to missed diagnoses due to fatigue. With the development of artificial intelligence technology, designing auxiliary detection devices based on deep learning and computer vision technologies has become a research hotspot.

[0003] However, applying computer vision technology to colonoscopy videos for intestinal target detection faces four inherent technical challenges: 1) Extreme temporal sparsity: Only about 3%-10% of the frames in a single examination contain polyps, resulting in extremely low efficiency in frame-by-frame processing; 2) High target concealment: Polyps are very similar to normal mucosa and are severely affected by reflections, bubbles, camera shake, and other interferences; 3) Temporal continuity constraints: Polyps persist in consecutive frames, containing important motion and morphological information; 4) Extremely high annotation costs and data scarcity: Pixel-level annotation requires a large amount of time from professional doctors, resulting in a scarcity of labeled data.

[0004] Existing technical solutions have many limitations: frame-by-frame detection methods ignore temporal information, resulting in low computational efficiency and limited detection accuracy; temporal detection methods use temporal information inefficiently, have high computational complexity, and are susceptible to noise from irrelevant frames. To improve efficiency, a dual-device approach can be used, with fast and slow devices performing coarse screening and fine inspection. However, the general dual-device approach has not been deeply optimized for the extreme sparsity of colonoscopy videos and cannot adapt to colonoscopy inspection tasks.

[0005] Therefore, there is an urgent need for a new detection method and device that can efficiently utilize time-series information, make intelligent decisions on detection timing, and adapt to the unique challenges of colonoscopy videos, so as to achieve the optimal balance between computational efficiency and diagnostic accuracy. Summary of the Invention

[0006] This invention provides a method and device for intelligent detection of colonoscopy video based on dual-system cascaded decision-making, which solves the defects of existing technologies such as serious waste of computing resources, low detection accuracy of occult lesions, and lack of time-adaptive capability.

[0007] In a first aspect, the present invention provides a colonoscopy video intelligent detection device based on dual-system cascaded decision-making, comprising: The rapid screening module is used to extract multidimensional feature information from the input colonoscopy video stream, calculate the diagnostic value score of the current frame based on the multidimensional feature information, and compare the diagnostic value score with a dynamic screening threshold to screen out high-value frames. The precise detection module is used to perform target detection on the high-value frames and output the detection results and confidence levels; the precise detection module includes at least a reference frame dynamic selection mechanism based on screening decisions and a temporal consistency mechanism based on feature fusion constraints. The collaborative optimization module is used to receive the target detection results and detection confidence, and generate a feedback control signal for dynamically correcting the dynamic screening threshold based on the multi-dimensional indicators of the target detection results.

[0008] According to the colonoscopy video intelligent detection device based on dual-system cascaded decision-making, the rapid screening module includes: A lightweight feature extraction network is used to process the input colonoscopy video stream frame by frame, extracting appearance and motion features from the downsampled image of the current frame and the inter-frame difference features; The diagnostic value calculation unit is used to perform parallel calculations on the appearance features and the motion features based on preset medical prior logic to obtain appearance salience score and motion abnormality score, and to perform weighted aggregation of the appearance salience score and the motion abnormality score to generate a diagnostic value score. The decision output layer is used to compare the diagnostic value score with the dynamic screening threshold and output a high-value frame based on the comparison result.

[0009] According to the colonoscopy video intelligent detection device based on dual-system cascade decision-making, the diagnostic value calculation unit calculates the appearance significance score through a geometric morphology operator; The geometric morphology operator is configured with a positive central region and a negative surrounding region to perform convolution operations on the downsampled image in response to circular or elliptical protrusions in the image that are brighter than the background. The appearance saliency score is obtained by calculating the local maxima in the convolution response map and performing normalization.

[0010] According to the colonoscopy video intelligent detection device based on dual-system cascaded decision-making, the diagnostic value calculation unit calculates the motion abnormality score through inter-frame motion energy analysis, including: Calculate the pixel-level grayscale difference map between the current frame and the previous frame; The variance of local regions in the grayscale difference image is statistically analyzed, and the area ratio of regions with variances higher than a preset noise benchmark is mapped to the motion anomaly score.

[0011] According to the colonoscopy video intelligent detection device based on dual-system cascaded decision-making, the reference frame dynamic selection mechanism based on screening decision-making is configured as follows: Maintain a high-confidence feature buffer pool, which is used to temporarily store target feature vectors whose detection confidence meets the preset standard in historical moments. Only when the detection result of a historical frame is determined to be a high-confidence confirmed state, the corresponding feature vector is included in the pool as valid reference information to construct a high-quality prior feature set for logical verification. Candidate reference frames are determined from the high-value frames; The candidate reference frame is comprehensively scored based on the visual similarity and temporal proximity between the candidate reference frame and the current frame. The visual similarity is used to characterize the consistency between the candidate reference frame and the current frame in the feature space, and the temporal proximity is used to constrain the correlation between the reference frame and the current frame on the time axis. The reference frame for accurate detection is determined based on the comprehensive score. The candidate frame with the highest score is selected as the reference frame. If there is no qualified candidate frame, the current frame is used as the reference frame.

[0012] According to the colonoscopy video intelligent detection device based on dual-system cascaded decision-making, the high-confidence feature buffer pool executes a dynamic update strategy based on feature compactness, which includes: When the detection confidence of the current frame is higher than the first threshold and the similarity with the existing features in the buffer pool is lower than the second threshold, the features of the current frame are written into the buffer pool as new independent feature vectors. If the similarity between the current frame features and existing features in the buffer pool is higher than the second threshold, then the existing features are updated with weight.

[0013] According to the colonoscopy video intelligent detection device based on dual-system cascaded decision-making, the temporal consistency mechanism based on feature fusion constraints is configured as follows: The feature map of the current frame is used as the query vector, and the features of the reference frame selected by the reference frame dynamic selection mechanism based on screening decision are used as the key vector and value vector. The dot product between the query vector and the key vector is calculated as the correlation degree. Combined with normalization processing, a temporal correlation degree weight matrix is ​​obtained. The temporal correlation degree weight matrix reflects the probability that each region in the current frame can find a corresponding confirmed region in the historical frames. The value vector is weighted and aggregated based on the temporal correlation weight matrix to generate contextual features containing historical prior information. The contextual features are residually fused with the original features of the current frame to generate temporal enhancement features; The current frame is compensated and reconstructed by utilizing the high-quality features of historical moments to achieve logical verification and feature enhancement from the historical confirmed status to the current suspected status.

[0014] According to the colonoscopy video intelligent detection device based on dual-system cascaded decision-making, the collaborative optimization module adopts a dual-track complementary optimization architecture based on rule logic policy network and reinforcement learning policy network. The rule-based logic strategy network is configured to generate an upward control signal for the dynamic screening threshold when the false positive rate within a preset time window exceeds a first threshold. When a disruption that violates the temporal continuity is detected in the target detection results in consecutive frames, a control signal for lowering the dynamic screening threshold is generated; The reinforcement learning policy network is configured to: construct a system state vector containing the current screening threshold, average detection confidence, false positive count, and processing frame rate, and input it into the reinforcement learning policy network to optimize the network parameters with the goal of maximizing the comprehensive reward function, wherein the comprehensive reward function is a weighted sum of adenoma detection rate gain, computational resource consumption penalty, and false positive rate penalty.

[0015] According to the aforementioned intelligent detection device for colonoscopy video based on dual-system cascaded decision-making, the reinforcement learning policy network adopts a hybrid training mode combining offline pre-training and online fine-tuning. It uses the policy gradient method to calculate the loss function based on the advantage function and injects prior knowledge into the policy to enable the reinforcement learning policy network to continuously optimize based on real-time feedback during actual operation.

[0016] Secondly, the present invention also provides an intelligent detection method for colonoscopy videos based on dual-system cascaded decision-making, employing any of the colonoscopy video target detection devices described above, comprising: Multidimensional feature information is extracted from the input colonoscopy video stream, and the diagnostic value score of the current frame is calculated based on the multidimensional feature information. The diagnostic value score is compared with a dynamic screening threshold to select high-value frames. The reference frame dynamic selection mechanism based on screening decision and the temporal consistency mechanism based on feature fusion constraint are used to perform target detection on the high-value frames and output the detection results and confidence scores. The system receives the target detection result and the detection confidence level, and generates a feedback control signal for dynamically correcting the dynamic screening threshold based on the multi-dimensional indicators of the target detection result.

[0017] The intelligent detection method and device for colonoscopy video based on dual-system cascaded decision-making provided by this invention has the following advantages compared with the prior art: The rapid screening module extracts multi-dimensional feature information from the input colonoscopy video stream and calculates the diagnostic value score of the current frame based on this information. The diagnostic value score is then compared with a dynamic screening threshold to select high-value frames. The precise detection module performs target detection on these high-value frames and outputs the detection results and confidence levels. This precise detection module includes at least a dynamic reference frame selection mechanism based on screening decisions and a temporal consistency mechanism based on feature fusion constraints. The collaborative optimization module receives the target detection results and detection confidence levels, and generates feedback control signals based on the multi-dimensional indicators of the target detection results to dynamically adjust the dynamic screening threshold, thus achieving dynamic adjustment of the device. This invention addresses the technical problems of wasteful computational resources, low detection accuracy, and poor generalization in colonoscopy video target detection caused by extreme temporal sparsity, high target concealment, and lack of adaptability. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0019] Figure 1 This is a schematic diagram of the framework of an optional intelligent detection device for colonoscopy video based on dual-system cascaded decision-making, provided in an embodiment of the present invention. Figure 2 This is a schematic diagram of a target detection process using an intelligent video detection device for colonoscopy that is optionally based on dual-system cascaded decision-making, according to an embodiment of the present invention. Figure 3 This is a schematic diagram comparing detection results and corresponding feature maps under different optional experimental settings provided by an embodiment of the present invention. Detailed Implementation

[0020] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0021] Figure 1This is a schematic diagram of the framework of an optional intelligent detection device for colonoscopy video based on dual-system cascaded decision-making, as provided in an embodiment of the present invention. Figure 1 As shown, this device includes: The rapid screening module is used to extract multidimensional feature information from the input colonoscopy video stream, calculate the diagnostic value score of the current frame based on the multidimensional feature information, and compare the diagnostic value score with a dynamic screening threshold to screen out high-value frames. The precise detection module is used to perform target detection on the high-value frames and output the detection results and confidence levels; the precise detection module includes at least a reference frame dynamic selection mechanism based on screening decisions and a temporal consistency mechanism based on feature fusion constraints. The collaborative optimization module is used to receive the target detection results and detection confidence, and generate a feedback control signal for dynamically correcting the dynamic screening threshold based on the multi-dimensional indicators of the target detection results.

[0022] In this embodiment, Figure 2 This is a schematic diagram of a target detection process using an optional dual-system cascaded decision-making intelligent detection device for colonoscopy video, as provided in an embodiment of the present invention. Figure 2 As shown, the rapid screening module is constructed as the first level of a cascaded architecture, used to perform frame-by-frame diagnostic value assessment on the images of the input colonoscopy video sequence and to divide the video stream into background frames and high-value frames.

[0023] Existing deep learning detection methods typically perform deep processing on each frame with equal computational power, resulting in a huge waste of computational resources in colonoscopy videos that can last for tens of minutes. To achieve real-time and efficient filtering, this module includes a "diagnostic value calculation unit" that uses lightweight computing to simulate the doctor's visual priors—namely, focusing on "geometric protrusions" and "abnormal movements."

[0024] Optionally, the rapid screening module includes: A lightweight feature extraction network is used to process the input colonoscopy video stream frame by frame, extracting appearance and motion features from the downsampled image of the current frame and the inter-frame difference features; The diagnostic value calculation unit is used to perform parallel calculations on the appearance features and the motion features based on preset medical prior logic to obtain appearance salience score and motion abnormality score, and to perform weighted aggregation of the appearance salience score and the motion abnormality score to generate a diagnostic value score. The decision output layer is used to compare the diagnostic value score with the dynamic screening threshold and output a high-value frame based on the comparison result.

[0025] The diagnostic value calculation unit calculates the appearance significance score using a geometric morphology operator. The geometric morphology operator is configured with a positive central region and a negative surrounding region to perform convolution operations on the downsampled image in response to circular or elliptical protrusions in the image that are brighter than the background. The appearance saliency score is obtained by calculating the local maxima in the convolution response map and performing normalization.

[0026] Furthermore, the diagnostic value calculation unit calculates the motion anomaly score through inter-frame motion energy analysis, including: Calculate the pixel-level grayscale difference map between the current frame and the previous frame; The variance of local regions in the grayscale difference image is statistically analyzed, and the area ratio of regions with variances higher than a preset noise benchmark is mapped to the motion anomaly score.

[0027] Specifically, the input video frames are first downsampled (e.g., scaled down to...). (resolution), and then proceeds to two scoring branches in parallel: 1. Appearance salience score ( ): This embodiment utilizes geometric morphology operators to identify local convex structures in images. It employs a "center-periphery" difference operator to convolve the downsampled image. This operator simulates the biological visual receptive field, configured as a central region with positive values ​​and a peripheral surrounding region with negative values.

[0028] Let the input image be The operator kernel is Convolutional response map The calculation is as follows:

[0029] This operator specifically responds to circular or elliptical regions with brightness higher than the background. The module calculates and normalizes the local maxima in the response map to obtain an appearance score.

[0030] in , These are the mean and standard deviation of the response plot, respectively.

[0031] 2. Motion abnormality score ( ): Inter-frame motion energy analysis was used to capture discontinuous parallax changes. Because polyps are protrusions, they produce more dramatic parallax edges than smooth intestinal walls when the endoscope moves.

[0032] First, we can calculate the current frame. With the preceding frame absolute difference plot :

[0033] The difference plot is then calculated in a local window. variance within The statistical variance is higher than the preset noise baseline. The regional proportion is mapped to the sports score:

[0034] in This is an indicator function.

[0035] Decision-making logic: The above scores are combined and weighted to generate a final diagnostic value score. .

[0036]

[0037] Only when , If the current dynamic threshold is reached, the subsequent precise detection module is activated; otherwise, the frame is skipped.

[0038] The precise detection module is constructed as the second level of the cascaded architecture. It performs fine-grained processing only on the selected high-value frames, and is used to perform target detection on the high-value frames, outputting the detection results and confidence levels.

[0039] The accurate detection module includes at least a reference frame dynamic selection mechanism based on screening decisions and a temporal consistency mechanism based on feature fusion constraints.

[0040] Optionally, the reference frame dynamic selection mechanism based on screening decisions is configured as follows: Maintain a high-confidence feature buffer pool, which is used to temporarily store target feature vectors whose detection confidence meets the preset standard in historical moments. Only when the detection result of a historical frame is determined to be a high-confidence confirmed state, the corresponding feature vector is included in the pool as valid reference information to construct a high-quality prior feature set for logical verification. Candidate reference frames are determined from the high-value frames; The candidate reference frame is comprehensively scored based on the visual similarity and temporal proximity between the candidate reference frame and the current frame. The visual similarity is used to characterize the consistency between the candidate reference frame and the current frame in the feature space, and the temporal proximity is used to constrain the correlation between the reference frame and the current frame on the time axis. The reference frame for accurate detection is determined based on the comprehensive score. The candidate frame with the highest score is selected as the reference frame. If there is no qualified candidate frame, the current frame is used as the reference frame.

[0041] Furthermore, the high-confidence feature buffer pool implements a dynamic update strategy based on feature compactness, the dynamic update strategy based on feature compactness including: When the detection confidence of the current frame is higher than the first threshold and the similarity with the existing features in the buffer pool is lower than the second threshold, the features of the current frame are written into the buffer pool as new independent feature vectors. If the similarity between the current frame features and existing features in the buffer pool is higher than the second threshold, then the existing features are updated with weight.

[0042] Optionally, the temporal consistency mechanism based on feature fusion constraints is configured as follows: The feature map of the current frame is used as the query vector, and the features of the reference frame selected by the reference frame dynamic selection mechanism based on screening decision are used as the key vector and value vector. The dot product between the query vector and the key vector is calculated as the correlation degree. Combined with normalization processing, a temporal correlation degree weight matrix is ​​obtained. The temporal correlation degree weight matrix reflects the probability that each region in the current frame can find a corresponding confirmed region in the historical frames. The value vector is weighted and aggregated based on the temporal correlation weight matrix to generate contextual features containing historical prior information. The contextual features are residually fused with the original features of the current frame to generate temporal enhancement features; The current frame is compensated and reconstructed by utilizing the high-quality features of historical moments to achieve logical verification and feature enhancement from the historical confirmed status to the current suspected status.

[0043] Specifically, the implementation process of the precision detection module includes the following three key steps: (1) Reference frame dynamic selection mechanism In order to select the most relevant historical features for the current frame from the buffer pool, the module calculates candidate frames. With the current frame Overall rating The scoring function design takes into account both visual similarity and time decay:

[0044] in: Cosine similarity between feature vectors; The number of frames in the time interval. The attenuation constant is taken in this embodiment. ; As the weighting adjustment parameter, this embodiment takes... .

[0045] Module priority selection The highest frame is used as the best reference frame.

[0046] (2) Enhanced timing consistency features (logic verification) This is the core mechanism of this invention. The current frame feature map is used as the query vector. The selected reference frame and buffer pool features are used as key vectors. Sum value vector .

[0047] First, calculate and Semantic Relationship Weight Matrix :

[0048] in The feature dimension is denoted by . This matrix reflects the probability that each region in the current frame will find a corresponding "confirmed" region in historical frames.

[0049] Subsequently, based on the aforementioned temporal correlation weight matrix value vector Perform weighted aggregation to generate contextual features that include historical prior information. ,

[0050] It then performs residual fusion with the current frame features to generate enhanced features. :

[0051] in This is a learnable adjustment factor, initially set to 0. Essentially, this process utilizes historical high-quality features to fill in feature gaps in the current frame caused by blurring or occlusion.

[0052] (3) Buffer pool “feature compactness” update strategy To avoid buffer pool redundancy, this embodiment employs a strict admission mechanism. The current frame features are only considered if the following two conditions are met. Only then can it be written to the buffer pool as a separate entry: 1. Confidence Gating: Detection Confidence To ensure that the target is confirmed; 2. Dissimilarity Gating: Maximum similarity with any feature in the pool. To ensure the provision of new information.

[0053] If the similarity is greater than 0.9, then the most similar existing features are updated with a weighted average:

[0054] The collaborative optimization module is used to receive the target detection results and detection confidence, and generate a feedback control signal for dynamically correcting the dynamic screening threshold based on the multi-dimensional indicators of the target detection results.

[0055] The collaborative optimization module is used to build a closed-loop feedback chain. It adopts a dual-track architecture that combines rule-based logic and reinforcement learning to dynamically adjust the dynamic screening threshold of the rapid screening module. .

[0056] Optionally, the collaborative optimization module adopts a dual-track complementary optimization architecture based on rule logic policy network and reinforcement learning policy network; The rule-based logic strategy network is configured to generate an upward control signal for the dynamic screening threshold when the false positive rate within a preset time window exceeds a first threshold. When a disruption that violates the temporal continuity is detected in the target detection results in consecutive frames, a control signal for lowering the dynamic screening threshold is generated; The reinforcement learning policy network is configured to: construct a device state vector containing the current screening threshold, average detection confidence, false positive count, and processing frame rate, and input it into the reinforcement learning policy network to optimize the network parameters with the goal of maximizing the comprehensive reward function, wherein the comprehensive reward function is a weighted sum of adenoma detection rate gain, computational resource consumption penalty, and false positive rate penalty.

[0057] Optionally, the reinforcement learning policy network adopts a hybrid training mode that combines offline pre-training with online fine-tuning. It uses the policy gradient method to calculate the loss function based on the advantage function and injects prior knowledge into the policy to enable the reinforcement learning policy network to continuously optimize based on real-time feedback during actual operation.

[0058] In this embodiment, the policy network, for example, first sets hard constraints based on prior knowledge from clinical experts: Noise suppression: If recently A false positive rate (FP Rate) exceeding 15% indicates overly lenient screening, necessitating an increase in the mandatory threshold. In this invention, N is set to 30 frames.

[0059] Recall and rescue: If an "anomaly violating the temporal continuity" is detected, i.e., the target is in Frame exists. Frame disappears. If another frame appears and its motion trajectory is discontinuous, it is determined to be a potential missed detection, and the threshold is immediately lowered. .

[0060] Reinforcement learning strategy: Construct a policy network to achieve end-to-end threshold decision-making.

[0061] state space : .

[0062] reward function Designed with the goal of maximizing clinical value:

[0063] in: Gain in adenoma detection rate (core clinical indicator). Computational resource consumption (percentage of frames processed); False positive penalty items; Training strategies include offline pre-training, which uses historical labeled data to initialize the strategy, combined with online fine-tuning, which uses a strategy gradient update mode during actual operation to guide the network to quickly converge to the optimal strategy that adapts to the current intestinal environment.

[0064] Furthermore, embodiments of the present invention also provide experimental results and analysis of implementing a colonoscopy video intelligent detection device based on dual-system cascaded decision-making.

[0065] To verify the effectiveness of each module of the present invention, an ablation experiment was conducted on a mixed colonoscopy video dataset containing 49 colonoscopy video segments extracted from 20 colonoscopy videos, totaling approximately 50,000 frames. Figure 3 This is a schematic diagram comparing detection results and corresponding feature maps under optional different experimental settings provided by an embodiment of the present invention, such as... Figure 3 As shown, the experiment compared the performance of the baseline method with that of the present invention under different configurations.

[0066] Table 1 Comparison of ablation test results for each module

[0067] Exp1 (Baseline): Employs a standard single-frame object detection network (Yona) to perform frame-by-frame detection on video sequences. It does not include fast screening, feature buffering, or collaborative optimization modules.

[0068] Exp 2 (System 1 Only): Adds a fast screening module to the baseline. It filters image frames using medical prior logic, namely appearance and motion information. Subsequent detection networks do not perform feature enhancement; the filtering uses a fixed threshold initialized to 0.5 and does not include collaborative optimization feedback.

[0069] Experiment 3 (System 1 + System 2): Based on Experiment 2, the detection module is updated to the precise detection module of this invention. Specifically, a high-confidence feature buffer pool is constructed using the screening results from the rapid screening module. A dynamic reference frame selection mechanism based on screening decisions and a temporal consistency mechanism based on feature fusion constraints are used to enhance the features to be detected. However, the rapid screening module uses a fixed threshold, initialized to 0.5, and does not include collaborative optimization feedback.

[0070] Experiment 4 (System 1 + System 3): Building upon Experiment 2, a collaborative optimization module is added to achieve dynamic adjustment of the screening threshold. Note: In this group of experiments, the detection module only performs basic detection using the original Yona method; the reference frame selection and timing enhancement of the precise detection module in this invention are not enabled. This group aims to verify the independent contribution of the feedback loop in simply adjusting sensitivity.

[0071] Exp 5 (System 1 + System 2 + System 3): The complete solution proposed in this invention. It includes a feature enhancement scheme for the medical preliminary rapid screening and precision testing modules, as well as a dual-track dynamic feedback closed loop for the collaborative optimization module.

[0072] Results analysis: 1. Comparing Experiments 1 and 2, after introducing the rapid screening module, the FPS jumped from 22 to 58, and the computational redundancy rate decreased significantly. Although the mAP dropped slightly by 0.4%, this proves that the vast majority of the filtered frames were indeed invalid backgrounds, and the medical prior logic has an extremely high filtering signal-to-noise ratio.

[0073] 2. Comparison of Experiments 2 and 3: Under a fixed threshold, the mAP significantly improved by 5.3% after introducing a buffer pool and a temporal enhancement mechanism. This indicates that using historical high-confidence features for logical verification can effectively solve the missed detections caused by motion blur and occlusion, and significantly improve the feature representation capability of the detector.

[0074] 3. Comparison of Experiments 2 and 4: Experiment 4 serves as the key control group. Without feature enhancement, by dynamically adjusting the threshold solely through the collaborative optimization module, the ADR improved from 83.8% to 88.5%. This demonstrates that the dual-track optimization architecture can keenly detect anomalies and automatically lower the threshold to recover missed targets, or raise the threshold when noise is high, proving the effectiveness of the feedback loop at the strategy level.

[0075] 4. Analysis of Experiment 5: The complete scheme combines "feature layer enhancement" and "decision layer adaptation," achieving the highest mAP of 90.1% and ADR of 92.5%, with the lowest computational redundancy. This demonstrates that the feature enhancement of the fine-detection module and the dual-track threshold tuning of the collaborative optimization module in the dual "coarse screening-fine detection" two-level structure form a virtuous cycle: stronger features provide more accurate feedback signals, while more accurate thresholds provide higher-quality input frames for feature extraction.

[0076] This invention also provides a method for intelligent detection of colonoscopy video using a dual-device cascaded decision-making system, employing any of the dual-device cascaded decision-making intelligent detection devices for colonoscopy video as described above, the method comprising: Multidimensional feature information is extracted from the input colonoscopy video stream, and the diagnostic value score of the current frame is calculated based on the multidimensional feature information. The diagnostic value score is compared with a dynamic screening threshold to select high-value frames. The reference frame dynamic selection mechanism based on screening decision and the temporal consistency mechanism based on feature fusion constraint are used to perform target detection on the high-value frames and output the detection results and confidence scores. The system receives the target detection result and the detection confidence level, and generates a feedback control signal for dynamically correcting the dynamic screening threshold based on the multi-dimensional indicators of the target detection result.

[0077] The present invention provides a colonoscopy video intelligent detection method based on dual-system cascade decision-making, which can also be applied to the colonoscopy video intelligent detection device based on dual-system cascade decision-making described in any of the above embodiments, and will not be described again here.

[0078] The intelligent detection method and device for colonoscopy video based on dual-system cascaded decision-making provided by this invention has the following advantages compared with the prior art: (1) The present invention designs a complete colonoscopy video detection process, providing a complete process from rapid screening and accurate detection to collaborative optimization. Through a rapid screening mechanism based on medical prior logic, irrelevant frames in the video are efficiently removed, so that the computationally intensive accurate detection module only acts on key frames, forming a dual-system cascaded decision architecture, which significantly improves detection efficiency and accuracy.

[0079] (2) This invention fully utilizes short-term temporal context information and feature fusion constraints through a reference frame dynamic selection mechanism based on screening decision and a temporal consistency mechanism based on feature fusion constraints. It uses historical high-quality features to perform logical verification and reconstruction on the current frame, thereby achieving stable detection of high-value frames and improving the ability to adapt to temporal continuity and sparse labeled data.

[0080] (3) This invention fully considers the motion occlusion and high concealment problem of colonoscopy video. Unlike the traditional optical flow method or simple frame difference method, it introduces geometric morphology operator and motion energy analysis, which effectively enhances the discriminative power of features, avoids interference from irrelevant frame noise, and significantly improves the detection accuracy of concealed intestinal targets and small targets.

[0081] (4) This invention constructs a closed-loop optimization mechanism based on multi-dimensional indicators, combining the safety baseline of rule strategy with the adaptive capability of reinforcement learning to realize the dynamic adjustment of the device. It can dynamically adjust the screening sensitivity according to the difficulty of the video content (such as the dense area of ​​the target and the normal intestinal segment) and the current load of the device, which solves the problem of unstable performance of the fixed threshold method when facing video data collected from different patients and different devices, and has extremely high clinical generalization potential.

[0082] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0083] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0084] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. An intelligent detection device for colonoscopy video based on dual-system cascade decision, characterized in that, include: The rapid screening module is used to extract multidimensional feature information from the input colonoscopy video stream, calculate the diagnostic value score of the current frame based on the multidimensional feature information, and compare the diagnostic value score with a dynamic screening threshold to screen out high-value frames. The precision detection module is used to perform target detection on the high-value frames and output the detection results and confidence levels. The precise detection module includes at least a reference frame dynamic selection mechanism based on screening decisions and a temporal consistency mechanism based on feature fusion constraints. The collaborative optimization module is used to receive the target detection results and detection confidence, and generate a feedback control signal for dynamically correcting the dynamic screening threshold based on the multi-dimensional indicators of the target detection results.

2. The intelligent video detection device for colonoscopy based on dual-system cascaded decision-making according to claim 1, characterized in that, The rapid screening module includes: A lightweight feature extraction network is used to process the input colonoscopy video stream frame by frame, extracting appearance and motion features from the downsampled image of the current frame and the inter-frame difference features; The diagnostic value calculation unit is used to perform parallel calculations on the appearance features and the motion features based on preset medical prior logic to obtain appearance salience score and motion abnormality score, and to perform weighted aggregation of the appearance salience score and the motion abnormality score to generate a diagnostic value score. The decision output layer is used to compare the diagnostic value score with the dynamic screening threshold and output a high-value frame based on the comparison result.

3. The intelligent video detection device for colonoscopy based on dual-system cascaded decision-making according to claim 2, characterized in that, The diagnostic value calculation unit calculates the appearance significance score using a geometric morphology operator; The geometric morphology operator is configured with a positive central region and a negative surrounding region to perform convolution operations on the downsampled image in response to circular or elliptical protrusions in the image that are brighter than the background. The appearance saliency score is obtained by calculating the local maxima in the convolution response map and performing normalization.

4. The intelligent video detection device for colonoscopy based on dual-system cascaded decision-making according to claim 2, characterized in that, The diagnostic value calculation unit calculates the motion anomaly score through inter-frame motion energy analysis, including: Calculate the pixel-level grayscale difference map between the current frame and the previous frame; The variance of local regions in the grayscale difference image is statistically analyzed, and the area ratio of regions with variances higher than a preset noise benchmark is mapped to the motion anomaly score.

5. The intelligent video detection device for colonoscopy based on dual-system cascaded decision-making according to claim 1, characterized in that, The reference frame dynamic selection mechanism based on screening decisions is configured as follows: Maintain a high-confidence feature buffer pool, which is used to temporarily store target feature vectors whose detection confidence meets the preset standard in historical moments. Only when the detection result of a historical frame is determined to be a high-confidence confirmed state, the corresponding feature vector is included in the pool as valid reference information to construct a high-quality prior feature set for logical verification. Candidate reference frames are determined from the high-value frames; The candidate reference frame is comprehensively scored based on the visual similarity and temporal proximity between the candidate reference frame and the current frame. The visual similarity is used to characterize the consistency between the candidate reference frame and the current frame in the feature space, and the temporal proximity is used to constrain the correlation between the reference frame and the current frame on the time axis. The reference frame for accurate detection is determined based on the comprehensive score. The candidate frame with the highest score is selected as the reference frame. If there is no qualified candidate frame, the current frame is used as the reference frame.

6. The intelligent video detection device for colonoscopy based on dual-system cascaded decision-making according to claim 5, characterized in that, The high-confidence feature buffer pool implements a dynamic update strategy based on feature compactness, which includes: When the detection confidence of the current frame is higher than the first threshold and the similarity with the existing features in the buffer pool is lower than the second threshold, the features of the current frame are written into the buffer pool as new independent feature vectors. If the similarity between the current frame features and existing features in the buffer pool is higher than the second threshold, then the existing features are updated with weight.

7. The intelligent video detection device for colonoscopy based on dual-system cascaded decision-making according to claim 1 or 5, characterized in that, The time-series consistency mechanism based on feature fusion constraints is configured as follows: The feature map of the current frame is used as the query vector, and the features of the reference frame selected by the reference frame dynamic selection mechanism based on screening decision are used as the key vector and value vector. The dot product between the query vector and the key vector is calculated as the correlation degree. Combined with normalization processing, a temporal correlation degree weight matrix is ​​obtained. The temporal correlation degree weight matrix reflects the probability that each region in the current frame can find a corresponding confirmed region in the historical frames. The value vector is weighted and aggregated based on the temporal correlation weight matrix to generate contextual features containing historical prior information. The contextual features are residually fused with the original features of the current frame to generate temporal enhancement features; The current frame is compensated and reconstructed by utilizing the high-quality features of historical moments to achieve logical verification and feature enhancement from the historical confirmed status to the current suspected status.

8. The intelligent video detection device for colonoscopy based on dual-system cascaded decision-making according to claim 1, characterized in that, The collaborative optimization module adopts a dual-track complementary optimization architecture based on rule logic policy network and reinforcement learning policy network; The rule-based logic strategy network is configured to generate an upward control signal for the dynamic screening threshold when the false positive rate within a preset time window exceeds a first threshold. When a disruption that violates the temporal continuity is detected in the target detection results in consecutive frames, a control signal for lowering the dynamic screening threshold is generated; The reinforcement learning policy network is configured to: construct a system state vector containing the current screening threshold, average detection confidence, false positive count, and processing frame rate, and input it into the reinforcement learning policy network to optimize the network parameters with the goal of maximizing the comprehensive reward function, wherein the comprehensive reward function is a weighted sum of adenoma detection rate gain, computational resource consumption penalty, and false positive rate penalty.

9. The intelligent video detection device for colonoscopy based on dual-system cascaded decision-making according to claim 8, characterized in that, The reinforcement learning policy network adopts a hybrid training mode that combines offline pre-training with online fine-tuning. It uses the policy gradient method to calculate the loss function based on the advantage function and injects prior knowledge into the policy to enable the reinforcement learning policy network to continuously optimize based on real-time feedback during actual operation.

10. A method for intelligent detection of colonoscopy videos based on dual-system cascaded decision-making, applied to the intelligent detection device for colonoscopy videos based on dual-system cascaded decision-making as described in any one of claims 1 to 9, characterized in that, The method includes: Multidimensional feature information is extracted from the input colonoscopy video stream, and the diagnostic value score of the current frame is calculated based on the multidimensional feature information. The diagnostic value score is compared with a dynamic screening threshold to select high-value frames. The reference frame dynamic selection mechanism based on screening decision and the temporal consistency mechanism based on feature fusion constraint are used to perform target detection on the high-value frames and output the detection results and confidence scores. The system receives the target detection result and the detection confidence level, and generates a feedback control signal for dynamically correcting the dynamic screening threshold based on the multi-dimensional indicators of the target detection result.

Images