Endoscopic image robust cavity detection method, system, terminal and medium

Abstract

The present invention discloses a robust lumen detection method, system, terminal, and medium for endoscopic image detection. The method comprises: dividing a given number of endoscopic images into clean images and artifact images, and then dividing the clean images and the artifact images into a training set and a test set, respectively; establishing a neural network model for robust lumen detection; training the neural network model using the training set; and testing the neural network model using the test set. Through the present invention, lumens in endoscopic images can be detected using clean endoscopic images and a small number of artifact images; the robustness of the endoscopic image detection method can be improved, and it can be widely used in endoscopic surgery.

Description

Technical Field

[0001] The present invention relates to the field of computer vision technology, and in particular to a robust cavity detection method, system, terminal and medium for endoscopic images. Background Art

[0002] Object detection in images is a hot topic in the fields of computer vision and image processing. Given an image, the algorithm aims to output bounding boxes for one or more classes of objects. Currently, object detection algorithms can be categorized by their framework into those based on traditional image processing and those based on deep learning and neural networks. Deep learning-based object detection methods have been widely used in recent years and typically involve model building, training, and testing. These methods can be further divided into one-stage and two-stage detection network approaches.

[0003] The target detection method based on traditional image processing technology mainly extracts several candidate boxes from the image, extracts features from the candidate boxes, and then classifies them through a classifier. Due to the complexity of the calculation and poor generalization of this method, it is currently only used in limited scenarios (such as when the target is simple and the image resolution is low). In recent years, deep learning algorithms have gradually been applied to target detection tasks. Target detection algorithms based on deep learning can be divided into two-stage methods and one-stage methods. The two-stage method mainly includes R-CNN, Fast R-CNN, Faster R-CNN and other algorithms. The candidate regions of the object are extracted in the first stage, and the candidate regions are classified in the second stage to obtain the image detection results. The representative method of the one-stage algorithm is YOLO, which directly predicts the results of each detection box through the features of the entire image, and has a higher speed improvement than the two-stage method. Among them, YOLOv3 [1] It is a widely used and effective target detection model.

[0004] However, none of the above methods consider the problem of cross-domain features and uneven distribution of training samples, which manifests itself as artifacts in endoscopic image detection. The appearance of artifacts significantly interferes with target detection methods, reducing the effectiveness of target detection algorithms. Summary of the Invention

[0005] In response to the problems existing in the above-mentioned prior art, the present invention proposes a robust lumen detection method, system, terminal and medium for endoscopic images, which can use clean endoscopic images and a small number of artifact-containing images to detect the lumens in endoscopic images; it can improve the robustness of the endoscopic image detection method and can be widely used in endoscopic surgery.

[0006] In order to solve the above technical problems, the present invention is achieved through the following technical solutions:

[0007] According to a first aspect of the present invention, a robust lumen detection method for an endoscopic image is provided, comprising:

[0008] S11: Construct dataset;

[0009] For a given number of endoscopic images, the images are divided into clean images and artifact images, and the clean images and the artifact images are divided into a training set and a test set respectively;

[0010] S12: Establish a neural network model for robust cavity detection;

[0011] S13: training model;

[0012] Using the training set obtained in S11 to train the neural network model established in S12;

[0013] S14: test model;

[0014] The test set obtained in S11 is used to test the neural network model trained in S13.

[0015] Preferably, the neural network model established in S12 is a target detection neural network model with feature extraction and fusion.

[0016] Preferably, the S12 specifically includes:

[0017] S121: performing image feature extraction on the clean image to obtain image features of the clean image; and performing image feature extraction and artifact feature extraction on the artifact image to obtain image features and artifact features of the artifact image;

[0018] S122: fusing the artifact feature with the image feature of the clean image and the image feature of the artifact image respectively to obtain two fused image features;

[0019] S123: Inputting the two fused image features into a target detection network for detection to obtain a detection result;

[0020] S124: Construct a loss function using the preset labeled true value and the detection result, train the neural network model using the training set of S11, and optimize the neural network model parameters.

[0021] Preferably, the S122 specifically includes: adding the artifact feature to the image feature of the clean image and the image feature of the artifact image respectively to obtain two fused image features;

[0022] The S123 specifically includes: inputting the image features of the clean image, the image features of the artifact image, and the fused image features into a detection network respectively to obtain a detection result.

[0023] Preferably, the loss function minimization problem in S124 is defined as:

[0024]

[0025] In the above formula, θ represents the model parameters, n is the number of images in each training, that is, the number of images contained in the image block input for each training of the model; w1,w2,w3,w4,w d is the weight of each loss function; L1 is the loss of the detection result of the image feature of the clean image and the true value, L2 is the loss of the detection result of the fusion of the image feature of the clean image and the artifact feature and the true value, L3 is the loss of the detection result of the fusion of the image feature of the artifact image and the artifact feature and the true value, L4 is the loss of the detection result of the image feature of the artifact image, L d is the artifact feature extraction network penalty term.

[0026] Preferably, the loss functions of L1, L2, L3, and L4 are all YOLO-type loss functions L YOLO , which has the following form:

[0027] L YOLO =L x +L y +L w +L h +L conf_obj +L conf_noobj +L cls

[0028] Among them L x ,L y ,L w ,L h They are the horizontal coordinate, vertical coordinate, width and height error losses of the target detection box, L conf_obj and L conf_noobj are the confidence losses of objects and non-objects respectively, L cls is the classification confidence loss;

[0029] The horizontal axis loss is:

[0030]

[0031] in Indicates that there is an object at the position of the predicted box, N×N is the size of the feature map to be detected, x, x′ are the horizontal coordinates of the real box and the predicted box respectively;

[0032] The ordinate loss is:

[0033]

[0034] Where y, y′ are the vertical coordinates of the real box and the predicted box respectively;

[0035] The width loss is:

[0036]

[0037] Where w, w′ are the widths of the real box and the predicted box respectively;

[0038] The height loss is:

[0039]

[0040] Where h, h′ are the heights of the real box and the predicted box respectively;

[0041] The object confidence loss is:

[0042]

[0043] where c ij Represents the predicted probability that there is an object at a certain position in the target box;

[0044] The non-object confidence loss is:

[0045]

[0046] in Indicates that there is no object at the location of the prediction box, c ij Represents the predicted probability that there is an object at a certain position in the target box;

[0047] The classification confidence loss is:

[0048]

[0049] where p ij (c) represents the probability that there is an object of category c at the position of the prediction box, p′ ij (c) represents the predicted probability that there is an object of category c at the position of the prediction box.

[0050] Preferably, the L d for:

[0051]

[0052] Where m is the artifact feature map, which is the output of the artifact feature extraction network.

[0053] According to a second aspect of the present invention, there is also provided an endoscopic image robust lumen detection system, which comprises: a data set construction unit, a model building unit, a model training unit and a model testing unit; wherein,

[0054] The data set construction unit is used to divide a given number of endoscopic images into clean images and artifact images, and then divide the clean images and the artifact images into a training set and a test set respectively;

[0055] The model building unit is used to build a neural network model for robust cavity detection;

[0056] The model training unit is used to train the neural network model established by the model building unit using the training set obtained by the data building unit;

[0057] The model testing unit is used to test the neural network model trained by the model training unit using the test set obtained by the data set construction unit.

[0058] According to the third aspect of the present invention, a terminal is also provided, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor can be used to execute the above-mentioned robust cavity detection method for endoscopic images when executing the program.

[0059] According to a fourth aspect of the present invention, there is further provided a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, can be used to execute the method for robust lumen detection in endoscopic images.

[0060] The present invention provides a robust lumen detection method, system, terminal, and medium for endoscopic images. By utilizing clean endoscopic images and a small number of artifact-bearing images, the method detects lumens in endoscopic images. This invention improves the robustness of endoscopic image detection methods and is widely applicable in endoscopic surgery. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0062] Figure 1 This is a flow chart of a robust lumen detection method for endoscopic images according to an embodiment of the present invention;

[0063] Figure 2 This is a schematic diagram of an endoscope image cavity detection network according to an embodiment of the present invention;

[0064] Figure 3 A schematic diagram of an image feature extraction module according to a preferred embodiment of the present invention;

[0065] Figure 4 A schematic diagram of a residual module resn according to a preferred embodiment of the present invention;

[0066] Figure 5 Schematic diagram of an artifact feature extraction module according to a preferred embodiment of the present invention;

[0067] Figure 6 A schematic diagram of a target detection module according to a preferred embodiment of the present invention;

[0068] Figure 7 This is a comparison chart of the detection results of the method of the present invention and the existing method according to a preferred embodiment of the present invention. DETAILED DESCRIPTION

[0069] The following is a detailed description of an embodiment of the present invention. This embodiment is implemented based on the technical solution of the present invention, and provides a detailed implementation method and specific operation process. However, the protection scope of the present invention is not limited to the following embodiment.

[0070] like Figure 1 FIG2 is a flowchart of a robust lumen detection method for endoscopic images according to an embodiment of the present invention.

[0071] Please refer to Figure 1 The robust cavity detection method of endoscope images of this embodiment includes:

[0072] S11: Construct dataset;

[0073] For a given number of endoscopic images, the data set is divided into clean images and artifact images, and then the clean images and artifact images are divided into training sets and test sets respectively;

[0074] S12: Establish a neural network model for robust cavity detection;

[0075] S13: training model;

[0076] Using the training set obtained in S11 to train the neural network model established in S12;

[0077] S14: test model;

[0078] The neural network model trained in S13 is tested using the test set obtained in S11. The endoscope image to be tested is input into the tested neural network model to obtain the cavity detection result.

[0079] In a preferred embodiment, the division in S11 also includes a validation set. Specifically, the endoscopic image data is divided into clean images and artifact images, where artifact images are defined as endoscopic images with visible light spots, dispersion stripes, and jitter blur. The clean images and artifact images are then divided into a training set, a validation set, and a test set. The training set is used for model training in S13, and the validation set is used for model selection and hyperparameter adjustment in S13. That is, different groups of model hyperparameters are set, and evaluation indicators are obtained using the validation set. The set of hyperparameters with the best indicators is selected as the optimal model for testing. The test set is used for model testing in S14.

[0080] In a preferred embodiment, the neural network model established in S12 is a target detection neural network model with feature extraction and fusion.

[0081] In one embodiment, the network model diagram in S12 is as follows: Figure 2 As shown in Figure 2, the model can extract both image features and noise features, making the model more robust to image detection containing artifacts. Indicates feature addition. S12 specifically includes:

[0082] S121: extracting image features of the clean image using an image feature extraction module to obtain image features of the clean image; extracting image features of the artifact image using the image feature extraction module to obtain image features of the artifact image; and extracting artifact features of the artifact image using the artifact feature extraction module;

[0083] S122: fusing the artifact feature with the image feature of the clean image and the image feature of the artifact image, respectively, here the image features are added together to obtain two fused image features;

[0084] S123: Inputting the image features of the clean image, the image features of the artifact image, and the two fused image features into the target detection network of the detection module for detection, and obtaining detection results, which are detection output 1, detection output 2, detection output 3, and detection output 4 respectively;

[0085] S124: Use the true value and the detection result to construct a loss function, use the training set of S11 to train the neural network model, and optimize the neural network model parameters.

[0086] In one embodiment, the image feature extraction module is as follows: Figure 3As shown in Figure 1, the DBL module (Darknet conv2D BN Leaky, DBL) is the basic convolutional module in the Darknet network, consisting of convolutional layers using batch normalization (BN) and a Leaky activation function. The residual module (residual module×n, resn) first pads the feature map with zeros, and then consists of a DBL module and n residual units. Each residual unit contains two DBL modules with residual connections, as shown in Figure 1. Figure 4 shown.

[0087] In a preferred embodiment, the structure of the artifact feature extraction module in S121 is as follows: Figure 5 As shown in the figure, for each input artifact image, it passes through 5 convolution modules.

[0088] In one embodiment, the structure of the target detection network in S123 is as follows: Figure 6 As shown in the figure, concat represents tensor concatenation, upsample represents upsampling the feature image through interpolation, and conv represents a convolutional layer. The final output includes the target box's horizontal and vertical coordinates, width, height, object confidence, and predicted probabilities for each category.

[0089] In a preferred embodiment, in order to simultaneously optimize the two images and impose constraints, the training set of S11 is used to train the neural network model in S124. The optimization of the neural network model parameters is specifically as follows: a joint loss function is constructed to simultaneously optimize the clean image and the artifact image to minimize the loss function. The loss function minimization problem is defined as:

[0090]

[0091] In the above formula, θ represents the model parameters, n is the number of images in each training, that is, the number of images contained in the image block input for each training of the model; w1,w2,w3,w4,w d is the weight of each loss function; L1 is the loss of the detection result of the image feature of the clean image and the true value, L2 is the loss of the detection result of the fusion of the image feature of the clean image and the artifact feature and the true value, L3 is the loss of the detection result of the fusion of the image feature of the artifact image and the artifact feature and the true value, L4 is the loss of the detection result of the image feature of the artifact image, L d is the artifact feature extraction network penalty term.

[0092] In a preferred embodiment, the loss functions of L1, L2, L3, and L4 are all YOLO-type loss functions L YOLO , which has the following form:

[0093] L YOLO =Lx +L y +L w +L h +L conf_obj +L conf_noobj +L cls

[0094] Among them L x ,L y ,L w ,L h They are the horizontal coordinate, vertical coordinate, width and height error losses of the target detection box, L conf_obj and L conf_noobj are the confidence losses of objects and non-objects respectively, L cls is the classification confidence loss;

[0095] The horizontal axis loss is:

[0096]

[0097] in Indicates that there is an object at the position of the predicted box, N×N is the size of the feature map to be detected, x, x′ are the horizontal coordinates of the real box and the predicted box respectively;

[0098] The vertical axis loss is:

[0099]

[0100] Where y, y′ are the vertical coordinates of the real box and the predicted box respectively;

[0101] The width loss is:

[0102]

[0103] Where w, w′ are the widths of the real box and the predicted box respectively;

[0104] The height loss is:

[0105]

[0106] Where h, h′ are the heights of the real box and the predicted box respectively;

[0107] The object confidence loss is:

[0108]

[0109] where c ij Represents the predicted probability that there is an object at a certain position in the target box;

[0110] The non-object confidence loss is:

[0111]

[0112] in Indicates that there is no object at the location of the prediction box, c ij Represents the predicted probability that there is an object at a certain position in the target box;

[0113] The classification confidence loss is:

[0114]

[0115] where p ij (c) represents the probability that there is an object of category c at the position of the prediction box, p′ ij (c) represents the predicted probability that there is an object of category c at the position of the prediction box. In this embodiment, there is only one type of object, which is the cavity, that is, c represents the cavity category.

[0116] L d for:

[0117]

[0118] Where m is the artifact feature map, which is the output of the artifact feature extraction network.

[0119] The effects of the above embodiment are verified below.

[0120] 1. Experimental Conditions

[0121] This experiment used 2288 clean images and 86 artifact-laden images as the model training set. The image size for detection was 416×416 pixels. Furthermore, a validation set of 763 clean images and 288 artifact-laden images outside the training set was used to adjust model hyperparameters. A test set of 765 clean images and 290 artifact-laden images outside the training and validation sets was used to verify the model's robustness for lumen detection in artifact-laden images and to test the model's effectiveness. The model was implemented using PyTorch and computed on an Nvidia GTX Titan Xp GPU.

[0122] The parameters in the training process are as follows: Adam is used as the optimizer, and the training process is 100 generations. The model with the highest index on the validation set is selected as the final model; the batch size of each training is set to 8, w1, w2 in the loss function are set to 2, w3, w4 are set to 0.1, and w d The initial learning rate of the training image feature extraction module and the artifact feature extraction module is set to 0.0001, and the initial learning rate of the training detection module is set to 0.001.

[0123] 2. Experimental content

[0124] The robust cavity detection method proposed in the present invention is verified from both qualitative and quantitative perspectives.

[0125] 2.1 Qualitative Experimental Results

[0126] Figure 7 The present invention was used to perform cavity detection on images in the test set, and the results were compared with those obtained by other methods.

[0127] Figure 7 In the paper, the detection results of the proposed method were compared under the same conditions: direct training with YOLOv3, training with YOLOv3 and then fine-tuning the model using artifact images, and the detection results of the proposed method. It can be seen that the proposed method performs better in lumen detection tasks for both clean and artifact images, verifying the good performance and robustness of the proposed method for lumen detection in endoscopic images.

[0128] Quantitative analysis

[0129] We used our method to perform a quantitative error analysis on object detection results for 765 clean images and 290 artifact images in the test set. The comparisons were between the YOLOv3 network and a YOLOv3+Finetune method trained with the YOLOv3 network and then fine-tuned using artifact images. The evaluation metric was mean average precision (mAP):

[0130]

[0131] Wherein, APi is the average precision (AP) of the i-th class of objects, C is the number of classes, and in the present invention, C=1, so mAP=AP.

[0132] The average precision on a certain category is calculated as:

[0133]

[0134] In the formula, N represents the number of images containing objects of this category, and precision(j) represents the detection accuracy of objects of this category on the jth image. The detection accuracy on a certain image is calculated as follows:

[0135]

[0136] The N(true_positives) in the formula represents the number of objects correctly identified as this type in the image; N(total_objects) represents the total number of objects of this type in the image.

[0137] The value range of mAP is 0-1, and the closer it is to 1, the better the algorithm effect.

[0138] After testing the images in the test set using different methods and taking the average, the experimental results are shown in Table 1:

[0139] Table 1. Comparison of mAP of different detection methods on the test set (the best results are in bold)

[0140]

[0141] The results in Table 1 show that the proposed method takes into account both endoscopic image features and artifact features, achieving an average precision of 0.675 on the entire test set. Compared to the traditional YOLOv3 detection network, this numerical result shows that the proposed robust lumen detection method based on artifact feature fusion training achieves better results on the test set, and artifact feature fusion improves the model's generalization and robustness.

[0142] Based on the same technical concept as above, in another embodiment, an endoscope robust lumen detection system is further provided, which is used to implement the endoscope image robust lumen detection method of any of the above embodiments, and includes: a data set construction unit, a model building unit, a model training unit and a model testing unit; wherein,

[0143] The data set construction unit is used to divide a given number of endoscopic images into clean images and artifact images, and then divide the clean images and artifact images into training sets and test sets respectively;

[0144] The model building unit is used to build a neural network model for robust cavity detection;

[0145] The model training unit is used to train the neural network model established by the model building unit using the training set obtained by the data building unit;

[0146] The model testing unit is used to test the neural network model trained by the model training unit using the test set obtained by the data set construction unit.

[0147] Based on the same technical concept above, in another embodiment, a terminal is also provided, which includes: a memory, a processor, and a computer program stored in the memory and runnable on the processor. When the processor executes the program, it can be used to execute the robust cavity detection method of endoscopic images of any of the above embodiments.

[0148] Based on the same technical concept above, in another embodiment, a computer-readable storage medium is further provided, on which a computer program is stored. When the program is executed by a processor, it can be used to execute the robust cavity detection method of endoscopic images of any of the above embodiments.

[0149] It should be noted that the steps in the method provided by the present invention can be implemented using the corresponding modules, devices, units, etc. in the system. Those skilled in the art can refer to the technical solution of the system to implement the step flow of the method, that is, the embodiments in the system can be understood as preferred examples for implementing the method, which will not be elaborated here.

[0150] Those skilled in the art will appreciate that, in addition to implementing the system and its various devices provided by the present invention in purely computer-readable program code, it is entirely possible to implement the same functions of the system and its various devices provided by the present invention in the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, embedded microcontrollers, etc. by logically programming the method steps. Therefore, the system and its various devices provided by the present invention can be considered a hardware component, and the devices included therein for implementing the various functions can also be considered as structures within the hardware component; the devices for implementing the various functions can also be considered as both software modules implementing the method and structures within the hardware component.

[0151] Those skilled in the art will understand that all features disclosed in this specification and all processes or units of any device disclosed herein may be combined in any combination, unless at least some of such features and / or processes or units are mutually exclusive.

[0152] Disclosed herein are only preferred embodiments of the present invention. These embodiments are selected and described in detail in this specification to better explain the principles and practical applications of the present invention and are not intended to limit the present invention. Any modifications and variations made by those skilled in the art within the scope of this specification are intended to fall within the scope of protection of the present invention.

Claims

1. A robust lumen detection method for endoscopic images, characterized in that: include: S11: Construct dataset; For a given number of endoscopic images, the images are divided into clean images and artifact images, and the clean images and the artifact images are divided into a training set and a test set respectively; S12: Establish a neural network model for robust cavity detection; S13: training model; Using the training set obtained in S11 to train the neural network model established in S12; S14: test model; Using the test set obtained in S11 to test the neural network model trained in S13; The neural network model established in S12 is a target detection neural network model with feature extraction and fusion; The S12 specifically includes: S121: performing image feature extraction on the clean image to obtain image features of the clean image; and performing image feature extraction and artifact feature extraction on the artifact image to obtain image features and artifact features of the artifact image; S122: fusing the artifact feature with the image feature of the clean image and the image feature of the artifact image respectively to obtain two fused image features; S123: Inputting the two fused image features into a target detection network for detection to obtain a detection result; S124: Construct a loss function using the preset labeled true value and the detection result, train the neural network model using the training set of S11, and optimize the neural network model parameters.

2. The robust cavity detection method of endoscopic images according to claim 1, characterized in that: The S122 specifically includes: adding the artifact feature to the image feature of the clean image and the image feature of the artifact image respectively to obtain two fused image features; The S123 specifically includes: inputting the image features of the clean image, the image features of the artifact image, and the fused image features into a detection network respectively to obtain a detection result.

3. The robust lumen detection method for endoscopic images according to claim 2, characterized in that: The loss function minimization problem in S124 is defined as: In the above formula, θ represents the model parameters, n is the number of images in each training, that is, the number of images contained in the image block input for each training of the model; w1,w2,w3,w4,w d is the weight of each loss function; L1 is the loss of the detection result of the image feature of the clean image and the true value, L2 is the loss of the detection result of the fusion of the image feature of the clean image and the artifact feature and the true value, L3 is the loss of the detection result of the fusion of the image feature of the artifact image and the artifact feature and the true value, L4 is the loss of the detection result of the image feature of the artifact image, L d is the artifact feature extraction network penalty term.

4. The robust lumen detection method for endoscopic images according to claim 3, characterized in that: The loss functions of L1, L2, L3, and L4 are all YOLO-type loss functions. YOLO , which has the following form: L YOLO =L x +L y +L w +L h +L conf_obj +L conf_noobj +L cls Among them L x ,L y ,L w ,L h They are the horizontal coordinate, vertical coordinate, width and height error losses of the target prediction box, L conf_obj and L conf_noobj are the confidence losses of objects and non-objects respectively, L cls is the classification confidence loss; The horizontal axis loss is: in Indicates that there is an object at the position of the predicted box, N×N is the size of the feature map to be detected, x, x′ are the horizontal coordinates of the real box and the predicted box respectively; The ordinate loss is: Where y, y′ are the vertical coordinates of the real box and the predicted box respectively; The width loss is: Where w, w′ are the widths of the real box and the predicted box respectively; The height loss is: Where h, h′ are the heights of the real box and the predicted box respectively; The object confidence loss is: where c ij Represents the predicted probability that there is an object at a certain position in the target box; The non-object confidence loss is: in Indicates that there is no object at the location of the prediction box, c ij Represents the predicted probability that there is an object at a certain position in the target box; The classification confidence loss is: where p ij (c) represents the probability that there is an object of category c at the position of the prediction box, and p′ij(c) represents the predicted probability that there is an object of category c at the position of the prediction box.

5. The robust cavity detection method of endoscope images according to claim 4, characterized in that: The L d for: Where m is the artifact feature map, which is the output of the artifact feature extraction network.

6. A robust cavity detection system for endoscope images, characterized in that: include: Data set construction unit, model building unit, model training unit and model testing unit; among them, The data set construction unit is used to divide a given number of endoscopic images into clean images and artifact images, and then divide the clean images and the artifact images into a training set and a test set respectively; The model building unit is used to build a neural network model for robust cavity detection; The model training unit is used to train the neural network model established by the model building unit using the training set obtained by the data set building unit; The model testing unit is used to test the neural network model trained by the model training unit using the test set obtained by the data set construction unit; The neural network model established by the model building unit is a target detection neural network model with feature extraction and fusion; The model building unit specifically includes: Performing image feature extraction on the clean image to obtain image features of the clean image; and performing image feature extraction and artifact feature extraction on the artifact image to obtain image features and artifact features of the artifact image; fusing the artifact features with the image features of the clean image and the image features of the artifact image respectively to obtain two fused image features; Inputting the two fused image features into the target detection network for detection to obtain a detection result; A loss function is constructed using the preset labeled true value and the detection result, and the neural network model is trained using the training set to optimize the neural network model parameters.

7. The robust cavity detection system for endoscopic images according to claim 6, characterized in that: In the model building unit: fusing the artifact feature with the image feature of the clean image and the image feature of the artifact image to obtain two fused image features, specifically comprising: adding the artifact feature with the image feature of the clean image and the image feature of the artifact image to obtain two fused image features; The two fused image features are input into a target detection network for detection to obtain a detection result, specifically comprising: inputting the image features of the clean image, the image features of the artifact image, and the fused image features into a detection network respectively to obtain a detection result.

8. The robust cavity detection system for endoscope images according to claim 6, characterized in that: In the model building unit, the loss function minimization problem is defined as: In the above formula, θ represents the model parameters, n is the number of images in each training, that is, the number of images contained in the image block input for each training of the model; w1,w2,w3,w4,w d is the weight of each loss function; L1 is the loss of the detection result of the image feature of the clean image and the true value, L2 is the loss of the detection result of the fusion of the image feature of the clean image and the artifact feature and the true value, L3 is the loss of the detection result of the fusion of the image feature of the artifact image and the artifact feature and the true value, L4 is the loss of the detection result of the image feature of the artifact image, L d is the artifact feature extraction network penalty term.

9. A terminal comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor can be used to execute the method according to any one of claims 1 to 7 when executing the program.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, it can be used to perform the method according to any one of claims 1 to 7.

Images