A target detection method for hysteroscopic images based on depth information and knowledge distillation

Abstract

The present invention discloses a hysteroscopic image target detection method based on depth information and knowledge distillation, and belongs to the field of medical image processing. The present invention uses depth information and knowledge distillation to deploy a trained student model in a target detection system to collect and preprocess data on hysteroscopic images. The preprocessed hysteroscopic image and the extracted hallucination depth information are input into the teacher model, a lightweight student model is trained using a gradient-based knowledge distillation technique, and the deployed student model is used to perform target detection on the hysteroscopic image to obtain information such as the location and size of the detected lesion area. The present invention can effectively improve the performance of hysteroscopic image target detection tasks, improve the accuracy and robustness of target detection, and provide doctors with accurate and reliable lesion detection results, thereby assisting clinical diagnosis and treatment.

Description

Technical Field

[0001] The invention relates to a hysteroscopic image target detection method based on depth information and knowledge distillation, and belongs to the field of medical image processing. Background Art

[0002] Hysteroscopy is a new, minimally invasive gynecological diagnosis and treatment technology. It is a fiber-light endoscope used for intrauterine examination and treatment. It uses the front part of the scope to enter the uterine cavity, which has a magnifying effect on the observed area. It is intuitive and accurate and has become the preferred examination method for gynecological hemorrhagic diseases and intrauterine lesions. Hysteroscopic image target detection refers to the use of computer vision and deep learning technologies to analyze and process images obtained during hysteroscopy to automatically identify and locate target objects or abnormal areas in the image. With the development of deep learning technology, more and more studies have applied deep learning models to hysteroscopic image target detection, such as deep convolutional neural network (CNN), recurrent neural network (RNN) and other models. However, these models have the characteristics of insufficient model generalization ability, high computing resources and time costs, and cannot meet the real-time requirements of medical image detection.

[0003] In order to solve this problem, a feasible solution is to use depth information and knowledge distillation to perform hysteroscopic image target detection. Among them, target detection based on hallucination depth fusion and prior enhancement refers to the use of deep learning and neural network technology to achieve target detection by fusing multiple hallucination depth information and prior knowledge in the target detection task. However, the hallucination depth information is subject to the quality and characteristics of the input image, and its accuracy may be limited. Therefore, a technology for deep neural network model compression and acceleration is introduced, that is, to guide the knowledge extraction process through gradients, and to guide the training process of the student model by using the gradient information between the teacher model and the student model, so as to more effectively transfer the knowledge of the teacher model. In summary, a method of enhancing target detection performance by fusing depth information and a new method of gradient-guided knowledge extraction can reduce the number of model parameters and computational complexity while maintaining model performance, and reduce the data requirements and cost of target detection tasks.

[0004] The present invention is based on the fusion of hallucination depth and prior enhancement and gradient-guided knowledge extraction technology, and proposes a hysteroscopic image target detection method based on depth information and knowledge distillation. Summary of the invention

[0005] The purpose of the present invention is to provide a hysteroscopic image target detection method based on depth information and knowledge distillation. Mainly by fusing depth information, the model can better understand the target structure and background information in the image, thereby improving the accuracy and robustness of detection. At the same time, the gradient-guided knowledge extraction method can guide the learning process of the student model through the knowledge of the teacher model, further improving the performance and generalization ability of the model.

[0006] The purpose of the present invention is achieved through the following technical solutions:

[0007] A method for hysteroscopic image target detection based on depth information and knowledge distillation comprises the following steps:

[0008] Step 1: Collect hysteroscopic image datasets for training and evaluation, including normal and pathological samples, and perform data preprocessing operations such as denoising and size standardization. These datasets only contain images and corresponding target bounding box annotations;

[0009] Step 2: Use a monocular depth estimation algorithm to obtain the depth of each pixel in the image. Convert the originally grayscale depth image into an RGB image with three channels using color mapping, and then concatenate it with the RGB channels as input.

[0010] Step 3: Use the Siamese network with shared parameters as the teacher network and apply RoI pooling to extract a fixed-length feature vector for each visual region from the RGB and depth inputs Proposal Features Feed into two parallel fully connected layers to calculate the detection score and classification scores

[0011] Step 4: Extract the RGB feature map ψ(I) and the depth feature map ψ(D) from the depth map of the scene predicted by the single RGB image and feed them into the adaptive pooling and fully connected layers to obtain the D-dimensional projected RGB feature map ψ proj (I) and the deep feature map ψ proj (D), after normalizing the two feature vectors, we get the cosine similarity S(I,D), and then use the depth estimation model to locate the target and predict the depth of the image. The first component L of the NCE loss in the estimation process D→I With the second component L I→D Calculate the average L NCE ;

[0012] Step 5: The derived image visual feature scores and deep feature scores Perform fusion, probability conversion, element-wise multiplication, image-level calculation, and loss optimization operations to obtain the region-level score Lmil ;

[0013] Step 6: For the image and image-level annotation information that combines visual features and virtual depth illusion features, use the captions and bounding box predictions of target detection to calculate the depth prior range dr c And define the depth mask indicator variable m i,c , in order to extract the illusion depth information of hysteroscopic images and enhance the image features;

[0014] Step 7: Integrate the above twin network with the fusion network and use the hysteroscopic image dataset as the teacher model for training; initialize a lightweight student model;

[0015] Step 8: Transfer the knowledge of the teacher model to the student model, and use the gradient to guide the knowledge distillation process to perform the following operations:

[0016] During the training process, the gradient of the detection loss relative to the feature map is used to represent the contribution of the feature to the final detection. The larger the gradient, the greater the influence of the corresponding feature on the decision, so as to obtain the final target map M of the distillation. l ; Use these gradients to weight the features and obtain the loss function L of knowledge distillation GKD ; Extract Gaussian mask F i,j , calculate the position attention mask M P and channel attention mask M C , we get the loss L of multi-granularity features in bounding box detection BMFI ; plus the gradient-guided knowledge extraction process loss L GKD , and the total distillation loss L is obtained KD ;

[0017] Step 9: After knowledge distillation is completed, use the validation set to verify and evaluate the trained student model, tune the model according to the validation results, and repeat steps 2-7 until the number of training times reaches the expected value;

[0018] Step 10: Deploy the trained lightweight optimal student model on the hysteroscopic image for target detection tasks to enhance real-time performance.

[0019] Furthermore, the calculation formulas for the detection score and classification score in step 3 are as follows:

[0020]

[0021] In which, let I∈R h×w×3 Represents RGB image, y c∈{0,1} is the corresponding true base class label, c∈{1,…,C} and C is the total number of object categories. Let vi,i∈{1,…,R}, where R is the number of region proposals, representing the visual proposals in image i, and w and b are the weights and biases respectively.

[0022] Furthermore, the cosine similarity calculation formula in step 4 is as follows:

[0023]

[0024] In which, let I∈R h×w×3 Represents an RGB image, D∈R h×w×3 Denote the depth image associated with the RGB image I, let ψ(I) be the feature map extracted from the RGB image, ψ(D) be the feature map of the depth image D extracted by the Siamese network backbone, and ρ be a learnable temperature parameter.

[0025] Furthermore, the first component L of the contrast loss in step 4 D→I The RGB image is compared with the negative depth image to measure the degree of depth matching between the RGB image and the other images in its batch. The first component of the contrast loss is calculated as follows:

[0026]

[0027] Where B is a batch of RGB depth pairs, RGB image and depth image pair (I,D)∈B, S(I,D) is the cosine similarity;

[0028] The second component of the contrast loss L I→D is similarly defined as comparing the depth image with the negative RGB image sample, estimating the first component of the contrast loss L D→I With the second component L I→D Average and calculate the contrast loss L NCE , the contrast loss calculation formula is as follows:

[0029]

[0030] Among them, L D→I is the first component of the contrast loss, L I→D is the second component of contrast loss.

[0031] Furthermore, the image visual feature score derived in step 5 is and deep feature scores To perform the fusion:

[0032]

[0033] in, and are the fusion detection score and classification score respectively, is the detection score, is the classification score;

[0034] These classification scores and detection scores are converted to probabilities, and the probability conversion formula is as follows:

[0035]

[0036] in, Is class c present in the current proposal f i The probability of Yes i For predicting image-level labels y c Probability of importance;

[0037] Multiply the classification score and the detection score element by element to get the combined score

[0038]

[0039] Finally, image-level prediction The calculation formula is as follows:

[0040]

[0041] in, is the combined score;

[0042] Then, the regional level score L is obtained by optimizing the loss mil , the calculation formula is as follows:

[0043]

[0044] in, is the image-level prediction, y c ∈{0,1} is the corresponding true base class label.

[0045] Furthermore, in step 6, for any new depth image D, the range dr of the object estimated depth prior c The formula is as follows:

[0046]

[0047] Where C is the set of object categories, S is the set of words in the title corresponding to D, and r c,s is the depth range of C categories and word set S;

[0048] Using the estimated depth prior range dr cTo identify the potential important areas in each category; for each area, define a depth mask indicator variable m i,c , which indicates the possibility that a certain area in the image contains a certain type of object. The calculation of this variable is as follows:

[0049]

[0050] Among them, if the proposed depth value pdi falls within the estimated depth prior range dr of class c c If the area is within , the area is considered to be a relevant area of ​​the class, and the corresponding mask variable m is i,c Set to 1 if false; otherwise, set to 0.

[0051] Furthermore, in step 8, during the training process, L is first calculated. task Relative to the feature The gradient of the detector layer l k feature map weight Defined as:

[0052]

[0053] Among them, L task represents the total detection loss, including bounding box regression loss and classification loss, is the single activation value at position (i, j) in layer k, W, H are the gradient and width of the reflow;

[0054] These reflowed gradients are globally average pooled in width and height dimensions to obtain feature channel importance Then, use To measure the k-th activation map

[0055]

[0056] in, is the k-th gradient weighted activation of the l-th layer, is the feature channel importance;

[0057] These maps are then linearly combined along the channel dimension, and the gradient of the detection loss with respect to the feature map is used to represent the contribution of the feature to the final detection before taking the absolute value and norm to obtain the distilled final target map M l , the final target mapping calculation formula is as follows:

[0058]

[0059] Among them, Norm represents the minimum-maximum normalization function, is the kth gradient weighted activation of the lth layer;

[0060] These gradients are used to weight the features and obtain the loss function L for gradient-guided knowledge extraction. GKD ;

[0061]

[0062] Where W, H are the gradient and width of the reflux, L is the total number of intermediate layers considered for distillation, l represents the intermediate layer, and are the generated teacher and student target mappings, respectively;

[0063] When extracting features, a Gaussian mask F is used i,j , which is defined as:

[0064]

[0065] Among them, o and Respectively represent the area inside the true value bounding box and the surrounding area, (x, y) represents a specific point in the surrounding area; Represents the center point of the ground-truth bounding box;

[0066] When extracting features, position and channel attention are also combined, and the position attention mask M P and channel attention mask M C The calculation formula is as follows:

[0067]

[0068] Where A is a common feature map, W, H, and C are the width, height, and channel number of A, respectively, with i, j, and k as indexes; T is the temperature hyperparameter introduced to adjust the distribution;

[0069] Based on the Gaussian mask F i,j , location attention mask M P , channel attention mask M C , we get the loss L of multi-granularity features in bounding box detection BMFI :

[0070]

[0071] Among them, the subscripts T and S are the teacher detector and the student detector respectively. and are the teacher and student feature maps, M P,T and M P,S are the teacher and student position attention masks, M C,T and M C,S They are the teacher and student channel attention masks, respectively;

[0072] Finally, we add the loss L of the gradient-guided knowledge extraction process GKD , and the total distillation loss L is obtained KD :

[0073] L KD =L GKD +βL BMFI

[0074] Among them, β is the balance hyperparameter, L GKD is the loss of the gradient-guided knowledge distillation process, L BMFI It is the loss of multi-granular features in bounding box detection.

[0075] The present invention improves the target detection performance by fusing depth information, adopts monocular depth estimation to obtain hallucinated depth information, and then trains the twin network using contrast loss. The twin network generates deep features of image features, and uses the region proposals generated by RPN to obtain regional image features and regional depth features in image features and depth features respectively. By analyzing the conditional depth range of each object class and co-occurring words, the depth prior is calculated to identify bounding box proposals that may contain objects of interest. These depth priors are then used to update the pseudo-ground truth box list or adjust the confidence of each box prediction.

[0076] The present invention introduces gradient to guide the distillation process and uses gradient information to measure the importance of features. It proposes the application of multi-granularity features in bounding box detection. In the distillation process, the information of the bounding box and its object class and co-occurring words is considered, and distillation is performed along different feature dimensions.

[0077] The beneficial effects of the present invention are as follows: the amplification method of fused depth information is used to introduce richer depth features, and enhance the model's understanding and detection capabilities of targets (such as lesion areas, instruments, etc.) in hysteroscopic images. At the same time, the gradient-guided knowledge extraction method guides the learning process of the student model through the knowledge of the teacher model, thereby improving the accuracy and stability of the model. The present invention comprehensively utilizes the advantages of deep information fusion and knowledge distillation, and can effectively improve the performance of hysteroscopic image target detection tasks, providing doctors with more accurate diagnosis and surgical support. BRIEF DESCRIPTION OF THE DRAWINGS

[0078] Figure 1 It is a flow chart of the hysteroscopic image target detection method in the present invention.

[0079] Figure 2 It is a rendering of the hysteroscopic image target detection method of the present invention. DETAILED DESCRIPTION

[0080] The present invention will be further described below in conjunction with the accompanying drawings and specific embodiments.

[0081] like Figure 1 As shown, the hysteroscopic image target detection method based on depth information and knowledge distillation of the present invention comprises the following steps:

[0082] Step 1: Collect hysteroscopic image datasets for training and evaluation, including normal and pathological samples, and perform data preprocessing operations such as denoising and size standardization. These datasets only contain images and corresponding target bounding box annotations;

[0083] Step 2: Use a monocular depth estimation algorithm to obtain the depth of each pixel in the image. Convert the originally grayscale depth image into an RGB image with three channels using color mapping, and then concatenate it with the RGB channels as input.

[0084] Step 3: Use the Siamese network with shared parameters as the teacher network to extract feature vectors from RGB and depth inputs (v i ), used to calculate the detection score and classification scores The calculation formulas for the detection score and classification score are as follows:

[0085]

[0086] In which, let I∈R h×w×3 Represents RGB image, y c ∈{0,1} is the corresponding true base class label, c∈{1,…,C} and C is the total number of object categories. Let vi,i∈{1,…,R} (where R is the number of region proposals) represent the visual proposals in image i, w and b are the weights and biases respectively;

[0087] Step 4: Extract the RGB feature map ψ(I) and the depth feature map ψ(D) from the depth map of the scene predicted by the single RGB image and feed them into the adaptive pooling and fully connected layers to obtain the D-dimensional projected RGB feature map ψ proj (I) and the deep feature map ψ proj (D), after normalizing the two feature vectors, we get the cosine similarity S(I,D). The formula for calculating the cosine similarity is as follows:

[0088]

[0089] In which, let I∈R h×w×3 Represents an RGB image, D∈R h×w×3 Denote the depth image associated with the RGB image I, let ψ(I) be the feature map extracted from the RGB image, ψ(D) be the feature map of the depth image D extracted by the Siamese network backbone, and ρ be a learnable temperature parameter.

[0090] The depth estimation model is then used to locate the target and predict the depth of the image. The first component of the contrast loss L D→I The RGB image is compared with the negative depth image to measure the degree of depth matching between the RGB image and the other images in its batch. The first component of the contrast loss is calculated as follows:

[0091]

[0092] Where B is a batch of RGB depth pairs, RGB image and depth image pair (I,D)∈B, S(I,D) is the cosine similarity;

[0093] The second component of the contrast loss L I→D is similarly defined as comparing the depth image with the negative RGB image sample, estimating the first component of the contrast loss L D→I With the second component L I→D Average and calculate the contrast loss L NCE , the contrast loss calculation formula is as follows:

[0094]

[0095] Among them, L D→I is the first component of the contrast loss, L I→D is the second component of contrast loss;

[0096] Step 5: The derived image visual feature scores and deep feature scores To perform the fusion:

[0097]

[0098] in, and are the fusion detection score and classification score respectively, is the detection score, is the classification score;

[0099] These classification scores and detection scores are converted to probabilities, and the probability conversion formula is as follows:

[0100]

[0101] in, Is class c present in the current proposal f i The probability of Yes i For predicting image-level labels y c Probability of importance;

[0102] Multiply the classification score and the detection score element by element to get the combined score

[0103]

[0104] Finally, image-level prediction The calculation formula is as follows:

[0105]

[0106] in, is the combined score;

[0107] Then, the regional level score L is obtained by optimizing the loss mil , the calculation formula is as follows:

[0108]

[0109] in, is the image-level prediction, y c ∈{0,1} is the corresponding true base class label;

[0110] Step 6: For the image and image-level annotation information that combines visual features and virtual depth illusion features, use the captions and bounding box predictions of target detection to calculate the depth prior range dr c And define the depth mask indicator variable m i,c , in order to extract the illusion depth information of hysteroscopic images and enhance the image features;

[0111] For any new depth image D, the range of the object estimated depth prior dr c The formula is as follows:

[0112]

[0113] Where C is the set of object categories, S is the set of words in the title corresponding to D, and r c,s is the depth range of C categories and word set S;

[0114] Using the estimated depth prior range dr c To identify the potentially important regions in each category. For each region, define a depth mask indicator variable m i,c , which indicates the possibility that a certain area in the image contains a certain type of object. The calculation of this variable is as follows:

[0115]

[0116] Among them, if the proposed depth value pdi falls within the estimated depth prior range dr of class c c If the area is within , the area is considered to be a relevant area of ​​the class, and the corresponding mask variable m is i,cSet to 1 if false; otherwise, set to 0.

[0117] Step 7: Use the hysteroscopic image dataset to train a deep learning object detection model as the teacher model and initialize a lightweight student model;

[0118] Step 8: Transfer the knowledge of the teacher model to the student model by performing the following operations through the gradient-guided knowledge extraction method:

[0119] During the training process, first calculate L task Relative to the feature The gradient of the detector layer l k feature map weight Defined as:

[0120]

[0121] Among them, L task represents the total detection loss (including bounding box regression loss and classification loss), is the single activation value at position (i, j) in layer k, W, H are the gradient and width of the reflow;

[0122] These reflowed gradients are globally average pooled in width and height dimensions to obtain feature channel importance Then, we use To measure the k-th activation map

[0123]

[0124] in, is the k-th gradient weighted activation of the l-th layer, is the feature channel importance;

[0125] These maps are then linearly combined along the channel dimension (before taking the absolute value and norm), and the gradient of the detection loss with respect to the feature map is used to represent the contribution of the feature to the final detection to obtain the distilled final target map M l , the final target mapping calculation formula is as follows:

[0126]

[0127] Among them, Norm represents the minimum-maximum normalization function, is the kth gradient weighted activation of the lth layer;

[0128] These gradients are used to weight the features and obtain the loss function L for gradient-guided knowledge extraction. GKD ;

[0129]

[0130] Where W, H are the gradient and width of the reflux, L is the total number of intermediate layers considered for distillation, l represents the intermediate layer, and are the generated teacher and student target mappings, respectively;

[0131] When extracting features, a Gaussian mask F is used i,j , which is defined as:

[0132]

[0133] Among them, o and Respectively represent the area inside the true value bounding box and the surrounding area, (x, y) represents a specific point in the surrounding area; Represents the center point of the ground-truth bounding box;

[0134] When extracting features, position and channel attention are also combined, and the position attention mask M P and channel attention mask M C The calculation formula is as follows:

[0135]

[0136] Where A is a common feature map, W, H, and C are the width, height, and channel number of A, respectively, with i, j, and k as indexes; T is the temperature hyperparameter introduced to adjust the distribution;

[0137] Based on the Gaussian mask F i,j , location attention mask M P , channel attention mask M C , we get the loss L of multi-granularity features in bounding box detection BMFI :

[0138]

[0139] Among them, the subscripts T and S are the teacher detector and the student detector respectively. and are the teacher and student feature maps, M P,T and M P,S are the teacher and student position attention masks, M C,T and M C,S They are the teacher and student channel attention masks, respectively;

[0140] Finally, we add the loss L of the gradient-guided knowledge extraction process GKD , and the total distillation loss L is obtained KD :

[0141] L KD =L GKD+βL BMFI

[0142] Among them, β is the balance hyperparameter, L GKD is the loss of the gradient-guided knowledge distillation process, L BMFI is the loss of multi-granular features in bounding box detection;

[0143] Step 9: After knowledge distillation is completed, use the validation set to verify and evaluate the trained student model, tune the model according to the validation results, and repeat steps 2-7 until the number of training times reaches the expected value;

[0144] Step 10: Deploy the trained lightweight optimal student model on the hysteroscopic image for target detection tasks to enhance real-time performance. Figure 2 shown.

[0145] In addition to the above embodiments, the present invention may also have other implementation modes. Any technical solutions formed by equivalent replacement or equivalent transformation shall fall within the protection scope required by the present invention.

Claims

1. A method for object detection in hysteroscopic images based on depth information and knowledge distillation, characterized in that: The following steps are involved: Step 1: Collect hysteroscopic image datasets for training and evaluation, including normal and pathological samples, and perform data preprocessing operations such as denoising and size standardization. These datasets only contain images and corresponding target bounding box annotations; Step 2: Use a monocular depth estimation algorithm to obtain the depth of each pixel in the image. Convert the originally grayscale depth image into an RGB image with three channels using color mapping, and then concatenate it with the RGB channels as input. Step 3: Use the Siamese network with shared parameters as the teacher network and apply RoI pooling to extract a fixed-length feature vector for each visual region from the RGB and depth inputs Proposal Features Feed into two parallel fully connected layers to calculate the detection score and classification scores Step 4: Extract the RGB feature map ψ(I) and the depth feature map ψ(D) from the depth map of the single RGB image prediction scene and feed them into the adaptive pooling and fully connected layers to obtain the D-dimensional projected RGB feature map ψ prpj (I) and the deep feature map ψ proj (D), after normalizing the two feature vectors, we get the cosine similarity S(I,D), and then use the depth estimation model to locate the target and predict the depth of the image. The first component L of the NCE loss in the estimation process D→I With the second component L I→D Calculate the average L NCE ; Step 5: The derived image visual feature scores and deep feature scores Perform fusion, probability conversion, element-wise multiplication, image-level calculation, and loss optimization operations to obtain the region-level score L mil ; Step 6: For the image and image-level annotation information that combines visual features and virtual depth illusion features, use the captions and bounding box predictions of target detection to calculate the depth prior range dr c And define the depth mask indicator variable m i,c , in order to extract the illusion depth information of hysteroscopic images and enhance the image features; Step 7: Integrate the above twin network with the fusion network and use the hysteroscopic image dataset as the teacher model for training; initialize a lightweight student model; Step 8: Transfer the knowledge of the teacher model to the student model, and use the gradient to guide the knowledge distillation process to perform the following operations: During the training process, the gradient of the detection loss relative to the feature map is used to represent the contribution of the feature to the final detection. The larger the gradient, the greater the influence of the corresponding feature on the decision, so as to obtain the final target map M of the distillation. l ; Use these gradients to weight the features and obtain the loss function L of knowledge distillation GKD ; Extract Gaussian mask F i,j , calculate the position attention mask M P and channel attention mask M C , we get the loss L of multi-granularity features in bounding box detection BMFI ; plus the gradient-guided knowledge extraction process loss L GKD , and the total distillation loss L is obtained KD ; Step 9: After knowledge distillation is completed, use the validation set to verify and evaluate the trained student model, tune the model according to the validation results, and repeat steps 2-7 until the number of training times reaches the expected value; Step 10: Deploy the trained lightweight optimal student model on the hysteroscopic image for target detection tasks to enhance real-time performance.

2. The method for hysteroscopic image target detection based on depth information and knowledge distillation according to claim 1, characterized in that: The calculation formulas for the detection score and classification score in step 3 are as follows: In which, let I∈R h×w×3 Represents RGB image, y c ∈{0,1} is the corresponding true base class label, c∈{1,…,C} and C is the total number of object categories. Let v i , i∈{1,…,R}, where R is the number of region proposals, represents the visual proposals in image i, and w and b are the weights and biases, respectively.

3. The method for hysteroscopic image target detection based on depth information and knowledge distillation according to claim 2, characterized in that: The cosine similarity calculation formula in step 4 is as follows: In which, let I∈r h×w×3 Represents an RGB image, D∈R h×w×3 Denote the depth image associated with the RGB image I, let ψ(I) be the feature map extracted from the RGB image, ψ(D) be the feature map of the depth image D extracted by the Siamese network backbone, and ρ be a learnable temperature parameter.

4. The method for hysteroscopic image target detection based on depth information and knowledge distillation according to claim 3, characterized in that: The first component L of the contrast loss in step 4 D→I The RGB image is compared with the negative depth image to measure the degree of depth matching between the RGB image and the other images in its batch. The first component of the contrast loss is calculated as follows: Where B is a batch of RGB depth pairs, RGB image and depth image pair (I,D)∈B, S(I,D) is the cosine similarity; The second component of the contrast loss L I→D is similarly defined as comparing the depth image with the negative RGB image sample, estimating the first component of the contrast loss L D→I With the second component L I→D Average and calculate the contrast loss L NCE , the contrast loss calculation formula is as follows: Among them, L D→I is the first component of the contrast loss, L I→D is the second component of contrast loss.

5. The method for hysteroscopic image target detection based on depth information and knowledge distillation according to claim 4, characterized in that: The image visual feature score derived in step 5 and deep feature scores To perform the fusion: in, and are the fusion detection score and classification score respectively, is the detection score, is the classification score; These classification scores and detection scores are converted to probabilities, and the probability conversion formula is as follows: in, Is class c present in the current proposal f i The probability of Yes i For predicting image-level labels y c Probability of importance; Multiply the classification score and the detection score element by element to get the combined score Finally, image-level prediction The calculation formula is as follows: in, is the combined score; Then, the regional level score L is obtained by optimizing the loss mil , the calculation formula is as follows: in, is the image-level prediction, y c ∈{0,1} is the corresponding true base class label.

6. The method for hysteroscopic image target detection based on depth information and knowledge distillation according to claim 5, characterized in that: In step 6, for any new depth image D, the range dr of the object's estimated depth prior c The formula is as follows: Where C is the set of object categories, S is the set of words in the title corresponding to D, and r c,s is the depth range of C categories and word set S; Using the estimated depth prior range dr c To identify the potentially important areas in each category; for each area, define a depth mask indicator variable m i,c , which indicates the possibility that a certain area in the image contains a certain type of object. The calculation of this variable is as follows: Among them, if the proposed depth value pdi falls within the estimated depth prior range dr of class c c If the area is within , the area is considered to be a relevant area of ​​the class, and the corresponding mask variable m is i,c Set to 1 if false; otherwise, set to 0.

7. The method for hysteroscopic image target detection based on depth information and knowledge distillation according to claim 6, characterized in that: In step 8, during the training process, first calculate L task Relative to the feature The gradient of the detector layer l k feature map weight Defined as: Among them, L task represents the total detection loss, including bounding box regression loss and classification loss, is the single activation value at position (i, j) in layer k, W, H are the gradient and width of the reflow; These reflowed gradients are globally average pooled in width and height dimensions to obtain feature channel importance Then, use To measure the k-th activation map in, is the k-th gradient weighted activation of the l-th layer, is the feature channel importance; These maps are then linearly combined along the channel dimension, and the gradient of the detection loss with respect to the feature map is used to represent the contribution of the feature to the final detection before taking the absolute value and norm to obtain the distilled final target map M l , the final target mapping calculation formula is as follows: Among them, Norm represents the minimum-maximum normalization function, is the kth gradient weighted activation of the lth layer; These gradients are used to weight the features and obtain the loss function L for gradient-guided knowledge extraction. GKD ; Where W, H are the gradient and width of the reflux, L is the total number of intermediate layers considered for distillation, l represents the intermediate layer, and are the generated teacher and student target mappings, respectively; When extracting features, a Gaussian mask F is used i,j , which is defined as: Among them, o and Respectively represent the area inside the true value bounding box and the surrounding area, (x, y) represents a specific point in the surrounding area; Represents the center point of the ground-truth bounding box; When extracting features, position and channel attention are also combined, and the position attention mask M P and channel attention mask M C The calculation formula is as follows: Where A is a common feature map, W, H, and C are the width, height, and channel number of A, respectively, with i, j, and k as indexes; T is the temperature hyperparameter introduced to adjust the distribution; Based on the Gaussian mask F i,j , location attention mask M P , channel attention mask M C , we get the loss L of multi-granularity features in bounding box detection BMFI : Among them, the superscripts T and S are the teacher detector and the student detector respectively. and are the teacher and student feature maps, M P,T and M P,S are the teacher and student position attention masks, M C,T and M C,S They are the teacher and student channel attention masks, respectively; Finally, we add the loss L of the gradient-guided knowledge extraction process GKD , and the total distillation loss L is obtained KD : L KD =L GKD +βL BMFI Among them, β is the balance hyperparameter, L GKD is the loss of the gradient-guided knowledge distillation process, L BMFI It is the loss of multi-granular features in bounding box detection.

Images