Training method of target detection model, target detection method and training device

Abstract

The application discloses a kind of training method of target detection model, target detection method and training device.The training method includes: obtaining the pyramid input feature obtained by the sample image with label information according to the main network to be trained;Pyramid output feature obtained by the feature pyramid network according to the pyramid input feature to be trained, the highest layer output feature of pyramid output feature is formed according to the highest layer input feature in pyramid input feature, the lowest layer output feature of pyramid output feature is formed according to the lowest layer input feature in pyramid input feature, and other layer output features of pyramid output feature are formed according to the fusion of several layer input features;The prediction result obtained by the prediction network according to the pyramid output feature to be trained;According to the difference between prediction result and the label information of sample image, update model parameter, the training process extracts rich context information and detail information, and is conducive to improving the detection accuracy of model.

Description

Technical Field

[0001] This invention belongs to the field of image processing technology. Specifically, it relates to a training method, target detection method, training device, target detection device, computer-readable storage medium, and computer equipment for a target detection model of lesion areas in capsule endoscopy images. Background Technology

[0002] Since the introduction of small bowel capsule endoscopy into clinical practice in the United States and Europe in 2001, its use has steadily increased. Using a capsule endoscope to take images of the human body's interior, and then reviewing these images for pathological examination, reduces patient discomfort and lowers the risk of missed diagnoses. However, wireless capsule endoscopy captures a large number of images in a short period, posing a significant challenge to physicians in reviewing them. Furthermore, for some images showing small lesions, even experienced physicians may miss some lesions during manual review, which can be extremely dangerous for patients.

[0003] In recent years, deep learning has attracted great interest from researchers in the field of artificial intelligence, and many researchers have applied it to medical image processing, such as wireless capsule endoscopy (WCE) images. Using deep learning technology for early diagnosis of cancer, ulcers, polyps, bleeding, and hookworm can potentially help with prognosis and the development of effective treatment plans. Compared to traditional endoscopy, wireless capsule endoscopy (WCE) can perform low-risk, non-invasive imaging examinations of the digestive tract. WCE is a popular and routine method for examining small bowel diseases; however, the images it detects often suffer from low contrast, complex backgrounds, lesion colors, and varying color intensities, which affects the accuracy of segmentation, detection, and classification. Therefore, improvements to wireless capsule endoscopy to address these issues are necessary.

[0004] Deep learning-based object detection algorithms have been extensively studied in the past few years. With the development of artificial intelligence and the proposal of some object detection algorithms based on deep convolutional neural networks (CNNs), such as Faster R-CNN in two-stage detectors and SSD and YOLO in one-stage detectors, deep learning-based detection methods have achieved good results. Moreover, as the performance of deep convolutional neural network architectures improves, their detection performance will become better and better.

[0005] In recent years, many researchers have applied some excellent object detectors in natural scenes to the detection of lesion regions in wireless capsule endoscopy images. For example, (Younghak S., Ali QH, Lars A, et al. Automatic Colon Polyp Detection Using Region Based Deep CNN and Post Learning Approaches[J].IEEE Access,2018,6:40950-40962.) proposed an improved Faster R-CNN network for detecting polyp lesion regions in capsule endoscopy; (Zhang Xu, Chen Fei, Yu Tao, et al. Real-time gastric polyp detection using convolutional neural networks[J]. Plos One,2019,14(3).) proposed an improved DSSD network for detecting polyp lesion regions in capsule endoscopy; (Jia Xiao, MaiXiao Chun, Cui Yi Xun, et al. Automatic Polyp Recognition in Colonoscopy Images Using Deep Learning and Two-Stage Pyramidal Feature Prediction[J].IEEE Transactions on Automation Science and Engineering, 2020, PP(99):1-15.) proposed an improved Mask-R-CNN network for segmenting polyp lesions in capsule endoscopy and an improved HED network for detecting hookworm lesions in capsule endoscopy images. These deep learning-based methods achieved good detection results for lesion detection in capsule endoscopy. However, when the polyp lesion area is small, the polyp detection methods proposed in the above literature cannot detect it correctly, and due to the complex internal environment of the endoscopic image, a large number of false detections will occur. Summary of the Invention

[0006] (I) The technical problem to be solved by the present invention

[0007] The technical problem solved by this invention is: how to provide a target detection model that can accurately detect lesions in small areas of capsule endoscopy images.

[0008] (II) Technical Solution Adopted in this Invention

[0009] A training method for a target detection model in lesion regions of capsule endoscopy images, wherein the target detection model to be trained includes a backbone network, a feature pyramid network, and a prediction network, and the training method includes:

[0010] Obtain the pyramid input features of the backbone network to be trained based on the labeled sample images;

[0011] Obtain the pyramid output features of the feature pyramid network to be trained based on the pyramid input features, wherein the highest layer output feature of the pyramid output feature is formed based on the highest layer input feature of the pyramid input features, the lowest layer output feature of the pyramid output feature is formed based on the lowest layer input feature of the pyramid input features, and the other layer output features of the pyramid output feature are formed by fusing several layer input features of the pyramid input features;

[0012] Obtain the prediction results obtained by the prediction network to be trained based on the pyramid output features;

[0013] Based on the difference between the prediction results and the label information of the sample images, the model parameters of the backbone network, feature pyramid network, and prediction network are updated.

[0014] Preferably, the backbone network to be trained includes several convolutional layers and several RFB modules, wherein each convolutional layer and each RFB module is connected sequentially in a predetermined order, and the method for obtaining pyramid input features from the sample images by the backbone network to be trained includes:

[0015] The lowest convolutional layer of the backbone network obtains the lowest-level input features in the pyramid input features based on the sample image;

[0016] The highest-level RFB module of the backbone network obtains the highest-level input feature in the pyramid input feature based on the output feature of the RFB module connected to the highest-level RFB module.

[0017] In the backbone network, other RFB modules besides the highest-level RFB module output the pyramid output features of other layer input features.

[0018] Preferably, the highest-level output feature of the pyramid output feature is formed based on the highest-level input feature in the pyramid input features, specifically including: directly using the highest-level input feature in the pyramid input features as the highest-level output feature of the pyramid output feature;

[0019] The lowest-level output feature of the pyramid output feature is formed based on the lowest-level input feature of the pyramid input feature, specifically including: performing convolution processing on the lowest-level input feature of the pyramid input feature to obtain the lowest-level output feature of the pyramid output feature.

[0020] Preferably, the other layer output features of the pyramid output features are formed by fusing several layer features from the pyramid input features, specifically including:

[0021] Convolutional processing and batch normalization processing are performed on the input features of the pyramid input features other than the lowest layer input features to obtain intermediate layer features of several scales.

[0022] The features of each intermediate layer are upsampled and fused from top to bottom to obtain fused features. The fused features are then pooled to obtain the output features of other layers of the pyramid output features.

[0023] Preferably, the prediction network to be trained includes an ARM module and an output network, and the prediction result obtained by the prediction network to be trained based on the pyramid output features specifically includes:

[0024] After the pyramid output features are convolved, they are input into the ARM module and the output network, respectively. The ARM module outputs a first prediction result, and the output network outputs a second prediction result.

[0025] The second prediction result is adjusted based on the first prediction result to obtain the prediction result.

[0026] This application also discloses a target detection method for lesion regions in capsule endoscopy images, the target detection method comprising:

[0027] Acquire images of the capsule endoscope to be examined;

[0028] The capsule endoscopy image is input into a target detection model for lesion areas in capsule endoscopy images, which is trained according to the above training method, and the target detection model outputs the detection results.

[0029] This application also discloses a training device for a target detection model of lesion regions in capsule endoscopy images, the training device comprising:

[0030] The first acquisition unit is used to acquire the pyramid input features obtained by the backbone network to be trained based on the sample images with labeled information.

[0031] The second acquisition unit is used to acquire the pyramid output features obtained by the feature pyramid network to be trained based on the pyramid input features, wherein the highest layer output feature of the pyramid output feature is formed based on the highest layer input feature of the pyramid input features, the lowest layer output feature of the pyramid output feature is formed based on the lowest layer input feature of the pyramid input features, and the other layer output features of the pyramid output feature are formed by fusing several layer input features of the pyramid input features.

[0032] The third acquisition unit is used to acquire the prediction results obtained by the prediction network to be trained based on the pyramid output features;

[0033] The parameter update unit is used to update the model parameters of the backbone network, feature pyramid network, and prediction network based on the difference between the prediction result and the label information of the sample image.

[0034] Preferably, a target detection device for lesion areas in capsule endoscopy images, the target detection device comprising:

[0035] The image acquisition module is used to acquire images of the capsule endoscope to be inspected;

[0036] A target detection model is used to obtain detection results based on the input capsule endoscopy image.

[0037] The target detection model is trained using the training method described above.

[0038] This application also discloses a computer-readable storage medium storing a computer program that, when executed by a processor, implements any of the methods described above.

[0039] This application also discloses a computer device, which includes a computer-readable storage medium, a processor, and a computer program stored in the computer-readable storage medium, wherein the computer program, when executed by the processor, implements any of the methods described above.

[0040] (III) Beneficial Effects

[0041] This invention discloses a training method and a target detection method for a target detection model in lesion areas of capsule endoscopy images, which have the following technical advantages compared to existing methods:

[0042] High-level and low-level semantic features are extracted from the pyramid input features obtained from the backbone network to acquire rich contextual information. The input features from several layers are then fused to obtain fused features, which provide richer details about polyps, thereby improving the accuracy of the target detection model in identifying small polyp regions. Simultaneously, adding an RFB module to the backbone network increases the model's receptive field, and adding an ARM module to the prediction network helps address the problem of imbalanced positive and negative samples. Attached Figure Description

[0043] Figure 1 This is a flowchart of a training method for a target detection model for lesion regions in capsule endoscopy images, according to Embodiment 1 of the present invention.

[0044] Figure 2 This is a schematic diagram of a target detection model for lesion areas in capsule endoscopy images according to Embodiment 1 of the present invention;

[0045] Figure 3 This is a schematic diagram of the RFB module according to Embodiment 1 of the present invention;

[0046] Figure 4 This is a flowchart of a target detection method for lesion areas in capsule endoscopy images according to Embodiment 2 of the present invention;

[0047] Figure 5 This is a schematic diagram of a training device for a target detection model of a lesion region in a capsule endoscopy image according to Embodiment 3 of the present invention;

[0048] Figure 6 This is a schematic diagram of a target detection device for lesion areas in capsule endoscopy images according to Embodiment 4 of the present invention;

[0049] Figure 7 This is a schematic diagram of a computer device according to Embodiment 4 of the present invention. Detailed Implementation

[0050] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0051] Before describing the various embodiments of this application in detail, the inventive concept of this application is first briefly described: In the prior art, when using deep learning methods to detect lesions in capsule endoscopy, the proposed polyp target detection methods cannot detect them correctly when the polyp lesion area is small. Furthermore, due to the complex internal environment of the endoscope image, a large number of false detections will occur. To address this, the target detection model training method based on feature pyramid network provided in this application fuses some features in the pyramid input features to obtain fused features, and extracts high-level semantic features and low-level semantic features from the pyramid input features. Together with the fused features, they constitute the pyramid output features. Further, a prediction network is used for prediction. This allows the model to learn richer polyp details and contextual information, thereby improving the detection accuracy of the model.

[0052] Specifically, the target detection model for lesion regions in capsule endoscopy images in this embodiment includes three parts: a backbone network, a feature pyramid network, and a prediction network. The training method for the target detection model for lesion regions in capsule endoscopy images includes the following steps:

[0053] Step S10: Obtain the pyramid input features of the backbone network to be trained based on the sample images with labeled information;

[0054] Step S20: Obtain the pyramid output features of the feature pyramid network to be trained based on the pyramid input features. The highest layer output features of the pyramid output features are formed based on the highest layer input features of the pyramid input features, the lowest layer output features of the pyramid output features are formed based on the lowest layer input features of the pyramid input features, and the other layer output features of the pyramid output features are formed by fusing several layers of input features of the pyramid input features.

[0055] Step S30: Obtain the prediction results obtained by the prediction network to be trained based on the pyramid output features.

[0056] Step S40: Update the model parameters of the backbone network, feature pyramid network, and prediction network based on the difference between the prediction results and the label information of the sample images.

[0057] In step S10, the backbone network to be trained includes several convolutional layers and several RFB (Receptive Field Block) modules, and each convolutional layer and each RFB module is connected to the backbone network in a predetermined order. The method for obtaining the pyramid input features of the backbone network to be trained based on the sample images includes the following steps:

[0058] Step S101: The lowest convolutional layer of the backbone network obtains the lowest input features in the pyramid input features based on the sample images;

[0059] Step S102: The highest-level RFB module of the backbone network obtains the highest-level input feature in the pyramid input features based on the output features of the RFB modules connected to the highest-level RFB module;

[0060] Step S103: Other layers of input features are output by the RFB modules in the backbone network, excluding the highest-level RFB module, which output the pyramid output features.

[0061] The number and position of each convolutional layer and each RFB module can be set arbitrarily, and this embodiment does not impose any limitations. For example, the backbone network can be modified from a VGG16 network by replacing the last two fully connected layers of the VGG16 network with convolutional layers. The dimensions of these two convolutional layers are 32×32×512 and 16×16×1024, respectively, and two additional convolutional blocks are added, with the last convolutional block having a dimension of 8×8×768. To increase the receptive field of the network while maintaining a relatively constant depth, three RFB modules are added to the VGG16 network, such as... Figure 2 The diagram shows the addition of RFB modules after the convolutional features of conv5_3 and conv7, respectively, and the replacement of the first added convolutional block with an RFB module. The RFB module is shown below. Figure 3 As shown, its input is the output of the previous layer. Three dilated convolutions with different eccentricities (1, 3, and 5) are performed on the input feature blocks. Finally, the feature blocks from the three branches are fused with the original feature block as the input to the next layer. The addition of the RFB module allows the backbone network to achieve a large receptive field without requiring a very deep network, thus improving network performance. Figure 3 The RFB module shown adds batch normalization after each convolutional block in each branch, which enables the network to converge well when training on the polyp dataset from scratch, and the model is more robust.

[0062] In step S20, when constructing the pyramid output features, the highest-level input feature in the pyramid input features is directly used as the highest-level output feature of the pyramid output features. The lowest-level input feature in the pyramid input features is convolved to obtain the lowest-level output feature of the pyramid output features. Convolutional processing is then performed on the other input features in the pyramid input features (excluding the lowest-level feature) to obtain intermediate-level features at several scales. These intermediate-level features are then upsampled and fused from top to bottom to obtain the final fused feature. The fused feature is then pooled to obtain the other output features of the pyramid output features. This approach preserves both high-level and low-level semantic features, enriching them with more contextual information, and also obtains richer polyp detail information through fusion processing.

[0063] For example, for ease of description, both the pyramid input features and pyramid output features are described using a four-layer model. The lowest layer input feature of the pyramid input features is the convolutional feature conv4_3 obtained by processing the convolutional feature conv4_3 convolutional block. The middle two layers input features are the convolutional feature conv7 obtained by processing the convolutional feature conv5_3 output by the first RFB module and the convolutional feature conv7 obtained by processing the convolutional feature conv7 by the second RFB module. The highest layer input feature is conv9 obtained by processing the output of the second RFB module by the third RFB module.

[0064] Furthermore, two 3×3 convolution operations are performed on each layer of the pyramid input features to achieve dimensionality reduction. To ensure good convergence of the network when training from scratch, batch normalization is performed after the convolution operations. Specifically, two 3×3 convolution operations are performed on the convolutional feature conv4_3 to achieve dimensionality reduction. To ensure good convergence of the network when training from scratch, batch normalization is performed on the convolutional feature conv4_3 after convolution processing to form the lowest layer output features of the pyramid output features. After performing two 3×3 convolution operations and batch normalization on the convolutional features output by the first RFB module, the second RFB module, and the third RFB module, three intermediate layer features at three scales are obtained. Each intermediate layer feature is upsampled and fused from top to bottom to obtain fused features. Pooling is then performed on the fused features to obtain the two middle layers of the pyramid output features, i.e., the other layers of the pyramid output features. In step S20, to broaden the network and improve its performance, the processing approach of the SPP (Spatial Pyramid Pooling) module of the PFPNet network is adopted. However, the difference lies in the fact that directly performing pooling operations of different sizes on the input polyp feature map might lead to the loss of detailed features after pooling due to the small effective area of ​​the polyp, and fusion could result in the loss of extracted high-level semantic information of the polyp. In this second embodiment, the processing of step S20 is used to construct a pyramid output feature with four layers, which effectively preserves richer polyp details and extracts richer contextual details. At this point, the dimensions of the four output feature layers are 64×64×768, 32×32×768, 16×16×768, and 8×8×768, respectively.

[0065] Further, in step S30, the prediction network to be trained includes an ARM module and an output network. The prediction result obtained by the prediction network based on the pyramid output features specifically includes: performing convolution processing on the pyramid output features, and then inputting them into the ARM module and the output network respectively. The ARM module outputs a first prediction result, and the output network outputs a second prediction result. The second prediction result is adjusted based on the first prediction result to obtain the final prediction result. Adding an ARM module to the model can solve the problem of imbalanced positive and negative samples in one-stage networks. Specifically, the input anchor points are binary classified (foreground and background), and the anchor points are roughly adjusted in position and size (where the anchor points correspond to the rectangular boxes drawn on the original image, and box regression is performed), and some negative samples are filtered out. The anchor points adjusted by the ARM module (i.e., the first prediction result) are used to initialize the anchor points of the output network (i.e., the second prediction result) to obtain the final prediction result.

[0066] The ARM module: After each convolution, an output is appended for classification and bounding box regression. The classification here is binary, used to distinguish foreground from background. Anchor points with higher scores are fed into subsequent networks for training. To ensure good convergence when training from scratch, batch normalization is added to the head network of each output branch. Its classification loss, regression loss, and output loss functions are the same.

[0067] The output network employs an SSD (Single Shot MultiBox Detector) network. Each branch's input consists of anchor points with high ARM module scores, which are then processed through a 3×3 convolution (256 channels) before being connected to the network's output. Each output includes two tasks: classification and bounding box regression. The classification task is for a specific class. The output classification loss function is as follows:

[0068]

[0069] in for:

[0070]

[0071] Where L conf (x,c) represents the output value of the classification loss function, i represents the predicted box number, j represents the ground truth box number, p is the class number, and 0 in the second term means p=0, indicating background. This represents the probability that the i-th predicted box belongs to category p. This indicates that the i-th predicted bounding box matches the j-th ground truth bounding box, and the category of the ground truth bounding box is p. The first part of the formula is the loss for positive samples, and the second part is the loss for negative samples.

[0072] The regression loss function for the bounding box is:

[0073]

[0074] Among them, smooth L1 The loss function is:

[0075]

[0076] Where L loc (x,l,g) represents the output of the regression loss function for the bounding box. Represents the coordinates of the encoded prediction box. The coordinates of the encoded truth box can be represented as: where d represents the center coordinates, length, and width of the prior box. The encoding of the prediction box is similar.

[0077]

[0078]

[0079]

[0080]

[0081] Its total output loss function is:

[0082]

[0083] Where N represents the number of ground truth boxes matched, and α is the scaling factor between classification loss and box regression loss, which defaults to 1.

[0084] In step S40, after obtaining the prediction result, the loss function value is calculated according to the above loss function formula, and the model parameters of the backbone network, feature pyramid network and prediction network are further updated according to the loss function value. The prediction result is the probability in the above loss function formula. The specific calculation process and model parameter update process are well known to those skilled in the art and will not be described in detail here.

[0085] The training method disclosed in Embodiment 1 extracts high-level and low-level semantic features from the pyramid input features obtained from the backbone network to acquire rich contextual information. This helps avoid a large number of false detections caused by the complex internal environment of endoscopic images. Furthermore, the input features from several layers are fused to obtain fused features, which can yield richer details about polyps, thereby further improving the accuracy of the target detection model in identifying small polyp regions. Simultaneously, adding an RFB module to the backbone network increases the model's receptive field, and adding an ARM module to the prediction network helps address the problem of imbalanced positive and negative samples.

[0086] like Figure 4 As shown in the second embodiment, this method provides a target detection method for lesion areas in capsule endoscopy images. The target detection method includes:

[0087] Step S100: Obtain the capsule endoscope image to be detected; Step S200: Input the capsule endoscope image into the target detection model for the lesion area of ​​the capsule endoscope image trained according to the training method of Example 1, and the target detection model outputs the detection result.

[0088] Specifically, after acquiring the capsule endoscope image to be detected, preprocessing is performed to enhance image quality, mainly including removing black borders, dark areas, and reflective areas. The trained backbone network processes the capsule endoscope image to obtain corresponding pyramid input features. The trained feature pyramid network obtains pyramid output features based on the pyramid input features, and the trained prediction network obtains the prediction result based on the pyramid output features, thus obtaining the detection result. The data processing processes of the backbone network, feature pyramid network, and prediction network are described in Example 1 and will not be repeated here.

[0089] like Figure 5 As shown in the figure, this embodiment three discloses a training device for a target detection model for lesion areas in capsule endoscopy images. The training device includes a first acquisition unit 10, a second acquisition unit 20, a third acquisition unit 30, and a parameter update unit 40.

[0090] The system comprises the following components: a first acquisition unit 10, which acquires the pyramid input features obtained by the backbone network to be trained based on the labeled sample images; a second acquisition unit 20, which acquires the pyramid output features obtained by the feature pyramid network to be trained based on the pyramid input features, wherein the highest layer output features of the pyramid output features are formed based on the highest layer input features of the pyramid input features, the lowest layer output features of the pyramid output features are formed based on the lowest layer input features of the pyramid input features, and the other layer output features of the pyramid output features are formed by fusing several layer input features of the pyramid input features; a third acquisition unit 30, which acquires the prediction results obtained by the prediction network to be trained based on the pyramid output features; and a parameter update unit 40, which updates the model parameters of the backbone network, the feature pyramid network, and the prediction network based on the difference between the prediction results and the label information of the sample images.

[0091] Furthermore, referring to the description in Embodiment 1, the structure of the backbone network is further configured to acquire the lowest layer input features of the lowest convolutional layer of the backbone network obtained from the sample image, the highest layer input features of the highest layer RFB module of the backbone network obtained from the output features of the RFB module connected to the highest layer RFB module, and other layer input features of the pyramid output features of other RFB modules in the backbone network other than the highest layer RFB module.

[0092] Furthermore, the second acquisition unit 20 is also used to acquire the highest layer input features in the pyramid input features, the lowest layer output features obtained after convolution processing the lowest layer input features in the pyramid input features, and other layer output features.

[0093] The formation process of other layer output features is as follows: Convolution and batch normalization are performed on the input features of the pyramid input features other than the lowest layer input features to obtain intermediate layer features of several scales; each intermediate layer feature is upsampled and fused from top to bottom to obtain fused features; pooling is performed on the fused features to obtain the other layer output features of the pyramid output features.

[0094] Furthermore, the third acquisition unit 30 is also used to acquire the first prediction result obtained by the ARM module based on the pyramid output features, the second prediction result obtained by the output network based on the pyramid output features, and the prediction result obtained after adjusting the second prediction result based on the first prediction result.

[0095] like Figure 6 As shown, this embodiment four also discloses a target detection device for lesion areas in capsule endoscopy images. The target detection device includes an image acquisition module 50 and a target detection model 60. The image acquisition module 50 is used to acquire the capsule endoscopy image to be detected; the target detection model 60 is used to obtain the detection result based on the input capsule endoscopy image. The image acquisition module 50 can be a capsule endoscopy device, and the target detection model 60 is trained according to the training method of embodiment one.

[0096] This fourth embodiment also discloses a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, it implements the training method for the target detection model of the lesion area in capsule endoscopy images of Embodiment 1 or the target detection method for the lesion area in capsule endoscopy images of Embodiment 2.

[0097] This fifth embodiment also discloses a computer device, at the hardware level, such as... Figure 7As shown, the computer device includes a processor 12, an internal bus 13, a network interface 14, and a computer-readable storage medium 11. The processor 12 reads the corresponding computer program from the computer-readable storage medium and runs it, forming a request processing device at the logical level. Of course, in addition to the software implementation, one or more embodiments of this specification do not exclude other implementation methods, such as logic devices or a combination of hardware and software, etc. That is to say, the execution subject of the following processing flow is not limited to each logic unit, but can also be hardware or logic devices. The computer-readable storage medium 11 stores a computer program, which, when executed by the processor, implements the training method for the target detection model of the lesion area in capsule endoscopy images of Embodiment 1 or the target detection method for the lesion area in capsule endoscopy images of Embodiment 2.

[0098] Computer-readable storage media include both permanent and non-permanent, removable and non-removable media that can store information by any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer-readable storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, disk storage, quantum memory, graphene-based storage media or other magnetic storage devices, or any other non-transfer medium that can be used to store information accessible by a computing device.

[0099] The specific embodiments of the present invention have been described in detail above. Although some embodiments have been shown and described, those skilled in the art should understand that modifications and improvements can be made to these embodiments without departing from the principles and spirit of the present invention as defined by the claims and their equivalents, and such modifications and improvements should also be within the protection scope of the present invention.

Claims

1. A training method for a target detection model in lesion regions of capsule endoscopy images, characterized in that, The target detection model to be trained includes a backbone network, a feature pyramid network, and a prediction network. The training method includes: Obtain the pyramid input features of the backbone network to be trained based on the labeled sample images; Obtain the pyramid output features of the feature pyramid network to be trained based on the pyramid input features, wherein the highest layer output feature of the pyramid output feature is formed based on the highest layer input feature of the pyramid input features, the lowest layer output feature of the pyramid output feature is formed based on the lowest layer input feature of the pyramid input features, and the other layer output features of the pyramid output feature are formed by fusing several layer input features of the pyramid input features; Obtain the prediction results obtained by the prediction network to be trained based on the pyramid output features; Based on the difference between the prediction results and the label information of the sample images, update the model parameters of the backbone network, feature pyramid network, and prediction network; The highest-level output feature of the pyramid output feature is formed based on the highest-level input feature in the pyramid input feature, specifically including: directly using the highest-level input feature in the pyramid input feature as the highest-level output feature of the pyramid output feature; The lowest-level output feature of the pyramid output feature is formed based on the lowest-level input feature of the pyramid input feature, specifically including: performing convolution processing on the lowest-level input feature of the pyramid input feature to obtain the lowest-level output feature of the pyramid output feature; The other layer output features of the pyramid output features are formed by fusing several layer features from the pyramid input features, specifically including: Convolutional processing and batch normalization processing are performed on the input features of the pyramid input features other than the lowest layer input features to obtain intermediate layer features of several scales. The features of each intermediate layer are upsampled and fused from top to bottom to obtain fused features. The fused features are then pooled to obtain the output features of other layers of the pyramid output features.

2. The training method for a target detection model for lesion regions in capsule endoscopy images according to claim 1, characterized in that, The backbone network to be trained includes several convolutional layers and several RFB modules, wherein each convolutional layer and each RFB module is connected sequentially in a predetermined order. The method for obtaining pyramid input features from the sample images by the backbone network to be trained includes: The lowest convolutional layer of the backbone network obtains the lowest-level input features in the pyramid input features based on the sample image; The highest-level RFB module of the backbone network obtains the highest-level input feature in the pyramid input feature based on the output feature of the RFB module connected to the highest-level RFB module. In the backbone network, other RFB modules besides the highest-level RFB module output the pyramid output features of other layer input features.

3. The training method for a target detection model for lesion regions in capsule endoscopy images according to claim 1, characterized in that, The prediction network to be trained includes an ARM module and an output network. The prediction result obtained by the prediction network to be trained based on the pyramid output features specifically includes: After the pyramid output features are convolved, they are input into the ARM module and the output network, respectively. The ARM module outputs a first prediction result, and the output network outputs a second prediction result. The second prediction result is adjusted based on the first prediction result to obtain the prediction result.

4. A method for target detection in lesion areas of capsule endoscopy images, characterized in that, The target detection method includes: Acquire images of the capsule endoscope to be examined; The capsule endoscopy image is input into the target detection model for the lesion region of the capsule endoscopy image obtained by training according to any one of claims 1 to 3, and the target detection model outputs the detection result.

5. A training device for a target detection model of lesion regions in capsule endoscopy images, characterized in that, The training device includes: The first acquisition unit is used to acquire the pyramid input features obtained by the backbone network to be trained based on the sample images with labeled information; The second acquisition unit is used to acquire the pyramid output features obtained by the feature pyramid network to be trained based on the pyramid input features, wherein the highest layer output feature of the pyramid output feature is formed based on the highest layer input feature of the pyramid input features, the lowest layer output feature of the pyramid output feature is formed based on the lowest layer input feature of the pyramid input features, and the other layer output features of the pyramid output feature are formed by fusing several layer input features of the pyramid input features. The third acquisition unit is used to acquire the prediction results obtained by the prediction network to be trained based on the pyramid output features; The parameter update unit is used to update the model parameters of the backbone network, feature pyramid network, and prediction network based on the difference between the prediction result and the label information of the sample image. The highest-level output feature of the pyramid output feature is formed based on the highest-level input feature in the pyramid input feature, specifically including: directly using the highest-level input feature in the pyramid input feature as the highest-level output feature of the pyramid output feature; The lowest-level output feature of the pyramid output feature is formed based on the lowest-level input feature of the pyramid input feature, specifically including: performing convolution processing on the lowest-level input feature of the pyramid input feature to obtain the lowest-level output feature of the pyramid output feature; The other layer output features of the pyramid output features are formed by fusing several layer features from the pyramid input features, specifically including: Convolutional processing and batch normalization processing are performed on the input features of the pyramid input features other than the lowest layer input features to obtain intermediate layer features of several scales. The features of each intermediate layer are upsampled and fused from top to bottom to obtain fused features. The fused features are then pooled to obtain the output features of other layers of the pyramid output features.

6. A target detection device for lesion areas in capsule endoscopy images, characterized in that, The target detection device includes: The image acquisition module is used to acquire images of the capsule endoscope to be inspected; A target detection model is used to obtain detection results based on the input capsule endoscopy image. The target detection model is trained using the training method according to any one of claims 1 to 3.

7. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the method according to any one of claims 1 to 3.

8. A computer device, characterized in that, The computer device includes a computer-readable storage medium, a processor, and a computer program stored in the computer-readable storage medium, which, when executed by the processor, implements the method according to any one of claims 1 to 3.

Images