Capsule endoscope image auxiliary diagnosis method based on deep learning

Abstract

The invention relates to the technical field of medical detection, and discloses a capsule endoscope image auxiliary diagnosis method based on deep learning, which comprises the following steps: acquiring an original image sequence of an alimentary canal shot by a wireless capsule endoscope, screening out an image sequence related to the alimentary canal in the original image sequence, and labeling the image sequence; sequentially performing data enhancement, geometric transformation and standardization processing on the marked image sequence, and clustering a prior anchor frame by using a K-means algorithm to form a capsule endoscope image data set; a deep learning model LD-YOLO is built, a YOLOv5 model is selected as a basis, optimization is carried out on the basis of the basic model, and a capsule endoscope image auxiliary diagnosis model is built; and training a capsule endoscope image auxiliary diagnosis model through the capsule endoscope image data set, and detecting an image to be diagnosed by using the trained capsule endoscope image auxiliary diagnosis model. According to the invention, the workload of doctors in image analysis can be greatly reduced, and the film reading efficiency is improved.

Description

Technical Field

[0001] The present application relates to the technical field of medical detection, and in particular to a capsule endoscope image-assisted diagnosis method based on deep learning. Background Art

[0002] Digestive tract diseases are common and frequent health problems worldwide, including but not limited to gastritis, gastric ulcer, colitis, Crohn's disease, gastrointestinal bleeding, and intestinal tumors. These diseases not only have a high incidence rate, but also have complex and diverse symptoms, which have a serious impact on the health and quality of life of patients. With the intensification of the aging trend of the population and changes in lifestyle, the incidence of digestive tract diseases has shown an upward trend, which has seriously affected people's quality of life and health level. Therefore, it is particularly important to accurately diagnose and effectively treat digestive tract diseases.

[0003] Medical endoscopes play an important role in diagnosing gastrointestinal diseases. Traditional endoscopic examinations usually require doctors to insert a long cable into the patient's body to image key areas and provide diagnostic and treatment options. However, this wired endoscope may cause discomfort and pain to the patient due to its invasive nature. In addition, the use of such endoscopes also poses the risk of potential complications, such as infection, perforation, and tearing. Capsule Endoscopy (CE) is an innovative medical device used in the field of modern gastroenterology diagnosis and treatment. It is usually composed of a miniature camera, a housing, a lighting system, a control circuit, a wireless communication module, and a battery. It examines the digestive tract in a painless, non-invasive manner. However, in the process of implementing this application, the applicant found that the current capsule endoscopy detection method has at least the following technical problems: 1. During each test, the wireless capsule endoscope will capture more than 60,000 images. The number of images collected by the capsule endoscope is huge, and manual testing is time-consuming and laborious, which increases the workload of clinicians; 2. Due to problems such as uneven illumination and blurred focus in capsule endoscopy images, feature extraction is difficult. Existing detection methods are difficult to accurately identify irregular lesions under complex backgrounds and lack dedicated detection. 3. Existing models show insufficient feature capture capabilities when detecting tiny lesions, which increases the risk of missed and misdiagnosed lesions, especially in early diagnosis, where critical lesions may be missed. 4. The deep learning models used in the existing technology have insufficient generalization capabilities in specific scenarios and are difficult to cope with various types of gastrointestinal images. In addition, although recent studies have proposed some improvement plans, such as enriching the diversity of data sets through data augmentation technology, or extracting the fractal dimension of images by introducing differential box counting methods to improve detection performance, these methods still fail to effectively solve the problems of insufficient feature extraction and limited detection of small lesions in capsule endoscopy images. In addition, these studies also have deficiencies in data set balance and model generalization capabilities, and cannot fully meet clinical needs.

[0004] Therefore, the above problems lead to low efficiency and low detection accuracy of capsule endoscopy and the risk of missed diagnosis and misdiagnosis, thus limiting the effective application of capsule endoscopy in clinical diagnosis. Summary of the invention

[0005] The embodiment of the present application provides a capsule endoscope image-assisted diagnosis method based on deep learning, which solves the current problems of low detection efficiency, low detection accuracy and the risk of missed diagnosis and misdiagnosis of capsule endoscopes.

[0006] In order to solve the above technical problems, this application is implemented as follows: A capsule endoscope image-assisted diagnosis method based on deep learning is provided, which comprises the following steps: collecting an original image sequence of the digestive tract taken by a wireless capsule endoscope, screening out an image sequence related to the digestive tract in the original image sequence, and annotating the image sequence, wherein the original image sequence records all the imaging data of the subject from swallowing the capsule until the capsule is naturally excreted from the body; performing data enhancement, geometric transformation and standardization processing on the annotated image sequence in turn, and clustering prior anchor frames using the K-means algorithm to form a capsule endoscope image data set; building a deep learning model LD-YOLO, selecting the YOLOv5 model as the basis, optimizing the basic model, and constructing a capsule endoscope image-assisted diagnosis model; training the capsule endoscope image-assisted diagnosis model using the capsule endoscope image data set, and using the trained capsule endoscope image-assisted diagnosis model to detect images to be diagnosed.

[0007] In one embodiment, the image sequences cover different lesion types, locations, and diverse patient samples.

[0008] In one embodiment, the steps of sequentially performing data enhancement, geometric transformation and standardization on the annotated image sequence include: taking the image sequence as input, inputting it into the image preprocessing stage, amplifying the lesion image in the image sequence by data enhancement technology, and processing the annotation accordingly to ensure that the enhanced image is consistent with its annotation, wherein the data enhancement technology includes at least one of affine transformation and color space transformation; making the lesion image set after data enhancement meet the input requirements of the deep learning model and improving the efficiency of feature extraction and target detection, geometrically transforming the amplified data and adjusting the image size; taking the lesion image set after geometric transformation as input, performing mean normalization on the image, and calculating the mean of a single image data set. and standard deviation , and by the formula: , perform standardization processing on the image, where is the value to be standardized, is the standardized value.

[0009] In one of the embodiments, the step of clustering prior anchor boxes using the K-means algorithm includes: using a statistically based K-means clustering algorithm to construct a lesion image set of bounding box size, analyzing and clustering the bounding box information of the lesion area in the lesion image set by iteratively updating the clustering center, and generating prior anchor boxes of different scales.

[0010] In one of the embodiments, the selected YOLOv5 model has an input end, a feature extraction backbone network, a feature fusion neck network and a classification and positioning detection head network. The input end receives a capsule endoscopy image data set, the feature extraction backbone network extracts image features, the feature fusion neck network is used to fuse features of different levels, and the classification and positioning detection head network is responsible for predicting the location and category of the target lesion. When optimizing the YOLOv5 model: in the feature extraction backbone network, a deformable convolution module is used to reconstruct the feature extraction module, the deformable convolution module is composed of DCNv4, batch normalization and SiLU activation function, and a CA attention mechanism is introduced to enhance the ability to extract lesions; in the feature fusion neck network, a Gather-and-Distribute mechanism is introduced to fuse and distribute features of different levels to enrich feature expression; in the classification and positioning detection head network, multiple features are used to detect lesions, and a detection head for tiny lesions is added to enhance the ability to detect complex lesions; in the loss function, the loss function includes a category loss function and a bounding box loss function, the classification loss is calculated by Focal Loss to adjust the sample balance, and the bounding box loss is calculated by the CIOU loss function.

[0011] In one embodiment, the step of optimizing the feature extraction backbone network includes: grouping the capsule endoscopy image data set, and calculating the offset and weight of each convolution kernel position for each group, and combining and outputting the multiplication results of the convolution kernels according to the calculation results. The deformable convolution allows the sampling position of the convolution kernel on the input feature map to be offset, and the offset of each sampling position is realized by an additional convolution branch. At the same time, the deformable convolution not only includes the offset, but also provides additional weight information. The weight is used to control the influence of the offset. The specific formula is as follows: , where x and y represent the grouped input and output respectively. is the weight of the kth sampling point, is the sampling location The CA attention mechanism is added to the tail of the backbone network structure. The CA attention mechanism aggregates features in the height and width spatial dimensions through two one-dimensional average pooling, calculates the attention in two directions, and enhances the model's positioning of the lesion of interest.

[0012] In one of the embodiments, in the optimized feature fusion neck network, the Gather-and-Distribute mechanism mainly includes two parts: Gather collection and Distribute allocation. The Gather collection part collects features of different levels through convolution operations and Transformer self-attention, and the Distribute allocation part separates the fused global features and injects them into different branches through attention operations. The specific steps include: first, features of different levels are collected and aligned through a feature alignment module, and the aligned features are fused through an information fusion module to obtain global information; the information distribution module is used to distribute the fused information to different levels using attention, and after passing through the neck feature fusion network, a multi-scale feature map of the lesion image is obtained, and these feature maps are used for subsequent lesion detection.

[0013] In one of the embodiments, in the optimized classification and positioning detection head network, a detection head with a size of 160x160 pixels is added to the YOLOv5 model. The detection head with a size of 160x160 pixels is used to capture subtle feature information in the digestive tract. The original classification and positioning detection head of the YOLOv5 model includes three output modules of different scales. These output modules divide the feature map into several grids and predict it according to the center of the lesion. The sizes of the feature maps output by these output modules are 80x80, 40x40 and 20x20 respectively. Feature maps of different scales are used to detect lesions of different sizes, among which the feature map with a size of 80x80 is used to identify smaller target lesions, and the feature map with a size of 20x20 is used to identify larger target lesions. The predicted results are compared with the true values ​​to calculate the loss.

[0014] In one embodiment, the step of calculating the classification loss by using Focal Loss to adjust the sample balance includes: the main principle of Focal Loss is to give a higher weight to the difficult-to-classify samples and a lower weight to the easy-to-classify samples on the basis of the cross entropy loss, thereby reducing the contribution of the easy-to-classify samples to the total loss and guiding the model to pay more attention to the difficult-to-classify samples. The calculation formula is as follows: ,in Represents the probability that the model predicts a positive sample, Represents the balance coefficient, which is used to adjust the balance between positive and negative categories.

[0015] In one embodiment, the step of calculating the bounding box loss by the CIOU loss function includes: using the CIOU loss function to calculate the bounding box loss, the loss function simultaneously considers the overlapping area between the predicted box and the true box, the distance between the center point coordinates and the aspect ratio of the bounding box for calculation, and the core of CIOU is to optimize the positioning accuracy of the model in lesion detection by minimizing the difference between the predicted box and the true box. On the basis of IOU, indicators such as center point distance and aspect ratio consistency are added. The calculation formula is as follows: in It is a parameter used to measure the predicted box and the bounding box. is a parameter used to measure the consistency of aspect ratio. .

[0016] In an embodiment of the present application, cavity ring-down technology is used to detect atomic spin precession signals. The size of the external magnetic field is measured by measuring the ring-down time of light absorbed by alkali metal atoms in the ring-down cavity. The ring-down time measured by this method is only related to the reflectivity of the cavity mirror and the absorption of alkali metal atoms in the cavity, and has nothing to do with the size of the incident light intensity. Therefore, the measurement result is not affected by photon fluctuations, which can effectively improve the sensitivity limit of magnetic field measurement.

[0017] In the embodiment of the present application, by building a deep learning model LD-YOLO, selecting the YOLOv5 model as the basis, optimizing the basic model, and then automatically analyzing and processing a large number of capsule endoscope image data sets, the suspicious lesion area in the image can be effectively detected. In this way, the improved model LD-YOLO assists doctors in endoscopic image detection, solves the problems of inaccurate position of the lesion detection model, insufficient feature extraction ability, and limited ability to detect small lesions, and has good detection accuracy and efficiency. Compared with the traditional film reading method, the capsule endoscope image-assisted diagnosis method based on deep learning in this application can greatly reduce the workload of doctors in image analysis, improve the efficiency of film reading, overcome the detection difficulties caused by image quality limitations, and have good generalization performance, which provides strong technical support for the accurate diagnosis and treatment of digestive tract diseases. This application has a significant effect on improving the efficiency of doctors' film reading and accurately detecting lesion areas, while reducing the risk of missed diagnosis and misdiagnosis, and improving the diagnosis and treatment efficiency of digestive tract diseases. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings: Figure 1 is a schematic diagram of a capsule endoscope image-assisted diagnosis method based on deep learning of the present application; Figure 2 It is a flowchart of the steps of sequentially performing data enhancement, geometric transformation and standardization processing on the annotated image sequence of the present application; Figure 3 It is a schematic diagram of deformable convolution of this application; Figure 4 It is the CA attention mechanism of this application; Figure 5 is a schematic diagram of the capsule endoscope image-assisted diagnosis method based on deep learning of the present application; Figure 6 This is a flowchart of an embodiment of the present application that implements a deep learning-based capsule endoscope image-assisted diagnosis method based on the PyTorch deep learning environment. DETAILED DESCRIPTION

[0019] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0020] See also Figure 1 , which is a schematic diagram of the capsule endoscope image-assisted diagnosis method based on deep learning of the present application; as shown in the figure, the capsule endoscope image-assisted diagnosis method based on deep learning includes the following steps S1 to S4. Among them: In step S1, the original image sequence of the digestive tract taken by the wireless capsule endoscope is collected, the image sequence related to the digestive tract in the original image sequence is screened out, and the image sequence is annotated. The original image sequence records all the image data of the subject from swallowing the capsule until it is naturally excreted from the body. In this embodiment, the image sequence covers different lesion types, locations and diverse patient samples.

[0021] Specifically, in the image acquisition stage, the original image sequences of the digestive tract are first captured by a wireless capsule endoscope. These sequences record all the imaging data from the time the subject swallows the capsule until it is naturally excreted from the body. Subsequently, these original image sequences are preliminarily screened, and only image sequences related to the digestive tract are retained. The acquired image sequences should cover different lesion types, locations, and diverse patient samples to ensure the comprehensiveness and representativeness of the dataset. Finally, the image sequences are annotated with categories and bounding boxes, such as polyps, ulcers, and bleeding. The acquired data and public datasets are used together to train diagnostic models to enhance the detection capabilities of various diseases.

[0022] In step S2, the annotated image sequence is subjected to data enhancement, geometric transformation and standardization in turn, and the K-means algorithm is used to cluster the prior anchor frames to form a capsule endoscopy image dataset. Step S2 is the preprocessing stage, in which the original images in the training dataset are preprocessed. The purpose of the preprocessing is to enhance the image sequence quality of the input model and control the noise. The preprocessing steps include data enhancement, geometric transformation and standardization. Data enhancement solves the class imbalance problem existing in the original dataset, increases the diversity of training data through random cropping, rotation and flipping operations, and improves the robustness of the model. Geometric transformation adjusts the size and scale of the input image to meet the input requirements of the model. The input data is standardized, and the input features are scaled to a normal distribution with a mean of 0 and a variance of 1 through standardization. Finally, the prior anchor frame of the capsule endoscopy image sample dataset in S100 is obtained using the statistical K-means clustering method.

[0023] In addition to the above, please refer to Figure 2, which is a flowchart of the steps of sequentially performing data enhancement, geometric transformation and standardization on the annotated image sequence of the present application; as shown in the figure, the steps of sequentially performing data enhancement, geometric transformation and standardization on the annotated image sequence include steps S21 to S23. First, in step S21, the image sequence is taken as input and input into the image preprocessing stage, and the lesion image in the image sequence is amplified by the data enhancement technology, and the annotation is processed accordingly to ensure that the enhanced image is consistent with its annotation, wherein the data enhancement technology includes at least one of affine transformation and color space transformation. Specifically, the capsule endoscope image sequence acquired by step S1 is taken as input and input into the image preprocessing stage. Since the abnormal image data accounts for a small proportion, in order to increase the robustness of the model and improve the detection ability of different types of lesions, the lesion image is amplified by the data enhancement technology, and the enhancement technology includes but is not limited to: affine transformation (random rotation, folding, scaling, cropping, etc.) and color space transformation (brightness adjustment, contrast enhancement, adding noise, etc.), and the label is processed accordingly to ensure that the enhanced image is consistent with its label.

[0024] Next, in step S22, the lesion image set after data enhancement is made to meet the input requirements of the deep learning model, and the efficiency of feature extraction and target detection is improved, the amplified data is geometrically transformed, and the image size is adjusted. Specifically, the lesion image set after data enhancement in step S21 is made to meet the input requirements of the deep learning model, and the efficiency of feature extraction and target detection is improved, the amplified data is geometrically transformed, and the image size is adjusted, mainly including: cropping and padding. For images that exceed the input requirements, the irrelevant areas at the edges are cropped to ensure that the core content can adapt to the model input. For images that are less than the input requirements, the size is expanded by zero-pixel padding to meet the input requirements.

[0025] Finally, in step S23, the lesion image set after geometric transformation is used as input, the image is subjected to mean normalization processing, and the mean of the single image data set is calculated. and standard deviation , and by the formula: , perform image standardization processing to make the distribution of image data more consistent with the standard normal distribution, reduce the deviation of the model during training, and improve the convergence speed and generalization ability of the model. is the value to be standardized, is the standardized value. By standardizing the lesion image set, the differences between different images can be eliminated, making them consistent in the input of the deep learning model, thereby improving the stability of the model and the detection accuracy.

[0026] In this embodiment, the step of clustering the prior anchor frame using the K-means algorithm includes: using the statistically based K-means clustering algorithm to construct a lesion image set of bounding box size, by iteratively updating the cluster center, analyzing and clustering the bounding box information of the lesion area in the lesion image set, and generating prior anchor frames of different scales. The target is divided into three categories according to the pixel size, and the pixel size less than 32x32 is defined as a small target object, the pixel size greater than 32x32 and less than 96x96 is defined as a medium target object, and the pixel size greater than 96x96 is defined as a large target object. By analyzing and clustering the bounding box information of the lesion area in the training data set (lesion image set), a priori anchor frames of different scales are generated to optimize the performance of the detection model. Among them, the K-means algorithm first randomly selects K bounding boxes as the initial cluster center, calculates the distance between the bounding box sample and the cluster center, assigns the sample to the nearest cluster center, recalculates the assigned cluster center, and updates the cluster center according to the average size of all bounding boxes assigned to the center, and continuously iterates and optimizes until the cluster center no longer changes significantly.

[0027] In step S3, a deep learning model LD-YOLO is built, and the YOLOv5 model is selected as the basis. The basic model is optimized to construct a capsule endoscopy image-assisted diagnosis model. In the model design stage, the capsule endoscopy image-assisted diagnosis model LD-YOLO is constructed based on the YOLOv5 model. The YOLOv5 model has an input end, a feature extraction backbone network, a feature fusion neck network, and a classification and positioning detection head network. The input end accepts a capsule endoscopy image dataset. The feature extraction backbone network extracts image features, the feature fusion neck network is used to fuse features at different levels, and the classification and positioning detection head network is responsible for predicting the location and category of the target lesion. Then, when optimizing the YOLOv5 model, the feature extraction module is reconstructed using deformable convolution in the backbone network, and the attention mechanism is introduced to enhance the model's ability to understand complex and diverse digestive tract lesion images. The feature pyramid structure is replaced by the Gather-and-Distribute mechanism to fuse features at different levels. A detection head for subtle lesions is added to improve the detection accuracy of auxiliary diagnosis. To address the problem of class imbalance in the data set, Focal Loss is introduced to adjust the weights of positive and negative samples and difficult and easy samples to improve generalization ability, and CIOU bounding box loss is used to optimize bounding box regression.

[0028] Specifically, the basic model is first built. The most classic single-stage target detection model YOLOv5 is used as the basis of the LD-YOLO model. YOLOv5 is one of the representative models of the One-Stage detector. It directly predicts the location and category of the target lesion from the capsule endoscopy image dataset processed in step S2 through grid division, and has a faster speed. The YOLOv5 model has an input end, a feature extraction backbone network, a feature fusion neck network, and a classification and positioning detection head network. The input end accepts the capsule endoscopy image dataset. The feature extraction backbone network extracts image features, the feature fusion neck network is used to fuse features at different levels, and the classification and positioning detection head network is responsible for predicting the location and category of the target lesion. Then, when optimizing the YOLOv5 model, the deformable convolution is used in the backbone network to reconstruct the feature extraction module, and the attention mechanism is introduced to enhance the model's ability to understand complex and diverse digestive tract lesion images.

[0029] Secondly, optimize the YOLOv5 model: (1) In the feature extraction backbone network, please refer to Figure 3 and Figure 4 , Figure 3 is a schematic diagram of the deformable convolution of this application, Figure 4 It is the CA attention mechanism of this application; as shown in the figure, the feature extraction module is reconstructed using a deformable convolution module. The deformable convolution module is composed of DCNv4, batch normalization and SiLU activation function, and the CA attention mechanism is introduced to enhance the ability to extract lesions. By reconstructing the C3 structure of the YOLOv5 backbone network with an efficient deformable convolution module (DCNv4), the model's ability to extract features from lesions can be improved. Specifically, when optimizing the feature extraction backbone network, the capsule endoscopy image data set is grouped, and the offset and weight of each convolution kernel position are calculated for each group. The multiplication results of the convolution kernels are combined and output according to the calculation results. The deformable convolution has a large receptive field, can enhance adaptive spatial aggregation, and optimize the processing speed, thereby accelerating model convergence. The deformable convolution allows the convolution kernel to shift its sampling position on the input feature map. The offset of each sampling position is realized by an additional convolution branch. At the same time, the deformable convolution not only contains the offset, but also provides additional weight information. The weight is used to control the degree of influence of the offset. The specific formula is as follows: , where x and y represent the grouped input and output respectively. is the weight of the kth sampling point, is the sampling location A CA attention (Coordinate Attention) mechanism is added to the tail of the backbone network structure. The CA attention mechanism aggregates features in the height and width spatial dimensions through two one-dimensional average pooling, calculates two directional attentions, and enhances the model's positioning of the lesion of interest.

[0030] (2) In the feature fusion neck network, the Gather-and-Distribute mechanism is introduced to fuse and distribute features at different levels to enrich feature expression. The Gather-and-Distribute mechanism mainly includes two parts: Gather collection and Distribute allocation. The Gather collection part collects features at different levels through convolution operations and Transformer self-attention. The Distribute allocation part separates the fused global features and injects them into different branches through attention operations. By introducing the Gather-and-Distribute mechanism to replace the traditional feature pyramid structure to fuse features at different levels, the performance of lesion detection can be improved. The specific steps include: first, the features of different levels are collected and aligned through the feature alignment module, and the aligned features are fused through the information fusion module to obtain global information; the information distribution module uses attention to distribute the fused information to different levels. After passing through the neck feature fusion network, multi-scale feature maps of the lesion image are obtained. These feature maps are used for subsequent lesion detection.

[0031] (3) In the classification and localization detection head network, multiple features are used to detect lesions, and detection heads for small lesions are added to enhance the ability to detect complex lesions. In order to improve the detection accuracy of early small lesions, a detection head with a size of 160x160 pixels is added to the YOLOv5 model. The detection head with a size of 160x160 pixels is used to capture subtle feature information in the digestive tract. The original classification and localization detection head of the YOLOv5 model contains three output modules of different scales. These output modules divide the feature map into several grids and predict it according to the center of the lesion. The sizes of the feature maps output by these output modules are 80x80, 40x40 and 20x20 respectively. Feature maps of different scales are used to detect lesions of different sizes. The feature map with a size of 80x80 is used to identify smaller target lesions, and the feature map with a size of 20x20 is used to identify larger target lesions. The predicted results are compared with the true values ​​to calculate the loss. In deep learning, as the number of layers increases, the size of the feature map continues to decrease. Small lesions may only occupy a small part of the image and are easily lost in the deep semantic information. In the YOLOv5 model, each original classification and positioning detection head is responsible for lesions of a specific scale. Small lesions need to be detected in feature maps with higher resolution. Therefore, a detection head with a size of 160x160 pixels is added to better capture the detailed features of the lesions.

[0032] (4) In the loss function, the loss function includes the category loss function and the bounding box loss function to solve the problem of class imbalance. The classification loss is calculated by Focal Loss to adjust the sample balance to adjust the balance between positive and negative samples. At the same time, the bounding box loss is calculated by the CIOU loss function. This loss function considers the overlapping area between the predicted box and the true box, the distance between the center point coordinates, and the aspect ratio of the bounding box for calculation, so as to make the bounding box regression more stable. The main principle of Focal Loss is to give higher weights to difficult-to-classify samples and lower weights to easy-to-classify samples on the basis of cross entropy loss, thereby reducing the contribution of easy-to-classify samples to the total loss and guiding the model to pay more attention to difficult-to-classify samples. The calculation formula is as follows: ,in Represents the probability that the model predicts a positive sample, Represents the balance coefficient, which is used to adjust the balance between positive and negative categories. is very small, and the sample loss contributes little to the overall result. Close to 1, the loss value of the sample is larger and its contribution to the total loss increases.

[0033] As mentioned above, the CIOU loss function is used to calculate the bounding box loss. The loss function considers the overlapping area between the predicted box and the true box, the distance between the center point coordinates, and the aspect ratio of the bounding box for calculation. The core of CIOU is to optimize the positioning accuracy of the model in lesion detection by minimizing the difference between the predicted box and the true box. On the basis of IOU, the center point distance and aspect ratio consistency are added. The calculation formula is as follows: in It is a parameter used to measure the predicted box and the bounding box. is a parameter used to measure the consistency of aspect ratio. .

[0034] Please also read Figure 5 , which is a schematic diagram of the capsule endoscope image-assisted diagnosis method based on deep learning of the present application; as shown in the figure, in step S4, the capsule endoscope image-assisted diagnosis model is trained by the capsule endoscope image data set, and the trained capsule endoscope image-assisted diagnosis model is used to detect the image to be diagnosed. In the model application stage, a capsule endoscope image-assisted diagnosis model based on LD-YOLO is established. The capsule endoscope image data set processed by step S2 is input into the model for multi-scene and multi-scale training. The lesion image features are extracted through the network structure of the improved model to obtain a trained lesion detection model. The capsule endoscope image sequence to be diagnosed is input into the trained model, and the category information and location coordinates of the disease are output. Assist clinicians in diagnosing digestive tract diseases, save film reading time and improve detection efficiency.

[0035] Specifically, first, the capsule endoscope image-assisted diagnosis model is built and optimized in step S3, and the capsule endoscope image data set processed in step S2 is used as the input image to iteratively train the LD-YOLO model, and finally the weight parameters of the trained auxiliary diagnosis model are obtained. In clinical applications, the capsule endoscope image sequence to be diagnosed is input into the LD-YOLO model, and the output image detection obtains the category information and location coordinates of the lesion, and visualizes the output on the display interface. This process assists clinicians in diagnosing digestive tract diseases, can save film reading time, improve detection efficiency and accuracy, reduce the workload of doctors, and ensure that patients can obtain timely and accurate medical services.

[0036] The deep learning-based capsule endoscopy image-assisted diagnosis method implemented in this paper builds a deep learning model LD-YOLO, selects the YOLOv5 model as the basis, and optimizes the basic model: (1) The feature extraction backbone C3 module of the basic model YOLOv5 is reconstructed by deformable convolution, which can flexibly capture the lesion features in complex environments. At the same time, the introduction of the attention mechanism further improves the model's ability to accurately locate the lesion area; (2) The Gather-and-Distribute mechanism is applied to optimize the feature fusion process. Features from different levels are uniformly aggregated in the global receptive field and distributed to different detection heads to avoid the problem of information loss when fusing cross-layer features; (3) For the common tiny lesions in capsule endoscopy images, a small lesion detection head is added, and Focal Loss and CIOU are combined as loss functions to optimize the classification and bounding box regression accuracy of the model, thereby improving the detection accuracy of tiny lesions.

[0037] As mentioned above, by automatically analyzing and processing a large number of capsule endoscope image data sets, suspicious lesion areas in the image can be effectively detected. In this way, the improved model LD-YOLO assists doctors in endoscopic image detection, solves the problems of inaccurate position of lesion detection model, insufficient feature extraction ability, and limited ability to detect tiny lesions, and has good detection accuracy and efficiency. Compared with traditional film reading methods, the capsule endoscope image-assisted diagnosis method based on deep learning in this application can greatly reduce the workload of doctors in image analysis, improve film reading efficiency, overcome the detection difficulties caused by image quality limitations, and has good generalization performance, which provides strong technical support for the accurate diagnosis and treatment of digestive tract diseases.

[0038] The following is a description of the beneficial effects of the capsule endoscope image-assisted diagnosis method based on deep learning in this application in conjunction with specific embodiments. This embodiment implements capsule endoscopy image-assisted lesion diagnosis based on deep learning based on PyTorch deep learning environment. The embodiment process is as follows: Figure 6 shown.

[0039] 1. Image acquisition and dataset construction: Gastrointestinal image data obtained by capsule endoscopy are collected. Labeling tool library LabelImg is used to label various types of lesions, and YOLO format annotation files suitable for target detection tasks are generated to complete the construction of the original image dataset. After the annotation is completed, the original dataset for training is constructed. 2. Image preprocessing: The dataset is expanded by performing data augmentation on the original images, adjusting the images to the same size using geometric transformations, and performing mean-variance normalization on the dataset to adapt to the model input. In addition, the K-means clustering algorithm is used to determine the size and scale of the prior anchor box.

[0040] 3. Experimental environment configuration. Select appropriate hyperparameters as needed, including the number of epochs, batch size, and optimizer. The classification loss function uses Focal Loss to solve the class imbalance problem, and the bounding box loss function uses CIOU to improve the accuracy of bounding box regression, and then perform model training.

[0041] 4. Model training: During the training process, we can adopt the idea of ​​transfer learning, use large-scale public gastrointestinal data sets for pre-training, and then use the collected data for fine-tuning, or we can input them together for training. We can continuously iterate and save the best model parameters during the training process, and set conditions to determine when the training should be terminated.

[0042] 5. Result evaluation: After training, load the best model parameters and predict the test set. Output the diagnosis results and calculate the evaluation indicators, mainly using the average detection accuracy mAP of each category. If the intersection-over-union ratio (IOU) between the predicted result and the actual target is greater than 0.5, the lesion detection is considered successful.

[0043] 6. Model application assists lesion diagnosis and deploys the final model in actual clinical practice. Lesion diagnosis is performed on capsule endoscopy images to be tested, helping clinicians improve work efficiency and diagnostic accuracy and reduce the risk of missed diagnosis and misdiagnosis.

[0044] In summary, the present application provides a capsule endoscope image-assisted diagnosis method based on deep learning. By building a deep learning model LD-YOLO, selecting the YOLOv5 model as the basis, optimizing the basic model, and then automatically analyzing and processing a large number of capsule endoscope image data sets, the suspicious lesion area in the image can be effectively detected. In this way, the improved model LD-YOLO assists doctors in endoscopic image detection, solves the problems of inaccurate position of the lesion detection model, insufficient feature extraction ability, and limited ability to detect small lesions, and has good detection accuracy and efficiency. Compared with the traditional film reading method, the capsule endoscope image-assisted diagnosis method based on deep learning in the present application can greatly reduce the workload of doctors in image analysis, improve the efficiency of film reading, overcome the detection difficulties caused by image quality limitations, and have good generalization performance, which provides strong technical support for the accurate diagnosis and treatment of digestive tract diseases. The present application has a significant effect on improving the efficiency of doctors' film reading and accurately detecting lesion areas, while reducing the risk of missed diagnosis and misdiagnosis, and improving the diagnosis and treatment efficiency of digestive tract diseases.

[0045] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element.

[0046] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present application, ordinary technicians in this field can also make many forms without departing from the purpose of the present application and the scope of protection of the claims, all of which are within the protection of the present application.

Claims

1. A capsule endoscope image-assisted diagnosis method based on deep learning, characterized in that: The following steps are involved: Collecting the original image sequence of the digestive tract taken by the wireless capsule endoscope, screening out the image sequence related to the digestive tract from the original image sequence, and marking the image sequence, wherein the original image sequence records all the image data of the subject from swallowing the capsule until the capsule is naturally excreted from the body; The labeled image sequence is subjected to data enhancement, geometric transformation and standardization processing in sequence, and a priori anchor frames are clustered using a K-means algorithm to form a capsule endoscopy image dataset; Build a deep learning model LD-YOLO, select the YOLOv5 model as the basis, optimize it on the basic model, and build a capsule endoscopy image-assisted diagnosis model; The capsule endoscope image-assisted diagnosis model is trained using a capsule endoscope image data set, and the trained capsule endoscope image-assisted diagnosis model is used to detect images to be diagnosed.

2. The capsule endoscope image-assisted diagnosis method based on deep learning according to claim 1, characterized in that: The image sequences cover different lesion types and locations as well as diverse patient samples.

3. The capsule endoscope image-assisted diagnosis method based on deep learning according to claim 1, characterized in that: The steps of sequentially performing data enhancement, geometric transformation and standardization on the labeled image sequence include: The image sequence is used as input to an image preprocessing stage, and the lesion image in the image sequence is amplified by a data enhancement technology, and the annotation is processed accordingly to ensure that the enhanced image is consistent with its annotation, wherein the data enhancement technology includes at least one of an affine transformation and a color space transformation; Make the lesion image set after data enhancement meet the input requirements of the deep learning model, improve the efficiency of feature extraction and target detection, perform geometric transformation on the amplified data, and adjust the image size; The lesion image set after geometric transformation is used as input, the image is mean normalized, and the mean of the single image data set is calculated. and standard deviation , and by the formula: , to standardize the image, where is the value to be standardized, is the standardized value.

4. The capsule endoscope image-assisted diagnosis method based on deep learning according to claim 3 is characterized in that: The steps of clustering prior anchor boxes using the K-means algorithm include: A statistically based K-means clustering algorithm is used to construct a set of lesion images with bounding box sizes. The bounding box information of the lesion area in the lesion image set is analyzed and clustered by iteratively updating the cluster center to generate prior anchor boxes of different scales.

5. The capsule endoscope image-assisted diagnosis method based on deep learning according to claim 1, characterized in that: The selected YOLOv5 model has an input end, a feature extraction backbone network, a feature fusion neck network and a classification and positioning detection head network. The input end receives a capsule endoscope image data set, the feature extraction backbone network extracts image features, the feature fusion neck network is used to fuse features of different levels, and the classification and positioning detection head network is responsible for predicting the location and category of the target lesion. When optimizing the YOLOv5 model: In the feature extraction backbone network, a deformable convolution module is used to reconstruct the feature extraction module, wherein the deformable convolution module is composed of DCNv4, batch normalization, and SiLU activation function, and a CA attention mechanism is introduced to enhance the ability to extract lesions; In the feature fusion neck network, the Gather-and-Distribute mechanism is introduced to fuse and distribute features at different levels to enrich feature expression; In the classification and positioning detection head network, multiple features are used to detect lesions, and detection heads for tiny lesions are added to enhance the ability to detect complex lesions; In the loss function, the loss function includes the category loss function and the bounding box loss function. The classification loss is calculated by Focal Loss to adjust the sample balance, and the bounding box loss is calculated by CIOU loss function.

6. The capsule endoscope image-assisted diagnosis method based on deep learning according to claim 5, characterized in that: The steps of optimizing the feature extraction backbone network include: The capsule endoscopy image dataset is grouped, and the offset and weight of each convolution kernel position are calculated for each group. The multiplication results of the convolution kernels are combined and output according to the calculation results. Deformable convolution allows the sampling position of the convolution kernel on the input feature map to be offset. The offset of each sampling position is realized by an additional convolution branch. At the same time, the deformable convolution not only contains the offset, but also provides additional weight information. The weight is used to control the influence of the offset. The specific formula is as follows: , where x and y represent the grouped input and output respectively, is the weight of the kth sampling point, is the sampling location The offset; A CA attention mechanism is added to the tail of the backbone network structure. The CA attention mechanism aggregates features in the height and width spatial dimensions through two one-dimensional average pooling, calculates two directional attentions, and enhances the model's positioning of lesions of interest.

7. The capsule endoscope image-assisted diagnosis method based on deep learning according to claim 5, characterized in that: In optimizing the feature fusion neck network, the Gather-and-Distribute mechanism mainly includes two parts: Gather collection and Distribute allocation. The Gather collection part collects features of different levels through convolution operations and Transformer self-attention, and the Distribute allocation part separates the fused global features and injects them into different branches through attention operations. The specific steps include: First, the features at different levels are collected and aligned through the feature alignment module, and the aligned features are fused through the information fusion module to obtain global information; The information distribution module uses attention to distribute the fused information to different levels. After passing through the neck feature fusion network, multi-scale feature maps of the lesion image are obtained. These feature maps are used for subsequent lesion detection.

8. The capsule endoscope image-assisted diagnosis method based on deep learning according to claim 5, characterized in that: In optimizing the classification and positioning detection head network, a detection head with a size of 160x160 pixels is added to the YOLOv5 model. The detection head with a size of 160x160 pixels is used to capture subtle feature information in the digestive tract. The original classification and positioning detection head of the YOLOv5 model includes three output modules of different scales. These output modules divide the feature map into several grids and predict it according to the center of the lesion. The sizes of the feature maps output by these output modules are 80x80, 40x40 and 20x20 respectively. Feature maps of different scales are used to detect lesions of different sizes, among which the feature map with a size of 80x80 is used to identify smaller target lesions, and the feature map with a size of 20x20 is used to identify larger target lesions. The predicted results are compared with the true values ​​to calculate the loss.

9. The capsule endoscope image-assisted diagnosis method based on deep learning according to claim 5, characterized in that: The steps to adjust sample balance by calculating classification loss through Focal Loss include: The main principle of Focal Loss is to give higher weights to difficult-to-classify samples and lower weights to easy-to-classify samples on the basis of cross entropy loss, thereby reducing the contribution of easy-to-classify samples to the total loss and guiding the model to pay more attention to difficult-to-classify samples. The calculation formula is as follows: ,in Represents the probability that the model predicts a positive sample, Represents the balance coefficient, which is used to adjust the balance between positive and negative categories.

10. The capsule endoscope image-assisted diagnosis method based on deep learning according to claim 5, characterized in that: The steps to calculate the bounding box loss through the CIOU loss function include: The CIOU loss function is used to calculate the bounding box loss. The loss function considers the overlapping area between the predicted box and the true box, the distance between the center point coordinates, and the aspect ratio of the bounding box for calculation. The core of CIOU is to optimize the positioning accuracy of the model in lesion detection by minimizing the difference between the predicted box and the true box. On the basis of IOU, the center point distance and aspect ratio consistency are added. The calculation formula is as follows: in It is a parameter used to measure the predicted box and the bounding box. is a parameter used to measure the consistency of aspect ratio. 。 11. The capsule endoscope image-assisted diagnosis method based on deep learning according to claim 1, characterized in that: The steps of training the capsule endoscope image-assisted diagnosis model by using the capsule endoscope image data set and detecting the image to be diagnosed by using the trained capsule endoscope image-assisted diagnosis model include: Conduct experimental configuration, train the deep learning-based diagnostic model with capsule endoscopy image data, and iteratively update and save the optimal model parameters; The trained model is used to assist in lesion detection, and the category information and location coordinates of the image to be diagnosed are obtained, which are visualized through the interface to improve detection efficiency.

Images