An image feature recognition method for improving YOLOv8x training precision

Abstract

The application discloses an image feature recognition method for improving training precision of YOLOv8x, and belongs to the field of combination of computer vision and medical images. An ECA module is added in a basic convolutional network of YOLOv8x, and a RepVgg reconstruction network structure is configured in the ECA module; in reasoning and deployment, a multi-branch structure is converted into a single-path structure by using a reparameterization technology, so that the calculation cost is better reduced and the speed is improved; meanwhile, an ESE attention mechanism (Effective Squeeze-and-Excitation Block) is added, one FC layer is reduced, feature channel information is maintained, and the convergence ability and execution speed of the model are further improved. Compared with the original YOLOv8x, the application is more excellent in the performance of accuracy, recall rate, F1 and the like, and the execution rate is improved to a certain extent.

Description

Technical Field

[0001] This invention relates to the field of combining computer vision with medical images, and specifically to an image feature recognition method that improves the training accuracy of YOLOv8x. Background Technology

[0002] Currently, the integration of deep learning with medical services is becoming increasingly diverse. It has become an important technology in areas such as medical image diagnosis, health management, and telemedicine, enabling real-time collection, transmission, analysis, and application of medical information. This improves the quality and efficiency of medical services while reducing medical risks and costs. Based on this research, this invention uses deep learning technology to identify medical image features and improves a target detection model that effectively identifies lesion features, helping doctors to understand patients more accurately and treat and intervene in care more effectively.

[0003] Object detection involves multiple disciplines such as image processing, artificial intelligence, machine learning, pattern recognition, and optimization. Its main task is to automatically predict the location and category of interest in an image through computation. Object detection algorithms can be divided into two main categories based on their implementation approach: One-Stage and Two-Stage object detection algorithms. They each have their strengths in different application areas. One-Stage algorithms offer higher inference speed, while Two-Stage algorithms offer higher accuracy in localization and object recognition. Classic One-Stage object detection algorithms include SSD and YOLO; representative Two-Stage object detection algorithms include the R-CNN series, especially Faster R-CNN. In January 2023, Ultralytics released YOLOv8, the latest deep learning-based object detection and image segmentation model in the YOLO series. It is touted as a cutting-edge, state-of-the-art detection model, not only inheriting the advantages of YOLOv5, YOLOv6, and YOLOv7 mode settings, but also allowing online adjustment of mosaic-enhanced images during training. This helps the model learn new locations, occluded locations, and other objects with different pixel ratios from their surroundings in different scene images, improving real-time evaluation on the COCO benchmark and prediction efficiency on Roboflow 100. YOLOv8 offers five available models for each category: YOLOv8n, YOLOv8s, YOLOv8m, YOLOv8l, and YOLOv8x, all usable for detection, segmentation, and classification. YOLOv8 Extra Large (YOLOv8x) is the most accurate but slowest, representing an improved version of the YOLOv8 object detection system. YOLOv8x has not yet been published in papers, therefore the direct research methods and ablation studies used in its creation are not readily available. To further develop feature recognition of medical images based on deep learning technology, this invention primarily uses the YOLOv8x model as the starting point for its technical research.

[0004] Combining the anchorless instance segmentation methods CenterMask and SAG-MASK attention branch structures mentioned in Lee Y et al.'s 2020 paper "Centermask: Real-time anchor-free instancesegmentation," this approach calculates the average value of each channel across the feature map space by inputting the feature map. A fully connected layer outputs a set of weights representing the importance of each channel. These learned weights are then applied to the input feature map, scaled by channel, to improve the focus on information pixels and balance training speed and prediction accuracy. In summary, the idea of ​​considering both a suitable network structure for the YOLOv8x model and employing relevant attention mechanisms to reduce the gap in prediction accuracy compared to larger models, while also improving training accuracy and speed, has significant application value and practical implications. Summary of the Invention

[0005] This invention utilizes classic object detection algorithms and Mask R-CNN algorithms in the first and second stages to challenge the best performance in feature recognition. Based on this, it proposes a YOLOv8x model based on the ESE attention mechanism combined with the RepEca network structure (where RepEca refers to the ECA module extended based on the RepVgg structure reparameterization technique). The ESE attention mechanism helps the model learn the location and importance of each feature in the image, reducing the influence of interference factors such as complex physiological structures, and improving the model's ability to detect and locate important features. As a result, the model outperforms the two-stage Faster R-CNN model, the improved Mask R-CNN model, and the original one-stage YOLOv8x model in terms of accuracy.

[0006] The technical solution adopted in this invention is as follows:

[0007] The specific implementation steps of an image feature recognition method to improve the training accuracy of YOLOv8x are as follows:

[0008] Step 1: Create an experimental.py file in ultralytics\models\v8 and add RepVgg to reconstruct the network structure, while modifying its input, output and step size parameters;

[0009] Step 2: Add the Eca module to the experimental.py file from Step 1, adjust the parameters according to its __init__ method, and modify its parameters in the ultralytics\models\v8\yolov8x.yaml configuration file.

[0010] Step 3: Create a parsing file Myblock.py in ultralytics\models\v8 and add the code to parse the Eca module;

[0011] Step 4: Add detection layer code to the YOLOv8.0x header section of the yolov8x.yaml file;

[0012] Step 5: Adjust the attention mechanism using the anchorless instance segmentation method CenterMask and the SAG-MASK attention branch from the article "Centermask: Real-time anchor-free instancesegmentation." Specifically, create the EseAttention.py file in ultralytics\nn and write the Effective Squeeze-and-Excitation Block code.

[0013] Step 6: Add the statement to import the attention mechanism from step 5 in ultralytics\nn\task.py, from ultralytics.nn.EseAttention import EseAttention, and at the same time adjust the code in the parse_model() method in that file;

[0014] Step 7: In the ultralytics\models\v8\yolov8x.yaml file, adjust the downsampling parameters starting from the YOLOv8.0x backbone and the upsampling parameters starting from the YOLOv8.0x head, according to the different convolutional layer scales.

[0015] Step 8: Create a main.py file in ultralytics\ to perform preprocessing operations such as image denoising and enhancement on the experimental image dataset; at the same time, build a YOLOv8x model object based on the ESE attention mechanism combined with the RepEca network structure, and load and run the parsing file Myblock.py at this time;

[0016] Step 9: Calculate the proportion of samples predicted as positive by the detector to contain true positive samples based on the values ​​of TP, FP, TN, and FN in the confusion matrix, as shown in Equation (1);

[0017]

[0018] Calculate the proportion (Recall) of all true positive samples that are predicted as positive by the detector, as shown in equation (2).

[0019]

[0020] Step 10: Calculate the F metric based on Precision and Recall from Step 9, as shown in Equation (3); it is used to compare the performance of different models on a given dataset and is a useful metric for evaluating model performance.

[0021]

[0022] Step 11: Obtain the AP value of the model for a certain feature category in the dataset by the area between the PR curve (Precision-Recall curve) and the X-axis in Step 10. At the same time, evaluate the average precision value mAP of all classes in the entire dataset under different IOU threshold settings to measure the detection quality of multiple classes; the mAP value ranges between (0,1), and the larger it is, the better the model is.

[0023] Step 12: Combining the changes in the parameters of backbone and head in the yolov8x.yaml file of steps 2 and 7, execute the code in step 8 and compare the magnitude of the evaluation index of the model in steps 9 to 11 under each change.

[0024] Step 13: Jump to step 7 to adjust the sampling parameters, and continue iterating until the last step, until the current model achieves a good optimization state when compared with the results of other object detection models.

[0025] The beneficial effects of this invention are:

[0026] (1) This image feature recognition method can quickly and effectively identify the lesion features in medical images.

[0027] (2) Add an ECA module to the basic convolutional network of YOLOv8x, and configure RepVgg in the ECA module to reconstruct the network structure. During inference and deployment, the reparameterization technique is used to transform the multi-branch structure into a single-path structure, which can better reduce computational overhead and improve speed.

[0028] (3) Adding an ESE attention mechanism (Effective Squeeze-and-Excitation Block) reduces one FC layer and maintains feature channel information, further improving the model's convergence ability and execution speed.

[0029] (4) Although this invention only processed the dataset of diabetes mellitus combined with pulmonary tuberculosis, it is representative of the data because it is based on comparisons at different levels. Attached Figure Description

[0030] Figure 1This is the RepEca network structure diagram proposed in this invention. The name comes from the combination of the names of the ECA module and the RepVgg reconstructed network structure, referring to the ECA module based on the modified RepVgg.

[0031] Figure 2 This invention applies the ESE attention mechanism to the RepEca structure, demonstrating its data processing flow and important parameter configuration diagram. Detailed Implementation

[0032] We have real medical data from 162 patients with diabetes and pulmonary tuberculosis. Based on the segmentation principle of the lungs, 1024 images were extracted, and the labelme tool was used to annotate the pixel contours of common lesions in the images. Training and test sets were generated proportionally. Simultaneously, six control group experiments were constructed with consistent parameters. At the One-Stage level, YOLOv5 and YOLOv8x models were used; at the Two-Stage level, Faster R-CNN and Mask R-CNN models based on different backbone network improvements were used. This was to explore the improvement of the YOLOv8x model based on the ESE attention mechanism combined with the RepEca network structure on the accuracy of feature category detection and model performance. The steps are as follows.

[0033] Step 1: Create an experimental.py file in ultralytics\models\v8 and add RepVgg to reconstruct the network structure, while modifying its input, output and step size parameters;

[0034] Step 2: Add the Eca module to the experimental.py file from Step 1, adjust the parameters according to its __init__ method, and modify its parameters in the ultralytics\models\v8\yolov8x.yaml configuration file.

[0035] Step 3: Create a parsing file Myblock.py in ultralytics\models\v8 and add the code to parse the Eca module;

[0036] Step 4: Add detection layer code to the YOLOv8.0x header section of the yolov8x.yaml file;

[0037] Step 5: Adjust the attention mechanism using the anchorless instance segmentation method CenterMask and the SAG-MASK attention branch from the article "Centermask: Real-time anchor-free instancesegmentation." Specifically, create the EseAttention.py file in ultralytics\nn and write the Effective Squeeze-and-Excitation Block code.

[0038] Step 6: Add the statement to import the attention mechanism from step 5 in ultralytics\nn\task.py, from ultralytics.nn.EseAttention import EseAttention, and at the same time adjust the code in the parse_model() method in that file;

[0039] Step 7: In the ultralytics\models\v8\yolov8x.yaml file, adjust the downsampling parameters starting from the YOLOv8.0x backbone and the upsampling parameters starting from the YOLOv8.0x head, according to the different convolutional layer scales.

[0040] Step 8: Create a main.py file in ultralytics\ to perform image denoising and enhancement on the experimental image dataset, completing the data preprocessing operation; at the same time, construct a YOLOv8x model object based on the ESE attention mechanism combined with the RepEca network structure, and load and run the parsing file Myblock.py at this time;

[0041] Step 9: Calculate the proportion of samples predicted as positive by the detector to contain true positive samples based on the values ​​of TP, FP, TN, and FN in the confusion matrix, as shown in Equation (1);

[0042]

[0043] Calculate the proportion (Recall) of all true positive samples that are predicted as positive by the detector, as shown in equation (2).

[0044]

[0045] Step 10: Calculate the F metric based on Precision and Recall from Step 9, as shown in Equation (3); it is used to compare the performance of different models on a given dataset and is a useful metric for evaluating model performance.

[0046]

[0047] Step 11: Obtain the AP value of the model for a certain feature category in the dataset by the area between the PR curve (Precision-Recall curve) and the X-axis in Step 10. At the same time, evaluate the average precision value mAP of all classes in the entire dataset under different IOU threshold settings to measure the detection quality of multiple classes; the mAP value ranges between (0,1), and the larger it is, the better the model is.

[0048] Step 12: Combining the changes in the parameters of backbone and head in the yolov8x.yaml file of steps 2 and 7, execute the code in step 8 and compare the magnitude of the evaluation index of the model in steps 9 to 11 under each change.

[0049] Step 13: Jump to step 7 to adjust the sampling parameters, and continue iterating until the last step, until the configuration file parameters are adjusted as shown in the figure below, and the improved model achieves the best result among the 6 control groups;

[0050]

[0051] The table below summarizes the maximum and minimum detection rates of each model when the IoU thresholds are 0.5 and 0.95. The YOLOv8x model based on the ESE attention mechanism combined with the RepEca network structure has the best performance with mAP50 = 0.88532 and mAP95 = 0.59803. The Mask R-CNN model with Swin Transformer as the backbone network ranks second, but it is slower than other models. Furthermore, the original YOLOv8x model also performs well in terms of mAP and running speed, surpassing models such as Mask R-CNN and Faster R-CNN. The training results demonstrate that the improved method of this invention can improve the computational efficiency of the experiment by taking advantage of their good convergence ability and the extreme speed of the simple structure, thereby achieving an efficient balance between speed and performance.

[0052]

[0053] The table below shows the performance of different models in terms of recall and precision, where P_m represents the maximum precision, R_m represents the maximum recall, and F1_m represents the maximum accuracy. In the feature detection process, the YOLOv8x model with the highest proportion of True positives is based on the ESE attention mechanism combined with the RepEca network structure. The addition of ESE enhances the model's adaptability to various lesion features and improves the detection accuracy of feature information. It can be used to construct a training object detection box model that meets the research needs of this invention.

[0054]

Claims

1. A medical image feature recognition method for improving YOLOv8x training precision, The features include the following steps: Step 1: Create an experimentral.py file in ultralytics\models\v8 and add the RepVgg reconfiguration network structure, and modify its input and output and step parameters; Step 2: Add the Eca module to the experimentral.py file in step 1, adjust the parameters according to its __init__ method, and modify its parameters in the ultralytics\models\v8\yolov8x.yaml configuration file; Step 3: Create a Myblock.py parsing file in ultralytics\models\v8 and add code to parse the Eca module; Step 4: Add detection layer code in the yolov8x.yaml file at YOLOv8.0x head; Step 5: Adjust the attention mechanism, create an EseAttention.py file in ultralytics\nn, and write the Effective Squeeze-and-Excitation Block code; Step 6: Add the attention mechanism statement in step 5 to the task.py file in ultralytics\nn, fromultralytics.nn.EseAttention import EseAttention, and adjust the code in the parse_model() method in this file; Step 7: In the ultralytics\models\v8\yolov8x.yaml file, adjust the down-sampling parameters from YOLOv8.0x backbone according to the size of different convolution layers, and adjust the up-sampling parameters from YOLOv8.0x head; Step 8: Create a main.py file in ultralytics\, perform image denoising and enhancement processing on the experimental image dataset, complete the data preprocessing operation, and build a YOLOv8x model object based on the ESE attention mechanism combined with the RepEca network structure, and load and run the parsing file Myblock.py at this time; Step 9: Calculate the proportion of true positive samples in the samples predicted as positive samples by the detector Precision, as shown in equation (1); calculate the proportion of all true positive samples that are predicted as positive samples by the detector Recall, as shown in equation (2) (1) ； (2) ； Step 10: Calculate the F metric value according to Precision and Recall in step 9, as shown in equation (3); it is used to compare the performance of different models on a given dataset, and is a useful indicator for evaluating model performance; (3) ； The tenth step: the area of the PR curve (Precision-Recall curve) and the X-axis in the tenth step is used to obtain the AP value of the model precision of a certain feature category in the data set, and the average precision value mAP of all classes in the entire data set under different loU threshold settings is used to measure the detection of multiple classes. The twelfth step: combine the changes of the parameters in the backbone and head of the yolov8x.yaml file in the second step and the seventh step, execute the eighth step code, and compare the size of the evaluation indicators of the model in the ninth step to the eleventh step under each change.

2. The medical image feature recognition method according to claim 1, characterized in that, The thirteenth step: jump to the seventh step to adjust the sampling parameters and continue to iterate to the last step until the current model and other target detection model results can achieve the best optimization state.

3. The medical image feature recognition method of claim 1, wherein, The CenterMask and SAG-MASK attention branch are used to adjust the attention mechanism. The mAP value ranges between (0, 1), and the larger it is, the better the model is.

Images