An image detection method based on improved FCOS network

Abstract

The present invention discloses an image detection method based on an improved FCOS network, comprising the following steps: 1) preparing a dataset; 2) labeling images in the dataset; 3) image preprocessing; 4) adding the names of the categories used in the initialization file; 5) setting parameters; 6) using the preprocessed images from step 3 as input to the network model; 7) obtaining output C1; 8) obtaining output C2; 9) obtaining output C3; 10) obtaining output C4; 11) obtaining output C5; 12) obtaining outputs S3, S4, and S5; 13) obtaining P3, P4, P5, P6, and P7; 14) setting a loss function before detection and classification; and 15) transmitting P3 to P7 obtained in step 13) to the detection head to obtain the final prediction result. This method achieves the purpose of expanding the convolution receptive field through intrinsic feature communication, thereby enhancing the diversity of output features and solving the information overload problem, allowing the model to focus on information that is more critical to the current task.

Description

Technical Field

[0001] The present invention belongs to the technical field of artificial intelligence and image detection, and particularly relates to an image detection method based on an improved FCOS network. Background Art

[0002] Object detection is a key technology in computer vision. Its primary goal is to accurately identify and localize objects of interest in images or videos. With the rapid development of artificial intelligence and deep learning, object detection technology has also made tremendous progress. Before the rise of deep learning, object detection primarily relied on traditional computer vision methods. These methods involved extracting features from images using feature engineering and classifying and localizing objects using traditional machine learning algorithms (such as support vector machines and decision trees). However, due to the diversity and complexity of objects, traditional methods often fail to effectively handle complex scenes and large amounts of data. With the rise of deep learning technology, particularly convolutional neural networks (CNNs), object detection has undergone a revolution. The superior performance of CNNs enables computers to automatically learn high-level features in images, significantly improving the accuracy and efficiency of object detection. The R-CNN (Region-based Convolutional Neural Networks) proposed by Yann LeCun et al. was the first end-to-end object detection framework, laying the foundation for subsequent developments. In 2014, Roos et al. proposed the two-stage network R-CNN, which marked a milestone in the application of CNNs to object detection. With the development of R-CNN algorithms, such as Faster R-CNN and Mask R-CNN, they have begun to be applied to image detection and segmentation. These two-stage object detection algorithms have achieved some success in image detection, but they are computationally complex, slow, and consume a lot of computing resources, requiring advanced hardware configurations. Summary of the Invention

[0003] The purpose of this invention is to address the problems of slow image detection speed, difficult feature extraction, and high computing resource consumption of the previous two-stage network model, and propose an image detection method based on an improved FCOS network. This method uses SCConv convolution to achieve the purpose of expanding the convolution receptive field through the intrinsic communication of features, thereby enhancing the diversity of output features. Through self-calibration operations, it adaptively establishes long-range spatial and channel dependencies around each spatial position, helping CNN generate more discriminative feature expressions. Because it has richer information, it uses the SE Attention mechanism module. Under the condition of limited computing power, computing resources are allocated to more important tasks, while solving the problem of information overload and allowing the model to focus on information that is more critical to the current task.

[0004] The technical solution for achieving the purpose of the present invention is:

[0005] An image detection method based on an improved FCOS network includes the following steps:

[0006] 1) First, create a dataset for training and testing. The dataset is a human lumbar intervertebral disc MRI-T2 image dataset, which is divided into a train dataset, a val dataset, and a test dataset in an 8:1:1 ratio.

[0007] 2) The pixels of the input images in the dataset are fixed to 768x768, and the COCO data format is used to annotate the images in the dataset;

[0008] 3) Perform data augmentation on all input images, including flipping and scaling, and preprocess the enhanced images using top-hat operations and grayscale stretching image preprocessing techniques;

[0009] 4) Use the general target detection platform MMDetection for detection. MMDetection is a target detection algorithm framework based on deep learning. Using MMDetection can quickly build a target detection network and implement target detection. First, you need to modify the COCO dataset code and replace the 80 categories in the COCO dataset code with the two categories of normal and diseased in the dataset. Then add the names of the categories to the initialization file.

[0010] 5) Set the stochastic gradient descent (SGD) method to optimize the training process, with an initial learning rate of 0.005 and a momentum of 0.9;

[0011] 6) The preprocessed image from step 3) is used as input to the network model;

[0012] 7) Backbone performs a convolution operation on the input image with a kernel size of 7x7 and a stride of 2, and then performs a maximum pooling operation with a kernel size of 3x3 and a stride of 2, obtaining the output result C1;

[0013] 8) Send C1 to the first self-calibration convolution module SCConv_1 to obtain the output result C2;

[0014] 9) Send C2 to the second self-calibration convolution module SCConv_2 to obtain the output result C3;

[0015] 10) Send C3 to the third self-calibration convolution module SCConv_3 to obtain the output result C4;

[0016] 11) Send C4 to the fourth self-calibration convolution module SCConv_4 to obtain the output result C5;

[0017] 12) C3, C4, and C5 are fed into the SE Attention module, where global average pooling is used as the Squeeze operation. Two FC layers then form a Bottleneck structure to model inter-channel correlations and output weights equal to the number of input features. The feature dimension is first reduced to 1 / r of the input, then activated by ReLU and then raised back to the original dimension by an FC layer. This approach, compared to using a single FC layer directly, has the advantage of having more nonlinearity, which can better fit the complex inter-channel correlations and greatly reduces the number of parameters and computations. A Sigmoid gate is then used to obtain normalized weights between 0 and 1. Finally, a Scale operation is used to apply the normalized weights to the features of each channel. After the activation operation, but without changing the size and number of channels before and after the operation, the SE Attention module processes the outputs S3, S4, and S5, respectively.

[0018] 13) S3, S4, and S5 are sent to the FPN module. FPN generates P3, P4, and P5 on the SE Attention outputs S3, S4, and S5 respectively. Then, a convolution layer with a kernel size of 3x3 and a stride of 2 is applied to P5 to obtain P6. Finally, a convolution layer with a kernel size of 3x3 and a stride of 2 is applied to P6 to obtain P7.

[0019] 14) Before detection and classification, a loss function needs to be set. The loss function has three output branches: classification, regression, and centrality. Therefore, the loss is composed of three parts: classification loss Lcls, positioning loss Lreg, and centrality loss Lctrness. The calculation method is shown in the following formula:

[0020]

[0021] p (x，y) represents the score of each category predicted at the feature map (x, y) point, Indicates the true category label corresponding to the point (x, y) in the feature map, The value is 1 when the feature map (x, y) point is matched as a positive sample, otherwise it is 0, t x,y Represents the target bounding box information predicted at the feature map (x, y) point, Indicates the true target bounding box information corresponding to the feature map (x, y) point, s x,y Represents the predicted centrality at the feature map (x, y) point, Indicates the true centrality corresponding to the point (x, y) in the feature graph;

[0022] 15) P3-P7 obtained in step 13) are sent to the detection head. P3-P7 share the same detection head, which has three subdivided branches: Classification, Regression, and Center-ness. Regression and Center-ness are two different small branches on the same branch. Classification, Regression, and Center-ness branches will first pass through 4 Conv2d+GN+ReLU combination modules, and then pass through a convolution layer with a convolution kernel size of 3x3 and a stride of 1 to obtain the final prediction result.

[0023] This technical solution is implemented through an anchor-free FCOS network model, the SCConv self-calibration convolution module, and the SE Attention mechanism module. The SCConv convolution expands the receptive field of the convolution through intrinsic feature communication, thereby enhancing the diversity of output features. The SE Attention mechanism can better utilize the dynamic relationship between feature channels.

[0024] Advantages or beneficial effects of this technical solution:

[0025] The target detection method proposed in this technical solution combines the latest outstanding network FCOS. The SCConv convolution used is different from the standard convolution, which uses a small kernel (such as 3×3 convolution) to simultaneously fuse information in the spatial dimension domain and the channel dimension. SCConv can adaptively establish long-range spatial and channel dependencies around each spatial location through self-calibration operations. Therefore, it can help CNN generate more discriminative feature representations because they contain richer information. The SE Attention mechanism module used in this technical solution allocates computing resources to more important tasks when computing power is limited, while also solving the problem of information overload and allowing the model to focus on information that is more critical to the current task. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 A network structure diagram of an embodiment;

[0027] Figure 2 SCConv module diagram in the embodiment;

[0028] Figure 3 This is the SC module structure in the embodiment;

[0029] Figure 4 This is a diagram of the attention mechanism module in the embodiment;

[0030] Figure 5 This is a structural diagram of the detection head in the embodiment;

[0031] Figure 6 is a network reasoning flow chart of an embodiment;

[0032] Figure 7 The original image of the spine MRI in the embodiment;

[0033] Figure 8 MRI test results of the spine in the embodiment. DETAILED DESCRIPTION

[0034] The present invention will be further described below with reference to the accompanying drawings and embodiments, but the present invention is not limited thereto.

[0035] Example:

[0036] Reference Figure 6 , an image detection method based on an improved FCOS network, comprising the following steps:

[0037] 1) First, create a dataset for training and testing. The dataset is a non-public human lumbar intervertebral disc MRI-T2 image dataset collected from the Internet, such as Figure 7 As shown, there are 470 images in total, which are divided into train data set (376 images), val data set (47 images) and test data set (47 images) according to the ratio of 8:1:1.

[0038] 2) The pixels of the input images in the dataset are fixed to 768x768, and the COCO data format is used to annotate the images in the dataset;

[0039] 3) Perform data augmentation on all input images, including flipping and scaling, and preprocess the enhanced images using top-hat operations and grayscale stretching image preprocessing techniques;

[0040] 4) Use the target detection platform MMDetection for detection. First, you need to modify the COCO dataset code, replace the 80 categories in the COCO dataset code with the two categories of normal and diseased in the dataset, and then add the names of the categories used in the initialization file;

[0041] 5) Set the stochastic gradient descent method (SGD) to optimize the training process, with an initial learning rate of 0.005 and a momentum of 0.9;

[0042] 6) The preprocessed image in step 3 is used as the input of the network model. The structure of the model is as follows Figure 1 As shown;

[0043] 7) Backbone performs a convolution operation on the input image with a kernel size of 7x7 and a stride of 2, and then performs a maximum pooling operation with a kernel size of 3x3 and a stride of 2, obtaining the output result C1;

[0044] 8) Send C1 to the first self-calibration convolution module SCConv_1. The SCConv module is Figure 2 As shown, Figure 2 (a) is the original structure, Figure 2 (b) is the improved structure of this example. The internal structure of the SC module is as follows: Figure 3 As shown, the output result C2 is obtained;

[0045] 9) Send C2 to the second self-calibration convolution module SCConv_2 to obtain the output result C3;

[0046] 10) Send C3 to the third self-calibration convolution module SCConv_3 to obtain the output result C4;

[0047] 11) Send C4 to the fourth self-calibration convolution module SCConv_4 to obtain the output result C5;

[0048] 12) Send C3, C4, and C5 to the SE Attention module. The structure of the SE Attention module in this example is as follows Figure 4 As shown in the figure, global average pooling is used as the Squeeze operation, followed by two FC layers to form a Bottleneck structure to model the correlation between channels and output weights equal to the number of input features. First, the feature dimension is reduced to 1 / r of the input, then activated by ReLU and then raised back to the original dimension by an FC layer. The advantage of this approach over using a single FC layer is that it has more nonlinearity, can better fit the complex correlation between channels, and greatly reduces the number of parameters and computations. Then, a Sigmoid gate is used to obtain normalized weights between 0 and 1. Finally, a Scale operation is used to weight the normalized weights to the features of each channel. After the activation operation, but without changing the size and number of channels before and after the operation, the SE Attention module is used to obtain outputs S3, S4, and S5 respectively.

[0049] 13) S3, S4, and S5 are sent to the FPN module. FPN generates P3, P4, and P5 on the SE Attention outputs S3, S4, and S5 respectively. Then, a convolution layer with a kernel size of 3x3 and a stride of 2 is applied to P5 to obtain P6. Finally, a convolution layer with a kernel size of 3x3 and a stride of 2 is applied to P6 to obtain P7.

[0050] 14) Before detection and classification, a loss function needs to be set. The loss function has three output branches: classification, regression, and centrality. Therefore, the loss is composed of three parts: classification loss Lcls, positioning loss Lreg, and centrality loss Lctrness. The calculation method is shown in the following formula:

[0051]

[0052] p (x，y) represents the score of each category predicted at the feature map (x, y) point, Indicates the true category label corresponding to the point (x, y) in the feature map, The value is 1 when the feature map (x, y) point is matched as a positive sample, otherwise it is 0, t x,y Represents the target bounding box information predicted at the feature map (x, y) point, Indicates the true target bounding box information corresponding to the feature map (x, y) point, s x,y Represents the predicted centrality at the feature map (x, y) point, Indicates the true centrality corresponding to the point (x, y) in the feature graph;

[0053] 15) P3 to P7 obtained in step 13) are transported to the detection head. P3 to P7 share one detection head. The structure of the detection head is as follows: Figure 5 As shown in , the detection head has three subdivision branches: Classification, Regression, and Center-ness. Regression and Center-ness are two different small branches on the same branch. Classification, Regression, and Center-ness branches will first pass through 4 Conv2d+GN+ReLU combination modules, and then pass through a convolution layer with a convolution kernel size of 3x3 and a stride of 1 to obtain the final prediction result, as shown in Figure 8 As shown;

[0054] 16) The trained network model was used to test the test set. The following table compares the test results of the original FCOS network with those of this method. It can be seen that this method has significantly improved the detection accuracy compared to the original FCOS network.

[0055]

Claims

1. An image detection method based on an improved FCOS network, characterized in that: The steps include: 1) First, create a training and testing dataset. The dataset is a human lumbar intervertebral disc MRI-T2 image dataset, which is divided into a train dataset, a val dataset, and a test dataset in an 8:1:1 ratio. 2) The pixels of the input images in the dataset are fixed to 768x768, and the COCO data format is used to annotate the images in the dataset; 3) Perform data augmentation on all input images, including flipping and scaling, and preprocess the enhanced images using top-hat operations and grayscale stretching image preprocessing techniques; 4) Use the general target detection platform MMDetection for detection. First, you need to modify the COCO dataset code, replace the 80 categories in the COCO dataset code with the two categories of normal and diseased in the dataset, and then add the names of the categories used in the initialization file; 5) Set the stochastic gradient descent method (SGD) to optimize the training process, with an initial learning rate of 0.005 and a momentum of 0.9; 6) The preprocessed image from step 3) is used as input to the network model; 7) Backbone performs a convolution operation on the input image with a kernel size of 7x7 and a stride of 2, and then performs a maximum pooling operation with a kernel size of 3x3 and a stride of 2, obtaining the output result C1; 8) Send C1 to the first self-calibration convolution module SCConv_1 to obtain the output result C2; 9) Send C2 to the second self-calibration convolution module SCConv_2 to obtain the output result C3; 10) Send C3 to the third self-calibration convolution module SCConv_3 to obtain the output result C4; 11) Send C4 to the fourth self-calibration convolution module SCConv_4 to obtain the output result C5; 12) C3, C4, and C5 are fed into the SE Attention module, where global average pooling is used as a squeeze operation. Two FC layers then form a bottleneck structure to model inter-channel correlations and output weights equal to the number of input features. The feature dimension is first reduced to 1 / r of the input, activated by ReLU, and then raised back to the original dimension by an FC layer. A sigmoid gate is then used to obtain normalized weights between 0 and 1. Finally, a scale operation is performed to apply the normalized weights to the features of each channel. After the activation operation, while maintaining the size and number of channels before and after the operation, the SE Attention module processes the outputs S3, S4, and S5, respectively. 13) S3, S4, and S5 are sent to the FPN module. FPN generates P3, P4, and P5 on the SE Attention outputs S3, S4, and S5 respectively. Then, a convolution layer with a kernel size of 3x3 and a stride of 2 is applied to P5 to obtain P6. Finally, a convolution layer with a kernel size of 3x3 and a stride of 2 is applied to P6 to obtain P7. 14) Before detection and classification, a loss function needs to be set. The loss function has three output branches: classification, regression, and centrality. Therefore, the loss is composed of three parts: classification loss Lcls, positioning loss Lreg, and centrality loss Lctrness. The calculation method is shown in the following formula: p (x，y) represents the score of each category predicted at the feature map (x, y) point, Indicates the true category label corresponding to the point (x, y) in the feature map, The value is 1 when the feature map (x, y) point is matched as a positive sample, otherwise it is 0, t x,y Represents the target bounding box information predicted at the feature map (x, y) point, Indicates the true target bounding box information corresponding to the feature map (x, y) point, s x,y Represents the predicted centrality at the feature map (x, y) point, Indicates the true centrality corresponding to the point (x, y) in the feature graph; 15) P3-P7 obtained in step 13) are sent to the detection head. P3-P7 share the same detection head, which has three subdivided branches: Classification, Regression, and Center-ness. Regression and Center-ness are two different small branches on the same branch. Classification, Regression, and Center-ness branches will first pass through 4 Conv2d+GN+ReLU combination modules, and then pass through a convolution layer with a convolution kernel size of 3x3 and a stride of 1 to obtain the final prediction result.

Images