Dual-channel feature pyramid method for peripheral blood image data feature extraction

Abstract

The invention discloses a two-channel feature pyramid method for peripheral blood image data feature extraction. The method comprises the following steps of S1, extracting a feature map from an original data image by adopting a MobileNetV3-Marge network; s2, constructing a dual-channel feature pyramid network composed of a small target feature pyramid network and a large target feature pyramid network; and S3, constructing a Faster-RCNN target detection network, wherein the Faster-RCNN target detection network comprises a feature extraction network, a region proposal network RPN, an ROI pooling layer and a classification layer. According to the dual-channel feature pyramid network, special feature extraction channels are designed for targets of different sizes, so that information interference between the targets is effectively reduced, and the situation that feature information of small-size targets is lost when a traditional feature extraction network performs feature extraction on large-size targets is avoided.

Description

Technical Field

[0001] The invention belongs to the field of computer vision and relates to a dual-channel feature pyramid method for extracting features of peripheral blood image data. Background Art

[0002] With the advancement of computer vision and image processing technology, artificial intelligence has begun to show its potential in the analysis of medical images. Image data recognition through machine learning or deep learning technology can significantly assist doctors in quickly identifying targets and locating areas of interest. However, in peripheral blood image data, multiple targets involved in disease diagnosis are often mixed together. If the feature extraction of different targets is inaccurate, the target recognition accuracy will be reduced. Especially when there are multiple targets with different sizes, large-sized targets occupy a higher proportion of the image area. Traditional feature extraction methods focus too much on the features of large targets and easily cause the loss of small-sized feature information. They ignore the feature extraction of both large and small targets, resulting in poor performance in simultaneous detection of large and small targets. Summary of the invention

[0003] The purpose of the present invention is to overcome the shortcomings of the prior art and provide a dual-channel feature pyramid method for feature extraction of peripheral blood image data. The dual-channel feature pyramid network effectively reduces the information interference between targets by designing dedicated feature extraction channels for targets of different sizes, thereby avoiding the situation in which the feature information of small-sized targets is lost when the traditional feature extraction network extracts features from large-sized targets.

[0004] The objective of the present invention is achieved through the following technical solution: a dual-channel feature pyramid method for peripheral blood image data feature extraction, comprising the following steps:

[0005] S1, collecting peripheral blood images and scaling the images to a fixed size;

[0006] S2. Use MobileNetV3-Large network as the backbone network to extract feature maps from the original data image; MobileNetV3-Large network includes two two-dimensional convolutional layers and 15 bneck blocks located between the two two-dimensional convolutional layers, and the two-dimensional convolutional layer and the 15 bneck blocks are connected in sequence; each convolutional layer and bneck block outputs a feature map respectively, and 17 feature maps are obtained;

[0007] The 1st, 2nd, 3rd, 4th, 8th, 9th, 10th, and 11th bneck blocks include 1×1 convolution, DW convolution, and 1×1 convolution respectively;

[0008] The 5th, 6th, 7th, 12th, 13th, 14th, and 15th bneck blocks include 1×1 convolution, DW convolution, SE module, and 1×1 convolution in sequence. Each bneck block first uses 1×1 convolution to increase the dimension of the input tensor, and then extracts features through DW convolution; then uses SE module to process the extracted features, and finally uses 1×1 convolution to reduce the dimension of the features output by SE module to obtain the output tensor; SE module includes pooling layer, fully connected layer + ReLU activation function, and fully connected layer + H-Swish activation function connected in sequence;

[0009] S3. Construct a dual-channel feature pyramid network consisting of a small target feature pyramid network and a large target feature pyramid network. The input of the small target feature pyramid network is the feature maps of the 2nd, 4th, and 16th layers of the MobileNetV3-Large network, and the three feature maps are recorded as S1, S2, and S3 respectively. The input of the large target feature pyramid network is the feature maps of the 8th, 13th, and 16th layers of the MobileNetV3-Large network, and the three feature maps are recorded as K1, K2, and K3 respectively.

[0010] The structure of the small target feature pyramid network is the same as that of the large target feature pyramid network, both of which include an inverse pyramid structure, that is, S1 / K1 at the top of the MobileNetV3-Large network is used as the bottom layer C1 / C1* of the pyramid, S2 / K2 at the middle of the pyramid is used as the middle layer C2 / C2* of the pyramid, and S3 / K3 at the bottom of the pyramid is used as the top layer C3 / C3* of the pyramid; first, S3 / K3 is downsampled to generate a new feature map S4 / K4, and a new top layer C4 / C4* of the pyramid is constructed; S1 / K1~S4 / K4 are respectively input into the 1×1 convolution layer for dimensional compression;

[0011] The compressed S4 / K4 is input into the 3×3 convolution layer and then into the prediction layer to obtain the feature map P4 / P4*;

[0012] The compressed S4 / K4 and S3 / K3 are fused by element-by-element addition, then input into the 3×3 convolution layer, and then into the prediction layer to obtain the feature map P3 / P3*;

[0013] The fused S3 / K3 and S2 / K2 are fused by element-by-element addition, then input into the 3×3 convolution layer, and then into the prediction layer to obtain the feature map P2 / P2*;

[0014] The fused S2 / K2 and S1 / K1 are fused by element-by-element addition, then input into the 3×3 convolution layer, and then into the prediction layer to obtain the feature map P1 / P1*;

[0015] S4. Construct a Faster-RCNN target detection network: The Faster-RCNN target detection network includes a feature extraction network, a region proposal network, a ROI pooling layer, and a classification layer. The feature extraction network uses the dual-channel feature pyramid network constructed in step S3. The feature map extracted by the feature extraction network is input into the RPN to generate a target candidate box, and then input into the ROI pooling layer. The pooling layer converts candidate boxes of different sizes into fixed sizes. The classification layer is used to perform target category classification and frame regression on the features output by the ROI pooling layer.

[0016] The region proposal network specifically includes the following processing steps:

[0017] S41, input the 8 feature maps output by the dual-channel FPN into the region proposal network respectively;

[0018] S42, extracting features from the input feature map through a 3×3 convolutional layer;

[0019] S43, the feature maps output by the 3×3 convolutional layer are input into two 1×1 convolutional layers respectively; the first 1×1 convolutional layer is used for the binary classification task of the anchor point to determine whether the anchor point area contains the foreground object; then the output of the first 1×1 convolutional layer is transposed once, and then a softmax operation is performed, and then another transposition is performed to obtain the final classification score;

[0020] The second 1×1 convolutional layer is used for the bounding box regression task, that is, predicting the exact position of each anchor point;

[0021] S44. Generate the final region proposal based on the classification score and regression results. These proposal boxes will be sent to the ROI pooling layer for processing.

[0022] The beneficial effects of the present invention are as follows: the present invention first builds a dual-channel feature pyramid network based on the feature map generated by the MobileNetV3-Large network, and effectively avoids the loss of its feature information in the feature extraction of large-size targets by designing a dedicated feature extraction channel for small-size targets. At the same time, the independent channel design reduces the feature interference between targets of different sizes and improves the accuracy of feature extraction. Finally, the dual-channel feature pyramid network is used to extract features from peripheral blood smear images, and the Faster-RCNN target detection algorithm is used to verify the recognition accuracy of the selected features. The results of the verification experiment show that due to the improved accuracy of feature extraction, the target detection accuracy of small-size targets and large-size targets has been significantly improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 The overall flow chart of the dual-channel feature pyramid method for feature extraction of peripheral blood image data;

[0024] Figure 2 This is the structure diagram of MobileNetV3-Large;

[0025] Figure 3 This is a dual-channel FPN architecture diagram;

[0026] Figure 4 This is a feature pyramid architecture diagram;

[0027] Figure 5 This is a functional diagram of the Faster-RCNN target detection algorithm;

[0028] Figure 6 This is the target detection effect diagram of peripheral blood image data. DETAILED DESCRIPTION

[0029] The present invention first uses the MobileNetV3-Large network based on a deep convolutional neural network as the backbone network (Backbone), which is mainly responsible for extracting effective information from the image and obtaining a feature map (Feature Map); then builds a multi-scale and hierarchical pyramid structure of the feature map, and constructs two dual-channel feature pyramid network (Feature Pyramid Network, FPN) architectures that integrate high-level feature semantic information; finally, the Faster-RCNN target detection model is used to process the features extracted by the dual-channel FPN network, and the target detection experimental results on the peripheral blood image data set are used to select appropriate FPN feature map parameters and verify the effectiveness of the invented method. The technical solution of the present invention is further explained below in conjunction with the accompanying drawings.

[0030] like Figure 1 As shown, a dual-channel feature pyramid method for peripheral blood image data feature extraction of the present invention comprises the following steps:

[0031] S1. Collect peripheral blood images and scale images of any size to fixed-size images of L×M (800×800 pixels by default); this is because the input size of the convolutional neural network is fixed. If the input image size is inconsistent, the convolution operation will not be performed correctly.

[0032] S2, using MobileNetV3-Large network as the backbone network to extract feature maps from the original data image; the structure of MobileNetV3-Large network is as follows Figure 2As shown in the figure, it includes two two-dimensional convolutional layers and 15 bneck blocks located between the two two-dimensional convolutional layers. The two-dimensional convolutional layer and the 15 bneck blocks are connected in sequence, and the 17-layer structure is recorded as 0 to 16 respectively; each convolutional layer and bneck block outputs a feature map (Feature Map) respectively, and 17 feature maps are obtained, which are numbered 0 to 16 respectively.

[0033] The 1st, 2nd, 3rd, 4th, 8th, 9th, 10th, and 11th bneck blocks include 1×1 convolution, DW convolution, and 1×1 convolution respectively.

[0034] The 5th, 6th, 7th, 12th, 13th, 14th, and 15th bneck blocks include 1×1 convolution, DW convolution, SE module, and 1×1 convolution in sequence. Each bneck block first uses 1×1 convolution to increase the dimension of the input tensor, and then extracts features through DW convolution (the convolution kernel size is selected as 3×3 or 5×5); then the extracted features are processed using the SE (Squeeze-and-Excitation) module, and finally the features output by the SE module are reduced in dimension using 1×1 convolution to obtain the output tensor. Such a structure with small ends and a large middle is used as a bneck block. This configuration is called an inverted residuals block (IRB). The concept of IRB helps to further reduce the computational cost of the model, and the SE module indicates whether there is squeezing and excitation in the bneck block.

[0035] The SE module includes a pooling layer (Pooling), a fully connected layer + ReLU activation function (FC relu), and a fully connected layer + H-Swish activation function (FC h-swish) connected in sequence. The role of the pooling layer is to reduce the size of the feature map while maintaining important feature information, thereby reducing the amount of calculation and preventing overfitting. In the fully connected layer + ReLU activation function, the fully connected layer connects all input nodes and all output nodes. It is often used in the last few layers of the network to convert the features extracted by the convolutional layer into specific outputs (such as classification results). The ReLU activation function is ReLU is a common activation function, defined as ReLU(x) = max(0,x). It truncates all negative values ​​to 0 and retains positive values, which helps introduce nonlinearity and accelerates training convergence. The role of the fully connected layer + H-Swish activation function is to improve the predictive ability of the model while maintaining efficient computing performance.

[0036] The SE module slightly increases the size of the system structure, but also enhances its accuracy. In addition, the h-swish activation function is added to the SE module. The swish activation function is defined as:

[0037]

[0038] Among them, β is a trainable parameter, σ(βx) is the sigmoid function, that is, when β = 1, the function is called the S-type weighted linear unit function. However, due to the high computational cost of calculating this function, the h-swish function is proposed, which is:

[0039]

[0040] Here RELU6 is a modified rectified linear unit. Through the above operations, the bneck block generates a feature map through the optimization of the SE module and IRB.

[0041] The network specifications of MobilenetV3-Large are shown in Table 1. exp size indicates the number of channels after the first convolutional layer in the module is upgraded. #out indicates the number of output channels. SE indicates whether the SE (Squeeze-and-Excitation) module is integrated in the corresponding bneck unit; NL indicates the type of nonlinearity used, where HS represents the h-swish activation function and RE represents the Relu activation function; NBN (No Batch Normalization) means that batch normalization is not included; stride is represented by s. There are a total of 17 feature maps output in MobilenetV3-Large. The feature maps of the upper layers have a coarser resolution, which contains the complete semantic information and location information of small-sized objects. Then the feature maps of the upper layers gradually pass the high-level semantic information to the deep feature maps through convolution operations, thereby extracting richer features and containing many detailed information. Therefore, the deeper feature maps of MobilenetV3-Large contain richer features of large-sized objects.

[0042] Table 1 MobilenetV3-Large network specifications

[0043] Feature Map Input Operator exp size #out SE NL s 0 <![CDATA[224 2 ×3]]> conv2d - 16 - HS 2 1 <![CDATA[112 2 ×16]]> bneck,3×3 16 16 - RE 1 2 <![CDATA[112 2 ×16]]> bneck,3×3 64 24 - RE 2 3 <![CDATA[56 2 ×24]]> bneck,3×3 72 24 - RE 1 4 <![CDATA[56 2 ×24]]> bneck,5×5 72 40 - RE 2 5 <![CDATA[28 2 ×40]]> bneck,5×5 120 40 √ RE 1 6 <![CDATA[28 2 ×40]]> bneck,5×5 120 40 √ RE 1 7 <![CDATA[28 2 ×40]]> bneck,3×3 240 80 √ RE 2 8 <![CDATA[14 2 ×80]]> bneck,3×3 200 80 - HS 1 9 <![CDATA[14 2 ×80]]> bneck,3×3 184 80 - HS 1 10 <![CDATA[14 2 ×80]]> bneck,3×3 184 80 - HS 1 11 <![CDATA[14 2 ×80]]> bneck,3×3 480 112 - HS 1 12 <![CDATA[14 2 ×112]]> bneck,3×3 672 112 √ HS 1 13 <![CDATA[14 2 ×112]]> bneck,5×5 672 160 √ HS 2 14 <![CDATA[7 2 ×160]]> bneck,5×5 960 160 √ HS 1 15 <![CDATA[7 2 ×160]]> bneck,5×5 960 160 √ HS 1 16 <![CDATA[7 2 ×160]]> conv2d,1×1 - 960 - HS 1

[0044] S3. Construct a dual-channel feature pyramid network (FPN) consisting of a small target feature pyramid network and a large target feature pyramid network. The input of the small target feature pyramid network is the feature maps of the 2nd, 4th, and 16th layers of the MobileNetV3-Large network, and the three feature maps are denoted as S1, S2, and S3 respectively. The input of the large target feature pyramid network is the feature maps of the 8th, 13th, and 16th layers of the MobileNetV3-Large network, and the three feature maps are denoted as K1, K2, and K3 respectively.

[0045] The MobilenetV3-Large used in the present invention has a total of 17 feature map outputs. The features in the input image data are continuously extracted through operations such as convolution. The deeper the feature map, the richer the semantic information, which is more suitable for feature extraction of large-sized targets. Since small-sized targets will lose their semantic information during continuous convolution for feature extraction, the deep feature maps do not necessarily contain complete semantic information. On the contrary, the upper-layer feature maps will have more complete semantic information, that is, the features of small-sized targets. Therefore, Figure 3 As shown in the figure, in the dual-channel feature pyramid network, the feature maps of the 2-4-16 layers (i.e., S1, S2, and S3 layers) in the upper layers of MobilenetV3-Large are selected to build an FPN architecture suitable for small target feature extraction, corresponding to the C1, C2, and C3 layers of the small target channel FPN. Because the 2nd and 4th layers are upper-layer feature maps with coarser resolution, which contain complete semantic information and location information of small-sized targets, and together with the 16th layer (with the richest detail information), FPN is built to achieve multi-scale feature extraction and fusion. For the feature extraction of large-sized targets, the upper-layer feature maps gradually transfer high-level semantic information to the deep feature maps through convolution operations, thereby extracting richer features and containing a lot of detail information. Therefore, the deeper feature maps of layers 8-13-16 in MobilenetV3-Large, namely the K1, K2, and K3 layers (K3 and S3 here are both feature maps of the same layer, namely the 16th layer) are selected to build an FPN suitable for feature extraction of large-size targets, corresponding to the C1*, C2*, and C3* layers of the large target channel FPN.

[0046] Whether it is the FPN of the large target channel or the FPN of the small target channel, after selecting the number of feature map layers output by the backbone network, the input feature map needs to be further processed. Taking the FPN of the small target channel as an example, Figure 4The internal working mechanism of FPN for the small target channel is shown. FPN builds a reverse pyramid structure according to the hierarchical ordering relationship of the selected feature maps S1, S2, and S3 layers in the original backbone network.

[0047] After the feature map is extracted from the backbone network (Backbone), it is fused at multiple levels through FPN, and finally feature maps of different resolutions are generated for the detection of small and large targets. The structure of the small target feature pyramid network is the same as that of the large target feature pyramid network, both of which include an inverse pyramid structure, that is, the upper S1 / K1 in the MobileNetV3-Large network is used as the bottom layer C1 / C1* of the pyramid, the middle S2 / K2 is used as the middle layer C2 / C2* of the pyramid, and the lower S3 / K3 is used as the top layer C3 / C3* of the pyramid; considering that S3 / K3 is at the lower level in the original backbone network and belongs to a low-resolution feature map, S3 / K3 is first downsampled to generate a new feature map S4 / K4 to improve its spatial resolution, and a new pyramid top layer C4 / C4* is constructed; S1 / K1~S4 / K4 are respectively input into the 1×1 convolution layer for dimensional compression;

[0048] The compressed S4 / K4 is input into the 3×3 convolution layer and then into the prediction layer to obtain the feature map P4 / P4*;

[0049] The compressed S4 / K4 and S3 / K3 are fused by element-by-element addition, then input into the 3×3 convolution layer, and then into the prediction layer to obtain the feature map P3 / P3*; the function of the 3×3 convolution layer is to extract features, fuse feature maps, and reduce information redundancy.

[0050] The fused S3 / K3 and S2 / K2 are fused by element-by-element addition, then input into the 3×3 convolution layer, and then into the prediction layer to obtain the feature map P2 / P2*;

[0051] The fused S2 / K2 and S1 / K1 are fused by element-by-element addition, then input into the 3×3 convolution layer, and then into the prediction layer to obtain the feature map P1 / P1*;

[0052] The prediction layer (predict) usually includes one or more convolutional layers, followed by an output layer to generate the prediction results. Generally, these layers include 1x1 convolutional layers (for channel compression) and Softmax or Sigmoid activation functions for classification or regression.

[0053] Through this series of processing steps, a set of final feature maps are generated externally, marked as {P1, P2, P3, P4}, {P1*, P2*, P3*, P4*}. They match the original {S1, S2, S3, S4}, {K1, K2, K3, K4} feature maps respectively, but have improved multi-scale fusion feature representation, which is more suitable for supporting the subsequent Faster-RCNN algorithm.

[0054] S4. Build Faster-RCNN target detection network: Faster-RCNN target detection network includes feature extraction network, region proposal network (RPN), ROI (Region of Interest, ROI) pooling layer, and classification layer; its structure is as follows Figure 5 As shown in the figure, the feature extraction network uses the dual-channel feature pyramid network constructed in step S3; the feature map extracted by the feature extraction network is input into the RPN to generate the target candidate box, and then input into the ROI pooling layer, which converts the candidate boxes of different sizes into fixed sizes for further processing; the classification layer is used to classify the target category and regress the border of the features output by the ROI pooling layer. These components work together to achieve fast and accurate detection of potential targets in the image.

[0055] The Faster-RCNN model first inputs the feature map into the RPN network to generate regions of interest (ROIs) (i.e., candidate boxes for targets). RPN uses Softmax to distinguish between positive and negative samples of anchor boxes, and then fine-tunes the anchor boxes through bounding box regression technology to improve the accuracy of the position. Next, the ROI pooling layer extracts and aggregates information from the input feature map and the predicted proposal box to form a feature map for each proposal box. These feature maps are then sent to the classification layer to identify the object category. In the final stage, the Faster-RCNN algorithm makes category decisions based on the feature map of the proposal box, and at the same time uses bounding box regression to determine the exact location of the detection box to achieve accurate detection of the target.

[0056] exist Figure 5 In the present invention, the dual-channel FPN is transplanted into the Faster-RCNN model and used as the feature extraction network of the Faster-RCNN model to replace its original feature extraction network. The feature maps output by the large target channel FPN and the small target channel FPN are passed in parallel to the RPN layer and the ROI pooling layer for further processing.

[0057] The region proposal network consists of two components: one part uses Softmax and performs a transposition (Reshape) before and after Softmax to distinguish the positive and negative anchor points; the other part is responsible for calculating the bounding box regression of the anchor points; then, the proposal layer integrates the positive sample anchor points and their corresponding bounding box regression offsets into the final proposal box (Proposal), and filters out proposal boxes that are too small or exceed the image boundary; the specific processing includes the following:

[0058] S41, input the 8 feature maps (P1, P2, P3, P4, P1*, P2*, P3*, P4*) output by the dual-channel FPN into the region proposal network respectively;

[0059] S42, extract features from the input feature map through a 3×3 convolution layer to integrate information from the local area and prepare for subsequent anchor classification and bounding box regression. The 3×3 convolution layer is a standard convolution operation, and its main function is to further extract features from the input feature map in order to better generate region proposals. Through this convolution layer, the network can integrate information from the local area and prepare for subsequent bounding box regression and classification tasks.

[0060] S43, input the feature map output by the 3×3 convolution layer into two 1×1 convolution layers respectively;

[0061] The first 1×1 convolutional layer is used for the binary classification task of the anchor point to determine whether the anchor point area contains the foreground object. This 1×1 convolutional layer will generate the corresponding classification score. Then the output of the first 1×1 convolutional layer is transposed (Reshaped), and then a softmax operation is performed, and then another transposition; the first Reshape is used to transform the dimension of the feature map so that there is a single dimension of 2, which is convenient for the softmax operation on this dimension, and then the second Reshape is used to restore it to its original state to obtain the final classification score.

[0062] The second 1×1 convolutional layer is used for the bounding box regression task, that is, predicting the exact position of each anchor. Through regression (output translation and scaling parameters), the network is able to adjust the position of the anchor to make it closer to the actual target box.

[0063] S44. Generate the final region proposal based on the classification score and the regression result output by the second 1×1 convolutional layer. These proposal boxes will be sent to the ROIPooling layer for further processing.

[0064] ROI pooling is to change each generated proposal box to a fixed size through pooling. The specific steps are: 1. Convert the proposal box to the feature map; 2. Divide the proposal box on each feature map into a grid; 3. Perform Max Pooling on each part of the grid.

[0065] The classification layer is responsible for determining the object category (e.g., red blood cells, white blood cells, platelets, etc.) corresponding to each proposal box. This process involves using the feature map generated by the proposal box. The classification layer includes a fully connected layer and a Softmax function to generate the predicted probability of each category, i.e., cls_prob. Faster-RCNN further uses bounding box regression technology to calculate the position adjustment bbox_pred for each proposal box, aiming to provide a more accurate target positioning box.

[0066] This paper uses MobileNetV3-Large as the backbone network to build a dual-channel FPN, and uses Faster-RCNN as the target detection algorithm for verification. AP and AR are both performance indicators used to evaluate target detection algorithms, where AP (Average Precision) represents the average precision value at different IOU levels, and AP S is the average precision of small objects, AP L is the average precision of large objects. AR (Average Recall) represents the average recall rate. Intersection over Union (IOU) represents the ratio of the overlap between the predicted box and the true box to the total overlap, which can be expressed as:

[0067]

[0068] The area of ​​overlap here refers to the area of ​​the intersection of the predicted bounding box and the actual bounding box, while the area of ​​union refers to the area covered by the two as a whole. Therefore, the higher the value of the intersection-union ratio (IOU), the higher the degree of overlap between the box predicted by the model and the real box. When the predicted box is perfectly aligned with the real box and overlaps seamlessly, the IOU reaches a maximum value of 1, which indicates that the model's prediction has achieved perfect accuracy. The present invention sets the threshold of IOU detection to 0.5 to 0.95, and takes the average value of ten IOUs with an interval of 0.05 as the evaluation index AP, and uses Faster-RCNN with MobileNetV3-Large as the backbone network as a model for testing on a peripheral blood image dataset.

[0069] The backbone network has many different levels of feature maps, and it is not necessary to use all feature levels to build FPN. Selecting different feature maps will have a significant impact on the results. Therefore, this method has been tested many times and uses different levels to build FPN to test the target detection accuracy, but always insists on extracting the upper feature map to build the FPN feature extraction network for small-sized targets, and the deeper feature map to build the FPN feature extraction network for large-sized targets. The target detection results of FPN built with some feature maps are shown in Table 2.

[0070] Table 2 Target detection results of FPN built with some feature maps

[0071]

[0072] The experimental results verify the idea of ​​the present invention. The FPN constructed using the upper feature map has a significant improvement in the recognition of small-sized targets, while the FPN constructed using the deep feature map can effectively improve the detection accuracy of large-sized targets. Therefore, based on the experimental results of this step, the present invention determines the feature map parameters of the aforementioned small target channel FPN as 1-3-16 layers, and the feature map parameters of the aforementioned large target channel FPN as 8-12-16 layers.

[0073] The following is to verify the effect of the proposed method on the peripheral blood image dataset to evaluate the feature extraction of dual-channel FPN. Figure 1 As shown in step e on the right. Consider feature extraction network solutions based on different backbone networks, as well as comparison solutions of single-channel FPN and dual-channel FPN. In order to compare with other mainstream backbone networks, the present invention re-experiments to obtain target detection results based on feature extraction networks of other backbone networks, and the dual-channel FPN feature extraction network also uses the same backbone network accordingly, for example, MobilenetV3-Large is replaced with Resnet series and other networks. Table 3 shows all experimental results. For example, when using dual-channel FPN to replace the traditional feature extraction network, the Faster-RCNN model is used for target detection, and MobileNetV3-Large is used as the backbone network. AP S ,AP L There are 7% and 4% improvements respectively. The model of this method has made comprehensive improvements in detecting small and large objects, achieving competitive performance.

[0074] Table 3 Comparison of dual-channel FPN and single-channel FPN target detection results

[0075]

[0076]

[0077] at last, Figure 6 The comparison results of the target detection performance of the dual-channel FPN feature extraction network and the traditional single-channel FPN network are intuitively demonstrated. When detecting small or large-sized targets, the dual-channel FPN produces satisfactory results, which are better than the results of the traditional single-channel FPN, especially for the detection of small-sized targets, the performance improvement is more obvious. The reason is that the dual-channel FPN provides global context information for the subsequent target detection model through multi-scale features, and performs separate feature extraction for the characteristics of large and small-sized targets, which improves the discrimination ability of the feature extraction network, allowing the Faster-RCNN target detection network to learn better, so the detection effect is better.

[0078] Those skilled in the art will appreciate that the embodiments described herein are intended to help readers understand the principles of the present invention, and should be understood that the protection scope of the present invention is not limited to such specific statements and embodiments. Those skilled in the art can make various other specific variations and combinations that do not deviate from the essence of the present invention based on the technical revelations disclosed by the present invention, and these variations and combinations are still within the protection scope of the present invention.

Claims

1. A dual-channel feature pyramid method for feature extraction of peripheral blood image data, characterized in that: The following steps are involved: S1, collecting peripheral blood images and scaling the images to a fixed size; S2. Use MobileNetV3-Large network as the backbone network to extract feature maps from the original data image; MobileNetV3-Large network includes two two-dimensional convolutional layers and 15 bneck blocks located between the two two-dimensional convolutional layers, and the two-dimensional convolutional layer and the 15 bneck blocks are connected in sequence; each convolutional layer and bneck block outputs a feature map respectively, and 17 feature maps are obtained; The 1st, 2nd, 3rd, 4th, 8th, 9th, 10th, and 11th bneck blocks include 1×1 convolution, DW convolution, and 1×1 convolution respectively; The 5th, 6th, 7th, 12th, 13th, 14th, and 15th bneck blocks include 1×1 convolution, DW convolution, SE module, and 1×1 convolution in sequence. Each bneck block first uses 1×1 convolution to increase the dimension of the input tensor, and then extracts features through DW convolution; then uses SE module to process the extracted features, and finally uses 1×1 convolution to reduce the dimension of the features output by SE module to obtain the output tensor; SE module includes pooling layer, fully connected layer + ReLU activation function, and fully connected layer + H-Swish activation function connected in sequence; S3, construct a dual-channel feature pyramid network consisting of a small target feature pyramid network and a large target feature pyramid network; The input of the feature pyramid network for small targets is the feature maps of the 2nd, 4th, and 16th layers of the MobileNetV3-Large network, and the three feature maps are recorded as S1, S2, and S3 respectively; the input of the feature pyramid network for large targets is the feature maps of the 8th, 13th, and 16th layers of the MobileNetV3-Large network, and the three feature maps are recorded as K1, K2, and K3 respectively; The structure of the small target feature pyramid network is the same as that of the large target feature pyramid network, both of which include an inverse pyramid structure, that is, S1 / K1 at the top of the MobileNetV3-Large network is used as the bottom layer C1 / C1* of the pyramid, S2 / K2 at the middle of the pyramid is used as the middle layer C2 / C2* of the pyramid, and S3 / K3 at the bottom of the pyramid is used as the top layer C3 / C3* of the pyramid; first, S3 / K3 is downsampled to generate a new feature map S4 / K4, and a new top layer C4 / C4* of the pyramid is constructed; S1 / K1~S4 / K4 are respectively input into the 1×1 convolution layer for dimensional compression; The compressed S4 / K4 is input into the 3×3 convolution layer and then into the prediction layer to obtain the feature map P4 / P4*; The compressed S4 / K4 and S3 / K3 are fused by element-by-element addition, then input into the 3×3 convolution layer, and then into the prediction layer to obtain the feature map P3 / P3*; The fused S3 / K3 and S2 / K2 are fused by element-by-element addition, then input into the 3×3 convolution layer, and then into the prediction layer to obtain the feature map P2 / P2*; The fused S2 / K2 and S1 / K1 are fused by element-by-element addition, then input into the 3×3 convolution layer, and then into the prediction layer to obtain the feature map P1 / P1*; S4. Construct a Faster-RCNN target detection network: The Faster-RCNN target detection network includes a feature extraction network, a region proposal network, a ROI pooling layer, and a classification layer. The feature extraction network uses the dual-channel feature pyramid network constructed in step S3. The feature map extracted by the feature extraction network is input into the RPN to generate a target candidate box, and then input into the ROI pooling layer. The pooling layer converts candidate boxes of different sizes into fixed sizes. The classification layer is used to perform target category classification and frame regression on the features output by the ROI pooling layer. The region proposal network specifically includes the following processing steps: S41, input the 8 feature maps output by the dual-channel FPN into the region proposal network respectively; S42, extracting features from the input feature map through a 3×3 convolutional layer; S43, the feature maps output by the 3×3 convolutional layer are input into two 1×1 convolutional layers respectively; the first 1×1 convolutional layer is used for the binary classification task of the anchor point to determine whether the anchor point area contains the foreground object; then the output of the first 1×1 convolutional layer is transposed once, and then a softmax operation is performed, and then another transposition is performed to obtain the final classification score; The second 1×1 convolutional layer is used for the bounding box regression task, that is, predicting the exact position of each anchor point; S44. Generate the final region proposal based on the classification score and regression results. These proposal boxes will be sent to the ROI pooling layer for processing.

Images