A brain tumor detection method based on deep learning

Abstract

The present invention provides a brain tumor detection method based on deep learning, comprising the following steps: extracting multi-scale features of an input magnetic resonance image through a backbone network, and outputting three effective feature layers: performing feature fusion enhancement on the three effective feature layers output by the backbone network through a neck network, and obtaining three feature output layers of different sizes: using a head-to-neck network to detect the three feature output layers finally obtained by the head-to-neck network, respectively, first using lightweight dynamic convolution to adjust the number of channels, and then using 1×1 convolution to predict brain tumor targets, and outputting a prediction result image containing brain tumor categories, bounding boxes, and confidence levels. The present application allows the neural network to focus on useful features, suppress interference from irrelevant features, and improve the generalization ability of the model; it can prevent the loss of detail information in the original image and improve the model detection effect; and at the same time, by using a lightweight dynamic convolution module for convolution processing, the computational complexity is reduced.

Description

Technical Field

[0001] The present invention relates to the technical field of medical target detection, and in particular to a brain tumor detection method based on deep learning. Background Art

[0002] Brain tumor is a disease that seriously endangers human health. The incidence and mortality of brain tumors continue to rise, seriously affecting the quality of life of patients. As the initial stage of treatment, timely detection of brain tumors can extend the life expectancy of patients. For doctors, it can provide a certain reference and auxiliary role in early cancer screening, surgical navigation and disease monitoring, provide a scientific basis for diagnosis and treatment, and reduce the workload of doctors.

[0003] In recent years, with the development and application of artificial intelligence in the medical field, deep learning has also made some progress in brain tumor detection. The existing single-stage YOLO series algorithm YOLOv7 model takes into account multiple indicators such as speed and accuracy. The architecture is relatively simple and easy to migrate. It is a highly competitive target detection model, suitable for use in multiple scenarios, and is also one of the current mainstream detection algorithms. However, the inventors of this application have found through research that the detection accuracy of the model on the brain tumor data set does not meet the requirements of higher standards, and the YOLOv7 detector still has the following problems: First, the detector will be interfered by irrelevant features when extracting features for the input brain tumor MR (Magnetic Resonance) image data set, causing the model to be overly dependent on the specific distribution of the data set and poor generalization ability; second, when the pooled convolution (MP) is downsampled, some detailed information in the original image is easily lost, resulting in poor model detection effect; third, the detection head of the head network contains multi-layer convolution processing, which makes the number of parameters and the amount of calculation large, resulting in a slow detection speed. Summary of the invention

[0004] In view of the technical problems that the existing YOLOv7 detector is interfered by irrelevant features when extracting features, some detail information in the original image is easily lost when downsampling by pooling convolution, and the detection head of the head network contains multiple layers of convolution processing, which slows down the detection speed, the present invention provides a brain tumor detection method based on deep learning.

[0005] In order to solve the above technical problems, the present invention adopts the following technical solutions:

[0006] A brain tumor detection method based on deep learning, comprising the following steps:

[0007] S1. Perform multi-scale feature extraction on the input magnetic resonance image through the backbone network and output three effective feature layers:

[0008] S11, the original magnetic resonance image is adjusted to a size of 640×640×3 and first enters the backbone network. After four sequentially connected CBS operations, namely convolution, batch normalization and activation function operations, the image size becomes 160×160×128;

[0009] S12, then enter a multi-branch stacking module for feature extraction, and the feature map size becomes 160×160×256; then perform MP downsampling operation and pass through a multi-branch stacking module to obtain an output feature layer of 80×80×512; finally, pass through MP downsampling operation and a multi-branch stacking module in sequence to obtain two output feature layers of size 40×40×1024 and 20×20×1024 respectively;

[0010] S2. The three effective feature layers output by the backbone network are enhanced through feature fusion by the neck network to obtain three feature output layers of different sizes:

[0011] S21, the three effective feature layers output by the backbone network are respectively subjected to a CBS operation, and then enter the squeezing and excitation module to suppress the interference of irrelevant features;

[0012] S22, then the feature layer of size 20×20×1024 enters the SPPCSPC module and RFE module in turn to increase the receptive field of the feature map, and changes the channel from 1024 to 512, that is, the size becomes 20×20×512 and is recorded as P5; then the UP upsampling operation is performed to obtain an output of size 40×40×256, and then the output and the feature layer of size 40×40×1024 after sequentially passing through the CBS operation, squeezing and excitation module are spliced ​​and fused, and the output size becomes 40×40×512; then the multi-branch stacking module operation is performed once, and the size becomes 40×40×256 and is recorded as P4, and then the UP upsampling operation is performed and the feature layer of size 80×80×512 after sequentially passing through the CBS operation, squeezing and excitation module is spliced ​​and fused, and the output size becomes 80×80×256; finally, the multi-branch stacking module feature extraction is performed once, and the output of size 80×80×128 is obtained and is recorded as P3;

[0013] S23, the fused feature layers P3 and P4, and the feature layer of size 20×20×1024 after sequentially passing through the CBS operation, the squeezing and excitation module, are fused from bottom to top, specifically, the feature layer P3 is first passed through the MP-SE downsampling module and then spliced ​​and fused with the feature layer P4, and then subjected to a multi-branch stacking module feature extraction to obtain an output of size 40×40×256, recorded as P4_out; then the MP downsampling operation is performed and the feature layer of size 20×20×1024 entering the SPPCSPC module is spliced ​​and fused, and finally subjected to a multi-branch stacking module feature extraction to obtain an output of size 20×20×512, recorded as SE_out', so that three feature output layers of different sizes, P3, P4_out and SE_out', are obtained;

[0014] S3, the three feature output layers P3, P4_out and SE_out' finally obtained by the head network detection head to neck network, respectively use lightweight dynamic convolution ODConv to adjust the number of channels, and then use 1×1 convolution Conv to predict the brain tumor target, and output the prediction result image containing the brain tumor category, bounding box and confidence.

[0015] Furthermore, the multi-branch stacking modules in steps S1 and S2 each include two branches, one branch performs only one CBS operation, and the other branch is composed of five CBSs stacked in series, and CBS considers the contribution value of each feature map to assign weights while performing convolution, and finally concatenates the results of the two branches on the channel and outputs them after another CBS operation.

[0016] Furthermore, the weight distribution of each feature map is defined as follows:

[0017]

[0018] Among them, w i is the normalized feature map weight value, w' i is the weight value of each feature map learned by the network during training, j is the current feature map number, N is the total number of feature maps, and ξ is a fixed value to prevent the denominator from being zero.

[0019] Furthermore, the MP downsampling in steps S12 and S23 includes two branches, one branch consists of a maximum pooling layer and a CBS operation, and the other branch consists of two CBSs connected in series. Finally, the results of the two branches are concatenated on the channel and then output.

[0020] Furthermore, the squeezing and excitation module in step S21 includes two operations: squeezing and excitation. Specifically, the input feature U is firstly passed through an F sqThe squeezing operation globally aggregates the feature map in the spatial dimension H×W, which is a global average pooling operation. Thus, a feature map with c channels, H image height, and W image width is compressed into a channel descriptor with C channels and 1×1. The specific calculation formula is as follows:

[0021]

[0022] Among them, Z C represents a scalar value representing the average value of the feature map across all spatial positions in the Cth channel; i represents the vertical coordinate of the feature map; j represents the horizontal coordinate of the feature map;

[0023] Next is F ex The excitation operation consists of two fully connected layers. This operation performs a connection transformation on the result obtained after squeezing the global pool. The channel descriptor after the two transformations is recorded as S C , the specific calculation formula is as follows:

[0024] S C =F ex (Z,W)=σ(W 2 δ(W 1 (Z)))

[0025] Among them, W 1 The representative size is R C×C / r The fully connected layer, W 2 The representative size is R C / r×C The fully connected layer, r is the channel bottleneck parameter, σ is the sigmoid function, and δ is the ReLU function;

[0026] Finally, the final output of the entire squeeze and excitation module is obtained by rescaling the transformation output U:

[0027] Furthermore, the SPPCSPC module in step S22 includes two branches, one branch is composed of three CBS in series, three maximum pooling layers in parallel, a splicing operation and two CBS in series stacked in sequence, and the other branch only performs a CBS operation once. Finally, the results of the two branches are spliced ​​on the channel and output after another CBS operation.

[0028] Furthermore, the RFE module in step S22 includes three parallel branches, each branch is composed of a 1×1 convolution, a 3×3 dilated convolution and another 1×1 convolution connected in series in sequence, and the input of each branch is weighted and summed with the output of another 1×1 convolution in each branch, and the final results of the three branches are converged and weighted on the average pooling layer and output, and the expansion rates of the 3×3 dilated convolutions in the three parallel branches are 1, 2, and 3, respectively.

[0029] Furthermore, the MP-SE downsampling module in step S23 includes two branches, one branch is formed by connecting the maximum pooling layer and CBS in series, and the other branch is formed by connecting the squeeze and excitation module and CBS in series, and finally the results of the two branches are spliced ​​on the channel and output.

[0030] Furthermore, the calculation formula of the lightweight dynamic convolution ODConv in step S3 is as follows:

[0031] y=(a w1 ⊙a i1 ⊙a c1 ⊙a s1 ⊙W 1 +...+a wi ⊙a fi ⊙a ci ⊙a si ⊙W i +...+a wn ⊙a fn ⊙a cn ⊙a sn ⊙W n )*x

[0032] Among them, a wi ∈R represents the convolution kernel W i The attention scalar, and a si ∈R k×k 、a si ∈R cin and a fi ∈R cout Represents the convolution kernel W respectively. i The newly introduced three types of attention along the spatial dimension, input channel dimension, and output channel dimension, ⊙ represents the multiplication operation between different dimensions in the convolution kernel space, and x represents the input feature layer.

[0033] Furthermore, the loss function used in the network model of the method is a new bounding box similarity metric MPDIoU, L MPDIoU The loss function is calculated as follows:

[0034]

[0035]

[0036]

[0037] L MPDIoU =1-MPDIoU

[0038] in, Represents the coordinates of the upper left and lower right corners of the predicted bounding box, It represents the coordinates of the upper left and lower right corners of the ground truth bounding box, IOU represents the intersection-over-union ratio, that is, the ratio of the intersection to the union between the predicted bounding box and the true bounding box, h represents the height of the input image, and w represents the width of the input image.

[0039] Compared with the prior art, the brain tumor detection method based on deep learning provided by the present invention has the following advantages: 1) A squeeze and excitation (SE) module and a multi-branch stacking module (W-ELAN) are added after the three effective feature layers output by the backbone network. The module automatically learns the importance of each channel in the feature map and assigns a weight value to each feature channel based on the importance, so that the neural network focuses on useful features, suppresses the interference of irrelevant features, and improves the generalization ability of the model; 2) MP-SE and RFE modules are constructed during pooled convolution (MP) downsampling to prevent the loss of detail information in the original image, thereby improving the model detection effect; 3) The detection head convolution of the head network uses a lightweight dynamic convolution ODConv module for convolution processing, thereby reducing the computational complexity; 4) The network model of the entire method uses L MPDIoU The loss function is used to solve the problem of inaccurate calculation, so that the model can achieve higher detection accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 It is a schematic diagram of the flow of the brain tumor detection method based on deep learning provided by the present invention.

[0041] Figure 2 This is a network model architecture diagram of the brain tumor detection method based on deep learning provided by the present invention.

[0042] Figure 3 It is a schematic diagram of the structure of the multi-branch stacking module provided by the present invention.

[0043] Figure 4 It is a schematic diagram of the structure of the MP downsampling module provided by the present invention.

[0044] Figure 5 It is a schematic diagram of the structure of the extrusion and excitation module provided by the present invention.

[0045] Figure 6 It is a schematic diagram of the structure of the SPPCSPC module provided by the present invention.

[0046] Figure 7 It is a schematic diagram of the structure of the RFE module provided by the present invention.

[0047] Figure 8 It is a schematic diagram of the structure of the MP-SE downsampling module provided by the present invention.

[0048] Fig. 9It is a schematic diagram of the structure of the lightweight dynamic convolution ODConv module provided by the present invention. DETAILED DESCRIPTION

[0049] In order to make the technical means, creative features, objectives and effects achieved by the present invention easy to understand, the present invention is further explained below with reference to specific diagrams.

[0050] Please refer to Figure 1 and Figure 2 As shown, the present invention provides a brain tumor detection method based on deep learning, comprising the following steps:

[0051] S1. Perform multi-scale feature extraction on the input magnetic resonance image through the backbone network and output three effective feature layers:

[0052] S11, the original magnetic resonance image is resized to 640×640×3 and first enters the backbone network. After four sequentially connected CBS operations, namely convolution (Conv), batch normalization (Batch Normalization) and activation function (Silu), the image size becomes 160×160×128;

[0053] S12, then enter a multi-branch stacking module (W-ELAN) for feature extraction, and the feature map size becomes 160×160×256; then perform MP downsampling operation and pass through a multi-branch stacking module to obtain an output feature layer of 80×80×512 (denoted as C3); finally, pass through MP downsampling operation and a multi-branch stacking module in turn to obtain two output feature layers of size 40×40×1024 (denoted as C4) and 20×20×1024 (denoted as C5), that is, the backbone network will eventually output effective feature layers of three scales: 80×80×512, 40×40×1024, and 20×20×1024.

[0054] As a specific embodiment, the multi-branch stacking module (W-ELAN) is composed of multiple CBSs, whose input and output feature sizes remain unchanged, the number of channels will change in the first two CBSs, and the following input channels are consistent with the output channels, and the output is the required channel after the last CBS. The new Weighted ELAN (W-ELAN) used in this application is different from the E-ELAN in the YOLOv7 network model architecture in that the purpose of W-ELAN is to promote dynamic feature fusion, which can solve the problem of relying on shallow features and lacking flexibility in adapting to the needs of different tasks or data in YOLOv7. In the W-ELAN architecture, each feature map will apply a weight learned in training when the feature maps are fused, but in order to prevent different feature maps from corresponding to weights w iTo solve the problem of large changes, the learned weights need to be normalized in a certain way. Figure 3 As shown, the multi-branch stacking module includes two branches, one branch performs only one CBS operation, and the other branch is composed of five CBSs stacked in series. During the convolution, the CBS will consider the contribution value of each feature map to assign weights. Finally, the results of the two branches are concatenated on the channel, and the feature map is output after another CBS operation.

[0055] As a specific embodiment, the weight allocation of each feature map is defined as follows:

[0056]

[0057] Among them, w i is the normalized feature map weight value, w' i is the weight value of each feature map learned by the network during training, j is the current feature map number, N is the total number of feature maps, and ξ is a fixed value to prevent the denominator from being zero. During training, the network will automatically assign a larger weight to a feature that contributes more to the final prediction of the network, and vice versa, a smaller weight will be assigned.

[0058] As a specific example, please refer to Figure 4 As shown, the MP downsampling includes two branches, one branch consists of a maximum pooling layer and a CBS operation, and the other branch consists of two CBSs in series. Finally, the results of the two branches are spliced ​​on the channel and then output. The difference between the MP downsampling in the backbone network (MP1) and the MP downsampling in the neck network (MP2) is mainly the change in the ratio of the number of channels.

[0059] S2. The three effective feature layers output by the backbone network are enhanced through feature fusion through the neck network to obtain three feature output layers of different sizes:

[0060] S21, the three effective feature layers output by the backbone network are respectively subjected to a CBS operation once, and then enter the Squeeze-Excitation (SE) module to suppress the interference of irrelevant features, that is, the three effective feature layers of size 80×80×512, 40×40×1024, and 20×20×1024 are first subjected to a CBS operation once, and then enter the Squeeze-Excitation module;

[0061] S22, then the feature layer of size 20×20×1024 enters the SPPCSPC module and RFE module in turn to increase the receptive field of the feature map, and changes the channel from 1024 to 512, that is, the size becomes 20×20×512 and is recorded as P5; then the UP upsampling operation is performed to obtain an output of size 40×40×256, and then the output and the feature layer of size 40×40×512 after sequentially passing through the CBS operation, squeezing and excitation module are spliced ​​and fused, and the output size becomes 40×40×512; then the multi-branch stacking module operation is performed once, and the size becomes 40×40×256 and is recorded as P4, and then the UP upsampling operation is performed and the feature layer of size 80×80×256 after sequentially passing through the CBS operation, squeezing and excitation module is spliced ​​and fused, and the output size becomes 80×80×256; finally, the multi-branch stacking module feature extraction is performed once, and the output of size 80×80×128 is obtained and is recorded as P3;

[0062] S23, the fused feature layers P3 and P4, as well as the feature layer of size 20×20×1024 (denoted as SE_out) after CBS operation, squeezing and excitation module are fused from bottom to top. Specifically, the feature layer P3 is first concatenated with the feature layer P4 after passing through the MP-SE downsampling module, and then subjected to a multi-branch stacking module feature extraction to obtain an output of size 40×40×256, denoted as P4_out; then the MP downsampling operation is performed and concatenated with the feature layer SE_out of size 20×20×1024 that enters the SPPCSPC module. A residual connection is made here to retain more original information, namely SE_out. Finally, the feature layer P3 is subjected to a multi-branch stacking module feature extraction to obtain an output of size 20×20×512, denoted as SE_out'. Thus, three feature output layers of different sizes, P3, P4_out and SE_out', are obtained.

[0063] As a specific example, please refer to Figure 5 As shown, the squeeze and excitation (SE) module in step S21 includes two operations, squeeze and excitation. The importance of each channel of the feature map is obtained through automatic learning, and then a weight value is assigned to each feature using the importance, so that the neural network focuses on certain feature channels. The specific operation of the SE module is as follows: first, the input feature U is passed through an F sq The squeezing operation globally aggregates the feature map in the spatial dimension H×W, which is a global average pooling operation. Thus, a feature map with c channels, H image height, and W image width is compressed into a C channel 1×1 channel descriptor to represent the global information. The specific calculation formula is as follows:

[0064]

[0065] Among them, Z C represents a scalar value representing the average value of the feature map across all spatial positions in the Cth channel; i represents the vertical coordinate of the feature map; j represents the horizontal coordinate of the feature map;

[0066] Next is F ex The excitation operation consists of two fully connected layers. This operation performs a connection transformation on the result obtained after squeezing the global pool. The channel descriptor after the two transformations is recorded as S C , the specific calculation formula is as follows:

[0067] S C =F ex (Z,W)=σ(W 2 δ(W 1 (Z)))

[0068] Among them, W 1 The representative size is R C×C / r The fully connected layer, W 2 The representative size is R C / r×C The fully connected layer, r is the channel bottleneck parameter, σ is the sigmoid function, and δ is the ReLU function;

[0069] Finally, the final output of the entire squeeze and excitation module is obtained by rescaling the transformation output U:

[0070] As a specific example, please refer to Figure 6 As shown, the SPPCSPC module in step S22 includes two branches, one branch is composed of three CBS series, three maximum pooling layers in parallel, a splicing operation and two CBS series stacked in sequence, and the other branch only performs a CBS operation once. Finally, the results of the two branches are spliced ​​on the channel and output after another CBS operation.

[0071] As a specific example, please refer to Figure 7As shown, in the stage of extracting features by the convolutional network in step S22, due to the different ranges of the receptive field of the convolutional layer, the dependency between distant pixels of the image captured by the convolutional network is often affected. Therefore, in order to make full use of the receptive field in the feature map, the RFE module is used in this application. Specifically, the RFE module includes three parallel branches, each of which is composed of a 1×1 convolution, a 3×3 hole convolution and another 1×1 convolution connected in series in sequence. After the input of each branch is weighted and summed with the output of another 1×1 convolution in each branch, the final results of the three branches are converged and weighted on the average pooling layer and output, and the expansion rates of the 3×3 hole convolutions in the three parallel branches are 1, 2, and 3, respectively. In this embodiment, the RFE module includes two parts: multi-branch hole convolution and convergence weighting. The multi-branch part uses 1, 2, and 3 as the expansion rates of different hole convolutions, respectively, and all use a fixed 3×3 convolution kernel, sharing weights to reduce the amount of parameters, while having different receptive fields.

[0072] As a specific example, please refer to Figure 8 As shown, the MP-SE downsampling module in step S23 includes two branches, one branch is formed by the maximum pooling layer and CBS connected in series, and the other branch is formed by the squeeze and excitation (SE) module and CBS connected in series, and finally the results of the two branches are spliced ​​on the channel and output.

[0073] S3, the three feature output layers P3, P4_out and SE_out' finally obtained by the head network (Head) detection head to neck network, respectively use lightweight dynamic convolution ODConv to adjust the number of channels, and then use 1×1 convolution Conv to predict the brain tumor target, and output the prediction result image containing the brain tumor category, bounding box and confidence.

[0074] As a specific embodiment, since the original RepConv module of the YOLOv7 head network has parameter redundancy, RepConv allows convolution kernels to be reused in different stages. Although this reduces the number of model parameters, it reduces the diversity of feature expression. ODConv can directly reduce the number of channels and is more efficient. Even if only one convolution kernel is used, it can still achieve performance comparable to or even better than the existing multi-core dynamic convolution. Therefore, this application uses the new dynamic convolution ODConv in the final head network. For details, please refer to Fig. 9 As shown, the calculation formula of the lightweight dynamic convolution ODConv in step S3 is as follows:

[0075] y=(a w1 ⊙a i1 ⊙a c1 ⊙a s1 ⊙W 1 +...+awi ⊙a fi ⊙a ci ⊙a si ⊙W i +...+a wn ⊙a fn ⊙a cn ⊙a sn ⊙W n )*x

[0076] Among them, a wi ∈R represents the convolution kernel W i The attention scalar, and a si ∈R k×k 、a si ∈R cin and a fi ∈R cout Represents the convolution kernel W respectively. i The newly introduced three types of attention along the spatial dimension, input channel dimension, and output channel dimension, ⊙ represents the multiplication operation between different dimensions in the convolution kernel space, and x represents the input feature layer.

[0077] As a specific embodiment, in order to solve the problems of complex and inaccurate loss function calculation, high sensitivity to parameters and difficulty in training in the existing network model, the loss function used in the network model of this method is a new bounding box similarity metric MPDIoU, L MPDIoU The loss function is calculated as follows:

[0078]

[0079]

[0080]

[0081] L MPDIoU =1-MPDIoU

[0082] in, Represents the coordinates of the upper left and lower right corners of the predicted bounding box, represents the coordinates of the upper left corner and the lower right corner of the ground truth bounding box, IOU represents the intersection over union ratio, i.e., the ratio of the intersection to the union between the predicted bounding box and the true bounding box, h represents the height of the input image, and w represents the width of the input image. MPDIoU The loss function is calculated accurately, so that the learned features are more effective for detection tasks and the model can easily achieve higher detection accuracy.

[0083] In order to better reflect the technical effect of the brain tumor detection method based on deep learning provided by the present invention, the present method is compared with the current advanced brain tumor detection methods in the following Table 1, and the best results are shown in bold.

[0084] Table 1 Comparison results between this method and current advanced brain tumor detection methods

[0085]

[0086] As can be seen from Table 1, the detection accuracy of the detector of the present invention and the original YOLOv7 model on the same MR brain tumor image dataset Br35H is AP 50 The AP 50：95 At the same time, the detection accuracy of the new SAOD-YOLO detector also exceeds the current 2023 SOTA model RCS-YOLO, in AP 50 The AP 50：95 It increased by 2.1%.

[0087] Compared with the prior art, the brain tumor detection method based on deep learning provided by the present invention has the following advantages: 1) A squeeze and excitation (SE) module and a multi-branch stacking module (W-ELAN) are added after the three effective feature layers output by the backbone network. The module automatically learns the importance of each channel in the feature map and assigns a weight value to each feature channel based on the importance, so that the neural network focuses on useful features, suppresses the interference of irrelevant features, and improves the generalization ability of the model; 2) MP-SE and RFE modules are constructed during pooled convolution (MP) downsampling to prevent the loss of detail information in the original image, thereby improving the model detection effect; 3) The detection head convolution of the head network uses a lightweight dynamic convolution ODConv module for convolution processing, thereby reducing the computational complexity; 4) The network model of the entire method uses L MPDIoU The loss function is used to solve the problem of inaccurate calculation, so that the model can achieve higher detection accuracy.

[0088] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solution of the present invention can be modified or replaced by equivalents without departing from the purpose and scope of the technical solution of the present invention, which should be included in the scope of the claims of the present invention.

Claims

1. A brain tumor detection method based on deep learning, characterized in that: The following steps are involved: S1. Perform multi-scale feature extraction on the input magnetic resonance image through the backbone network and output three effective feature layers: S11, the original magnetic resonance image is adjusted to a size of 640×640×3 and first enters the backbone network. After four sequentially connected CBS operations, namely convolution, batch normalization and activation function operations, the image size becomes 160×160×128; S12, then enter a multi-branch stacking module for feature extraction, and the feature map size becomes 160×160×256; then perform MP downsampling operation and pass through a multi-branch stacking module to obtain an output feature layer of 80×80×512; Finally, after MP downsampling operation and a multi-branch stacking module, two output feature layers with sizes of 40×40×1024 and 20×20×1024 are obtained respectively; S2. The three effective feature layers output by the backbone network are enhanced through feature fusion by the neck network to obtain three feature output layers of different sizes: S21, the three effective feature layers output by the backbone network are respectively subjected to a CBS operation, and then enter the squeezing and excitation module to suppress the interference of irrelevant features; S22, then the feature layer of size 20×20×1024 enters the SPPCSPC module and RFE module in turn to increase the receptive field of the feature map, and changes the channel from 1024 to 512, that is, the size becomes 20×20×512 and is recorded as P5; then the UP upsampling operation is performed to obtain an output of size 40×40×256, and then the output and the feature layer of size 40×40×1024 after sequentially passing through the CBS operation, squeezing and excitation module are spliced ​​and fused, and the output size becomes 40×40×512; then the multi-branch stacking module operation is performed once, and the size becomes 40×40×256 and is recorded as P4, and then the UP upsampling operation is performed and the feature layer of size 80×80×512 after sequentially passing through the CBS operation, squeezing and excitation module is spliced ​​and fused, and the output size becomes 80×80×256; finally, the multi-branch stacking module feature extraction is performed once, and the output of size 80×80×128 is obtained and is recorded as P3; S23, the fused feature layers P3 and P4, and the feature layer of size 20×20×1024 after sequentially passing through the CBS operation, the squeezing and excitation module, are fused from bottom to top, specifically, the feature layer P3 is first passed through the MP-SE downsampling module and then spliced ​​and fused with the feature layer P4, and then subjected to a multi-branch stacking module feature extraction to obtain an output of size 40×40×256, recorded as P4_out; then the MP downsampling operation is performed and the feature layer of size 20×20×1024 entering the SPPCSPC module is spliced ​​and fused, and finally subjected to a multi-branch stacking module feature extraction to obtain an output of size 20×20×512, recorded as SE_out', so that three feature output layers of different sizes, P3, P4_out and SE_out', are obtained; S3, the three feature output layers P3, P4_out and SE_out' finally obtained by the head network detection head to neck network, respectively use lightweight dynamic convolution ODConv to adjust the number of channels, and then use 1×1 convolution Conv to predict the brain tumor target, and output the prediction result image containing the brain tumor category, bounding box and confidence.

2. The brain tumor detection method based on deep learning according to claim 1, characterized in that: The multi-branch stacking modules in steps S1 and S2 each include two branches, one branch performs only one CBS operation, and the other branch is formed by stacking five CBSs in series. During the convolution, the CBS considers the contribution value of each feature map to assign weights. Finally, the results of the two branches are concatenated on the channel and then output after one CBS operation.

3. The brain tumor detection method based on deep learning according to claim 2, characterized in that: The weight assignment of each feature map is defined as follows: Among them, w i is the normalized feature map weight value, w' i is the weight value of each feature map learned by the network during training, j is the current feature map number, N is the total number of feature maps, and ξ is a fixed value to prevent the denominator from being zero.

4. The brain tumor detection method based on deep learning according to claim 1, characterized in that: The MP downsampling in steps S12 and S23 includes two branches, one branch consists of a maximum pooling layer and a CBS operation, and the other branch is composed of two CBSs in series. Finally, the results of the two branches are concatenated on the channel and then output.

5. The brain tumor detection method based on deep learning according to claim 1, characterized in that: The squeezing and excitation module in step S21 includes two operations: squeezing and excitation. Specifically, the input feature U is firstly transformed into an F sq The squeezing operation globally aggregates the feature map in the spatial dimension H×W, which is a global average pooling operation. Thus, a feature map with c channels, H image height, and W image width is compressed into a channel descriptor with C channels and 1×1. The specific calculation formula is as follows: Among them, Z C represents a scalar value representing the average value of the feature map across all spatial positions in the Cth channel; i represents the vertical coordinate of the feature map; j represents the horizontal coordinate of the feature map; Next is F ex The excitation operation consists of two fully connected layers. This operation performs a connection transformation on the result obtained after squeezing the global pool. The channel descriptor after the two transformations is recorded as S C , the specific calculation formula is as follows: S C =F ex (Z,W)=σ(W2δ(W1(Z))) Among them, W1 represents the size of R C×C / r The fully connected layer, W2 represents the size of R C / r×C The fully connected layer, r is the channel bottleneck parameter, σ is the sigmoid function, and δ is the ReLU function; Finally, the final output of the entire squeeze and excitation module is obtained by rescaling the transformation output U:

6. The brain tumor detection method based on deep learning according to claim 1, characterized in that: The SPPCSPC module in step S22 includes two branches, one branch is composed of three CBS series, three maximum pooling layers in parallel, a splicing operation and two CBS series stacked in sequence, and the other branch only performs one CBS operation. Finally, the results of the two branches are spliced ​​on the channel and output after another CBS operation.

7. The brain tumor detection method based on deep learning according to claim 1, characterized in that: The RFE module in step S22 includes three parallel branches, each branch is composed of a 1×1 convolution, a 3×3 dilated convolution and another 1×1 convolution connected in series in sequence, and the input of each branch is weighted and summed with the output of another 1×1 convolution in each branch, and the final results of the three branches are converged and weighted on the average pooling layer and then output, and the expansion rates of the 3×3 dilated convolutions in the three parallel branches are 1, 2, and 3, respectively.

8. The brain tumor detection method based on deep learning according to claim 1, characterized in that: The MP-SE downsampling module in step S23 includes two branches, one branch is formed by connecting the maximum pooling layer and CBS in series, and the other branch is formed by connecting the squeeze and excitation module and CBS in series. Finally, the results of the two branches are spliced ​​on the channel and then output.

9. The brain tumor detection method based on deep learning according to claim 1, characterized in that: The calculation formula of the lightweight dynamic convolution ODConv in step S3 is as follows: and (a) w1 ⊙a i1 ⊙a c1 ⊙a s1 ⊙W1+...+a wi ⊙a fi ⊙a ci ⊙a si ⊙W i +...+a wn ⊙a fn ⊙a cn ⊙a sn ⊙W n )*x Among them, a wi ∈R represents the convolution kernel W i The attention scalar, and a si ∈R k×k 、a si ∈R cin and a fi ∈R cout Represents the convolution kernel W respectively. i The newly introduced three types of attention along the spatial dimension, input channel dimension, and output channel dimension, ⊙ represents the multiplication operation between different dimensions in the convolution kernel space, and x represents the input feature layer.

10. The brain tumor detection method based on deep learning according to claim 1, characterized in that: The loss function used in the network model of the method is a new bounding box similarity metric MPDIoU, L MPDIoU The loss function is calculated as follows: L MPDIoU =1-MPDIoU in, Represents the coordinates of the upper left and lower right corners of the predicted bounding box, It represents the coordinates of the upper left and lower right corners of the ground truth bounding box, IOU represents the intersection-over-union ratio, that is, the ratio of the intersection to the union between the predicted bounding box and the true bounding box, h represents the height of the input image, and w represents the width of the input image.

Images