A detection method for PET / CT medical images

Abstract

The present invention discloses a detection method for PET / CT medical images, comprising the following steps: acquiring a target PET image, a target CT image, and a target PET / CT image; extracting features from the target PET image, target CT image, and target PET / CT image using a backbone network based on the CL-YOLOv5 model, and fusing all extracted features to obtain fused features; enhancing the fused features using the feature enhancement neck of the CL-YOLOv5 model to obtain enhanced features; and processing the enhanced features using the predicted head of the CL-YOLOv5 model to obtain a segmentation feature map. This method fully extracts features from multimodal PET / CT medical images using the CL-YOLOv5 model, improving the detection accuracy of multimodal PET / CT images and providing an important basis for understanding medical images.

Description

Technical Field

[0001] The present invention belongs to the technical field of image detection, and in particular to a detection method for PET / CT medical images. Background Art

[0002] Medical image detection primarily involves automatic semantic segmentation of medical images, which is crucial for understanding them. Positron emission tomography (PET) images typically have low spatial resolution, blurred feature map edges, and low semantic segmentation accuracy. Computed tomography (CT) imaging provides tomographic images with high density resolution and clear anatomical relationships, but it is difficult to extract all features.

[0003] PET / CT images fully integrate the enhanced capabilities of CT images, physiological and anatomical information, and the clear display of features in PET images. The continuous development, combination, and research of single-modality medical imaging technologies, such as PET, CT, X-rays, magnetic resonance imaging (MRI), and single-photon emission computed tomography (SPECT), have led to the increasingly widespread application of multimodal technologies, providing a crucial foundation for understanding medical images.

[0004] Compared with single-modality medical images, multimodal medical images contain richer feature information, and PET / CT images are difficult to fully integrate the advantageous information of CT images and PET images, making it difficult to fully utilize and automatically analyze the cross-modal feature information in multimodal images.

[0005] Therefore, how to make full use of the semantic correlation of cross-modal images and fully extract the features in multimodal PET / CT medical images, thereby improving the detection accuracy of multimodal PET / CT images, has become a key issue in current research. Summary of the Invention

[0006] In view of the above problems, the present invention provides a detection method for PET / CT medical images that solves at least some of the above technical problems.

[0007] An embodiment of the present invention provides a detection method for PET / CT medical images, comprising:

[0008] S1. Acquire a target PET image, a target CT image, and a target PET / CT image;

[0009] S2. A backbone network based on the CL-YOLOv5 model extracts features from the target PET image, the target CT image, and the target PET / CT image, respectively, and fuses all the extracted features to obtain fused features;

[0010] S3. Perform feature enhancement processing on the fused feature through the feature enhancement neck of the CL-YOLOv5 model to obtain enhanced features;

[0011] S4. Process the enhanced features through the prediction head of the CL-YOLOv5 model to obtain a segmentation feature map.

[0012] Furthermore, in step S2, the backbone network includes three branches and four stages;

[0013] The three branches include a PET branch, a CT branch, and a PET / CT branch, corresponding to the target PET image, the target CT image, and the target PET / CT image, respectively;

[0014] The four stages include: stage one, stage two, stage three and stage four; each stage is implemented by a dual-branch lightweight block and a cross-modal interactive enhancement block; the dual-branch lightweight block includes recursive reparameterized convolution and ACON loss function;

[0015] Both the first and fourth stages include 3 layers of recursive reparameterized convolution, ACON loss function and cross-modal interactive enhancement block;

[0016] Both stage 2 and stage 3 include 8 layers of recursive reparameterized convolution, ACON loss function and cross-modal interactive enhancement block.

[0017] Furthermore, the first stage specifically includes the following steps:

[0018] Slice the image corresponding to each branch separately to obtain the slicing results of the three branches;

[0019] Using 1×1 multi-parameter convolution to extract features from the slice results of the three branches respectively, to obtain the first initial feature maps of the three branches;

[0020] Performing feature extraction again on the first initial feature maps of the three branches through the three-layer recursive reparameterized convolution in the stage one to obtain the stage one three-layer convolution feature maps of the three branches;

[0021] The three-layer convolution feature map of the stage one is compensated by the ACON loss function in the stage one to obtain the stage one two-branch output features of the three branches;

[0022] Through the cross-modal interactive enhancement block in the stage one, the stage one double-branch output features of the three branches are fused to obtain the stage one fused features; the stage one fused features are divided into three feature maps with the same number of channels, and are passed to the three branches of the stage two as the second initial feature maps.

[0023] Furthermore, the second stage specifically includes the following steps:

[0024] Performing feature extraction again on the second initial feature maps of the three branches through the 8-layer recursive reparameterized convolution in the second stage to obtain the 8-layer convolution feature maps of the three branches in the second stage;

[0025] The 8-layer convolutional feature map of the second stage is compensated by the ACON loss function in the second stage to obtain the two-branch output features of the third branch;

[0026] The cross-modal interactive enhancement block in the stage 2 is used to fuse the dual-branch output features of the three branches in the stage 2 to obtain the fused features of the stage 2; the fused features of the stage 2 are divided into three feature maps with the same number of channels, which are passed to the three branches of the stage 3 as the third initial feature map.

[0027] Furthermore, the third stage specifically includes the following steps:

[0028] Performing feature extraction again on the third initial feature maps of the three branches through the three-layer recursive reparameterized convolution in the third stage to obtain the three-layer convolution feature maps of the three branches in the third stage;

[0029] The three-layer convolutional feature map of the stage three is compensated by the ACON loss function in the stage three to obtain the stage three dual-branch output features of the three branches;

[0030] The stage three dual-branch output features of the three branches are fused through the cross-modal interactive enhancement block in the stage three to obtain the stage three fused features; the stage three fused features are divided into three feature maps with the same number of channels, which are passed to the three branches of the stage four as the fourth initial feature map.

[0031] Furthermore, the fourth stage specifically includes the following steps:

[0032] Performing feature extraction again on the fourth initial feature maps of the three branches through the 8-layer recursive reparameterized convolution in the fourth stage to obtain the 8-layer convolution feature maps of the three branches in the fourth stage;

[0033] The 8-layer convolutional feature map of the stage 4 is compensated by the ACON loss function in the stage 4 to obtain the stage 4 dual-branch output features of the three branches;

[0034] The stage four dual-branch output features of the three branches are fused through the cross-modal interactive enhancement block in the stage four to obtain the stage four fused features.

[0035] Furthermore, the step S3 specifically includes:

[0036] Feature enhancement neck in the CL-YOLOv5 model:

[0037] Performing 1×1 re-parameterized convolution processing on the stage 2 fusion features, stage 3 fusion features, and stage 4 fusion features respectively to obtain corresponding stage 2 features, stage 3 features, and stage 4 features;

[0038] The four features of the stage are processed using a three-layer recursive re-parameterized convolution to obtain the first convolution feature;

[0039] Performing upsampling processing on the first convolution feature to obtain a first sampling result;

[0040] After concatenating the first sampling result and the third feature of the stage, a three-layer recursive reparameterized convolution is used to obtain the second convolution feature;

[0041] Performing upsampling on the second convolution feature to obtain a second sampling result;

[0042] After concatenating the second sampling result and the stage two feature, a three-layer recursive reparameterized convolution is used to process the result to obtain a first enhanced feature;

[0043] After performing 1×1 reparametric convolution on the first enhanced feature, concatenating it with the second convolution feature, and processing the concatenated result using 3 layers of recursive reparametric convolution to obtain a second enhanced feature;

[0044] After the second enhanced feature is processed by 1×1 reparametric convolution, it is spliced ​​with the fourth feature of the stage, and the splicing result is processed by 3 layers of recursive reparametric convolution to obtain the third enhanced feature.

[0045] Furthermore, the step S4 specifically includes:

[0046] In the prediction head of the CL-YOLOv5 model:

[0047] 1×1 multi-parameter convolution processing is performed on the first enhanced feature, the second enhanced feature and the third enhanced feature respectively, and the three 1×1 multi-parameter convolution processing results are spliced ​​to obtain a segmentation feature map.

[0048] Furthermore, each layer of recursive re-parameterized convolution specifically includes the following steps:

[0049] Use 1×1 multi-parameter convolution to process the input feature map with c channels and obtain an output feature map with 2c channels;

[0050] Slicing the output feature map to obtain a first feature map and a second feature map; the number of channels of the first feature map and the second feature map are 1 / 4c and 7 / 4c, respectively;

[0051] The second feature map is processed by a 7×7 depthwise convolution, and the convolved second feature map is sliced ​​to obtain a third feature map, a fourth feature map, and a fifth feature map; the number of channels of the third feature map, the fourth feature map, and the fifth feature map are 1 / 4c, 1 / 2c, and c, respectively;

[0052] Multiplying the third feature map by the first feature map and performing a 1×1 multi-parameter convolution process to obtain a first output feature map with a channel number of 1 / 2c;

[0053] Multiplying the first output feature map by the fourth feature map and performing a 1×1 multi-parameter convolution process to obtain a second output feature map with a channel number c;

[0054] Multiply the second output feature map by the fifth feature map and perform a 1×1 reparametric convolution process to obtain a final output feature map with c channels.

[0055] Furthermore, the cross-modal interactive enhancement block specifically includes the following steps:

[0056] The double-branch output features of the three branches are concatenated and then subjected to 1×1 re-parameterized convolution to obtain a feature map with a channel number of c.

[0057] The Relu activation function is used to process the feature map No. 1 to obtain matrix Q, matrix K and matrix V;

[0058] Multiply the matrix Q, the matrix K, and the matrix V pixel by pixel to obtain a second feature map;

[0059] After residual addition of the first feature map and the second feature map, the third feature map is obtained by sequentially performing 3×3 depthwise convolution, BN function, ReLU activation function, 1×1 reparametric convolution and residual addition processing;

[0060] The second feature map and the third feature map are concatenated and then subjected to 1×1 re-parameterized convolution processing to obtain a fourth feature map;

[0061] The four feature maps are cut and reorganized to obtain three feature maps with c channels each, which are sequentially transmitted to the PET branch, the CT branch, and the PET / CT branch.

[0062] Compared with the prior art, the detection method for PET / CT medical images described in the present invention has the following beneficial effects:

[0063] 1. This paper fully extracts features from multimodal PET / CT medical images through the CL-YOLOv5 model, improves the detection accuracy of multimodal PET / CT images, and provides an important basis for understanding medical images.

[0064] 2. The present invention proposes a three-branch backbone network to extract and fully utilize the functional and anatomical information of PET, CT and PET / CT.

[0065] 3. The present invention designs a cross-modal interactive enhancement block to fully learn the semantic relevance of multimodal images, uses the Transformer with cosine reweighting calculation method to learn the interdependence of long-range features, and interactively enhances the network's ability to extract features from multimodal images, thereby improving detection performance.

[0066] 4. The present invention proposes a dual-branch lightweight block. One branch adopts the bottleneck structure of the ACON activation function to reduce parameters while increasing network depth and robustness; the other branch adopts densely connected progressive reparameterized convolution to maximize the transmission of features in all layers of the network. The progressive reparameterized convolution uses progressive self-attention for spatial interaction, gradually increasing the channel width, avoiding the problem of excessive model complexity caused by spatial interaction of the self-attention mechanism, while learning richer multimodal features.

[0067] Other features and advantages of the present invention will be described in the following description, and in part will become apparent from the description, or will be understood by practicing the present invention. The purposes and other advantages of the present invention can be realized and obtained by the structures particularly pointed out in the written description, claims, and drawings.

[0068] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0069] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings:

[0070] Figure 1 A schematic diagram of the framework of a detection method for PET / CT medical images provided by an embodiment of the present invention.

[0071] Figure 2 Schematic diagram of the progressive reparameterized convolution structure provided by an embodiment of the present invention.

[0072] Figure 3 A schematic diagram of a dual-branch lightweight block structure provided in an embodiment of the present invention.

[0073] Figure 4 A schematic diagram of the cross-modal interactive enhancement block structure provided by an embodiment of the present invention.

[0074] Figure 5 Schematic diagram of a registered PET image, a target CT image, and a target PET / CT image provided by an embodiment of the present invention.

[0075] Figure 6 A schematic diagram of the visualization results of different modal detection of cross-modal semantic relevance provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0076] Exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments set forth herein. Rather, these embodiments are provided to enable a more thorough understanding of the present disclosure and to fully convey the scope of the present disclosure to those skilled in the art.

[0077] See also Figure 1 As shown, the embodiment of the present invention provides a detection method for PET / CT medical images, which specifically includes the following steps:

[0078] S1. Acquire a target PET image, a target CT image, and a target PET / CT image;

[0079] S2. The backbone network based on the CL-YOLOv5 model extracts features from the target PET image, target CT image, and target PET / CT image respectively, and fuses all the extracted features to obtain fused features;

[0080] S3, enhance the fusion features through the feature enhancement neck of the CL-YOLOv5 model to obtain enhanced features;

[0081] S4. Process the enhanced features through the prediction head of the CL-YOLOv5 model to obtain the segmentation feature map.

[0082] The above steps are described in detail below.

[0083] In the above steps S1-S2, the backbone network includes three branches and four stages; wherein:

[0084] The three branches include: a PET branch, a CT branch, and a PET / CT branch, which correspond to the target PET image, the target CT image, and the target PET / CT image acquired in step S1, respectively;

[0085] The four stages include: stage 1, stage 2, stage 3 and stage 4; each stage is implemented by a dual-branch lightweight block and a cross-modal interactive enhancement block; one branch of the dual-branch lightweight block is recursive reparameterized convolution, and the other branch is the ACON loss function;

[0086] The above-mentioned stages 1 and 4 both include 3 layers of recursive reparameterized convolution, ACON loss function and cross-modal interactive enhancement blocks; stages 2 and 3 both include 8 layers of recursive reparameterized convolution, ACON loss function and cross-modal interactive enhancement blocks.

[0087] The above-mentioned stage 1 specifically includes the following steps:

[0088] The image corresponding to each branch is sliced ​​(Focus) respectively to obtain the slicing results of the three branches; the slicing results of the three branches are extracted by 1×1 reparametric convolution to obtain the first initial feature maps of the three branches; the first initial feature maps of the three branches are extracted again by the 3-layer recursive reparametric convolution in stage one to obtain the stage one 3-layer convolution feature map of the three branches; the stage one 3-layer convolution feature map is compensated by the ACON loss function in stage one to obtain the stage one dual-branch output features of the three branches; the stage one dual-branch output features of the three branches are fused by the cross-modal interactive enhancement block in stage one to obtain the stage one fusion features; the stage one fusion features are divided into three feature maps with the same number of channels, which are passed to the three branches of stage two as the second initial feature maps.

[0089] The above-mentioned Phase 2 specifically includes the following steps:

[0090] The second initial feature maps of the three branches are extracted again through the 8-layer recursive reparameterized convolution in the second stage to obtain the 8-layer convolution feature maps of the three branches in the second stage; the 8-layer convolution feature maps of the second stage are compensated by the ACON loss function in the second stage to obtain the dual-branch output features of the three branches in the second stage; the dual-branch output features of the three branches in the second stage are fused through the cross-modal interactive enhancement block in the second stage to obtain the fused features of the second stage; the fused features of the second stage are divided into three feature maps with the same number of channels, which are passed to the three branches in the third stage as the third initial feature maps.

[0091] The above-mentioned stage three specifically includes the following steps:

[0092] The third initial feature maps of the three branches are extracted again through the 3-layer recursive reparameterized convolution in stage 3 to obtain the stage 3 3-layer convolution feature maps of the three branches; the stage 3 3-layer convolution feature maps are compensated by the ACON loss function in stage 3 to obtain the stage 3 double-branch output features of the three branches; the stage 3 double-branch output features of the three branches are fused through the cross-modal interactive enhancement block in stage 3 to obtain the stage 3 fused features; the stage 3 fused features are divided into three feature maps with the same number of channels, which are passed to the three branches of stage 4 as the fourth initial feature maps.

[0093] The above-mentioned stage 4 specifically includes the following steps:

[0094] The fourth initial feature maps of the three branches are again extracted through the 8-layer recursive reparameterized convolution in stage four to obtain the stage four 8-layer convolution feature maps of the three branches; the stage four 8-layer convolution feature maps are compensated through the ACON loss function in stage four to obtain the stage four dual-branch output features of the three branches; the stage four dual-branch output features of the three branches are fused through the cross-modal interactive enhancement block in stage four to obtain the stage four fused features.

[0095] Next, we will explain the recursive reparameter convolution in the above content:

[0096] CNNs extract rich local features through connections between local adjacent pixels, demonstrating their superiority in image feature extraction. Channel and spatial attention mechanisms learn attention weights for feature maps, improving the model's modeling capabilities through additional channel and spatial interactions. The Transformer model, through its dot-product-based self-attention mechanism, can better capture global image information. Convolutional operations struggle to fully account for spatial interactions between features. Spatial attention mechanisms achieve this through additional convolutional calculations, improving the model's modeling capabilities and adding additional computation to learn the correlations between local features. However, they lack the ability to learn long-range feature dependencies. Although the Transformer achieves two consecutive spatial interactions through two consecutive matrix multiplications, enabling modeling of long-range feature dependencies, its two-fold complexity introduces a significant amount of additional computation, and the size of the input feature map significantly hinders its application, particularly in feature detection tasks requiring higher resolution. To this end, an embodiment of the present invention designs a progressive reparametric convolution based on a progressive method and gated reparametric convolution. The convolution-based design can avoid the quadratic complexity of self-attention, and progressively adopts self-attention spatial interaction to further improve the modeling ability. During the execution of spatial interaction, the channel width is gradually increased to achieve progressive interaction of limited complexity. The progressive reparametric convolution can effectively combine the advantages of Transformer and CNN, wherein the reparametric convolution uses a complex multi-branch structure to ensure the feature extraction ability of the model, and reconverts the parameters of the multi-branch structure model into another set of parameters with a simple structure to achieve lightweight and maintain the feature expression ability of the multi-branch structure. Since the progressive reparametric convolution is based on convolution design and has translation invariance, it can introduce beneficial inductive biases for image feature detection tasks. The schematic diagram of the progressive reparametric convolution structure is shown as follows. Figure 2 shown.

[0097] 1×1 reparametric convolution (RC1×1) fully extracts features through multi-scale and multi-branch structures, and then uses linear scaling to replace the nonlinear operation layer to improve the nonlinear expression ability of the model. First, two serial 1×1 convolutions are reparameterized in a serial structure to obtain a reparametric 1×1 convolution, and then the parallel structure is reparameterized with the parallel branches to obtain the final reparametric 1×1 convolution, reducing the dimension and reducing the amount of calculation while more fully integrating the features of each channel. For the input feature map X, the 1×1 reparametric convolution kernel It can be expressed as:

[0098]

[0099] Among them, S1 and S2 represent the linear scaling coefficients of the two branches, respectively, to maintain the nonlinear expression ability during dense unit training; the superscript W represents the kernel corresponding to the operation; Represents a 1×1 convolution kernel.

[0100] like Figure 2 As shown in the figure, the input feature map X with the number of channels c is first learned using 1×1 reparametric convolution (RC1×1) to obtain an output feature map with the number of channels 2c; the output feature map with the number of channels 2c is divided into a first feature map X1 and a second feature map X2 using a split operation (Split); the number of channels of the first feature map X1 and the second feature map X2 are 1 / 4c and 7 / 4c respectively; the calculation formula of the first feature map X1 and the second feature map X2 is expressed as:

[0101] X1,X2=Split(RC 1×1 (X)) (2)

[0102] Then, a 7×7 depthwise convolution (DW Conv) is used to learn the second feature map X2 with a channel number of 7 / 4c. The second feature map X2 is then divided into a third feature map X3, a fourth feature map X4, and a fifth feature map X5 with channel numbers of 1 / 4c, 1 / 2c, and c, respectively, using a slicing operation (Split). The 7×7 depthwise convolution enables the Transformer to calculate self-attention within a relatively large local window. To this end, the embodiment of the present invention introduces the convolution of the 7×7 receptive field into the progressive reparameterized convolution to maximize the ability to learn feature interactions and long-distance dependencies. The calculation formulas for the third feature map X3, the fourth feature map X4, and the fifth feature map X5 are expressed as follows:

[0103] X3,X4,X5=Split(DWConv 7×7 (X2)) (3)

[0104] Finally, the multiplication operation and 1×1 reparametric convolution are performed progressively, and the third progressive result is used as the final output. Specifically, the first feature map X1 and the third feature map X3 are multiplied and 1×1 reparametric convolution is performed to obtain the first output feature map with a channel number of 1 / 2c; the second output feature map and the fourth feature map X4 are multiplied and 1×1 reparametric convolution is performed to obtain the second output feature map with a channel number of c; the third output feature map and the fifth feature map X5 are multiplied and 1×1 reparametric convolution is performed to obtain the final output feature map X with a channel number of c. out , which is also the output feature map of the final progressive re-parameterized convolution module, and the final output feature map X out The calculation formula is

[0105] X out =RC 1×1 (X5×RC 1×1 (X4×RC 1×1 (X1×X3))) (4)

[0106] Next, we will explain the dual-branch lightweight block in the above content:

[0107] Deep learning models consume significant computational and storage resources, limiting the detection model's ability to extract image features. The lightweight, single-stage detection model, YOLO, combines candidate boxes and object recognition, and utilizes a dual-branch architecture to reduce parameters and improve efficiency. YOLOv5 employs parallel residual blocks and 1×1 convolutions to learn local feature information, but this lacks spatial interaction, and the continuous stacking of residual blocks also leads to a certain degree of feature redundancy. The dense connection approach utilizes feature reuse to concatenate the current layer with the previous layer in the channel direction. This concatenation of features from different layers increases the variance of the subsequent layer's input, improving parameter efficiency and avoiding the information flow disruption caused by element-by-element addition in residual blocks. This can further mitigate vanishing gradients and overfitting. The bottleneck architecture first compresses the number of channels in the input feature map using 1×1 convolutions, and then restores the number of channels in the output feature map using 1×1 convolutions after computation, reducing both model computation and parameter requirements. Based on this, in the embodiment of the present invention, a progressive lightweight block with a parallel dual-branch structure is set in the above-mentioned stages 1 to 4, namely, a dual-branch lightweight block. Figure 3 As shown in the figure: one branch uses densely connected progressive reparameterized convolution to enhance the feature transfer and feature reuse capabilities, maximize the information flow of features, reduce parameters and further fully learn the features of spatial interaction; the other branch uses a bottleneck structure to reduce the number of parameters and calculations, while increasing the network depth to improve network performance, and adopts the general form of the Swish function commonly used in YOLOv5 in the structure, a simple and effective ACON activation function, to improve the robustness of the model and better nonlinear expression capabilities.

[0108] in, Figure 3 (a) Schematic diagram of the structure of the dual-branch lightweight block in stage 1 and stage 4, corresponding to Figure 1 The logo in (b1); Figure 3 (b) shows the schematic diagram of the structure of the dual-branch lightweight block in stage 2 and stage 3, corresponding to Figure 1 The logo in (b2); Figure 3 (c) shows the schematic diagram of the ACON loss function structure in the dual-branch lightweight block, corresponding to Figure 1 The logo in (c);

[0109] like Figure 3 As shown in (a), Figure 3 (a) Three densely connected progressive reparameterized convolutions are used on the left, where the growth rate k is set to 32. The calculation formula of the output feature map Y1 of the first progressive reparameterized convolution (RRC) on the left branch is expressed as:

[0110] Y1=RRC(RC 1×1 (Y)) (5)

[0111] The calculation formula for the output feature map Y2 of the second progressive re-parameter convolution is

[0112] Y2=RRC(RC 1×1 (Y)×Y1) (6)

[0113] The calculation formula for the output feature map Y3 of the third progressive re-parameter convolution is

[0114] Y3=RRC(RC 1×1 (Y)×Y1×Y2) (7)

[0115] The right branch adopts the structure of the bottleneck ACON activation function. The bottleneck structure reduces the number of parameters while increasing the network depth to improve network performance. The number of output channels of the first progressive reparameterized convolution on the right branch is one-third of the number of output channels on the right branch. The calculation formula of the output feature map Y4 is:

[0116] Y4=RC 1×1 (X) (8)

[0117] The Relu activation function expression is max(x,0), where max is the maximum value function. The commonly used Swish function expression of YOLOv5 is:

[0118] swish(x)=x×sigmoid(b x ) (9)

[0119] Among them, Sigmoid is an S-type activation function, b is a learnable parameter, and Swish can be interpreted as a smooth approximation of ReLU. The general form of the ReLU activation function is Maxout, which is a layer of the neural network and can be regarded as a learnable piecewise linear function. The expression of the Maxout activation function is:

[0120] Maxout=max(wx+b) (10)

[0121] Where w and b are learnable parameters. By applying the same smooth approximation to them, we can get the ACON activation function. Swish is a special form of ACON. The ACON activation function structure is as follows: Figure 3 As shown in (c), for the input feature map Y4, the expression of the output feature map Y5 of the 3×3 depth convolution (DW3×3) branch of ACON is:

[0122] Y5=BN(DW 3×3 (Y4)) (11)

[0123] ACON's 3×3 maximum pooling (Max 3×3 ) The expression of the branch output feature map Y6 is

[0124] Y6=BN(MaxP3×3 (Y4)) (12)

[0125] Among them, BN is batch normalization, the step size of depth convolution and maximum pooling is 1, and then the BN result and Sigmoid result of the difference between the output feature maps Y5 and Y6 are calculated respectively. Finally, they are multiplied and added to the output feature map Y5 and BN. The output feature map Y7 of the ACON activation function is expressed as

[0126] Y7=BN(Y6-Y5)×Sigmoid(Y6-Y5)+Y5 (13)

[0127] like Figure 3 As shown in (a), the second progressive reparameterized convolution of the right branch is used to restore the number of channels of Y7, and the output feature maps of the left and right branches are concatenated to obtain Y8. The expression of Y8 is

[0128]

[0129] Y8 is input into the ACON activation function, where the stride of the depthwise convolution and maximum pooling is 2, and finally a three-layer dense structured dual-branch lightweight block output feature map is obtained.

[0130] like Figure 3 As shown in (b), the left branch employs eight densely connected progressive reparameterized convolutions, using the same computational approach as the three-layer dual-branch lightweight block. The right branch also employs the same computational approach as the three-layer dual-branch lightweight block. Eight-layer dual-branch lightweight blocks are used in the second and third stages of the model. The progressive lightweight block with a parallel dual-branch structure enhances feature transfer and reuse, maximizing the flow of feature information while reducing parameters and further learning spatially interactive features. The bottleneck structure also increases network depth, thereby improving performance. The ACON activation function, a general form of the Swish function, is simple and effective, enhancing the model's robustness and nonlinear expression capabilities.

[0131] The terms “left” and “right” in the above description are only for the convenience of explanation and do not limit the double-branch structure of the present invention.

[0132] Next, we will explain the cross-modal interactive enhancement block in the above content:

[0133] For feature detection and recognition, there is a lot of inconsistent information between multimodal images with different imaging mechanisms. Irrational use will lead to low detection accuracy. Existing multimodal detection methods usually compromise the processing of inconsistent information and cannot fully utilize the advantages of multimodal images. To this end, the embodiment of the present invention designs a cross-modal interactive enhancement block, whose structure is as follows: Figure 4 shown; Figure 4 (a) represents the cross-modal self-attention block; Figure 4 (b) represents the Transformer model;

[0134] This cross-modal interactive enhancement block fully learns the semantic relevance of cross-modal images, learns the interdependencies between multimodal features, and recalibrates them, interactively enhancing target features in multimodal images. The Transformer is used to simultaneously learn long-range feature correlations between the backbone network and the auxiliary network. To avoid excessive parameter and computational overhead, deep convolution is used to learn Transformer features. The two features are concatenated to form the output of the cross-modal interactive enhancement block. The output of the cross-modal interactive enhancement block is split and passed to the PET, CT, and PET / CT branches respectively. Furthermore, the computational complexity of the Softmax calculation method in the Transformer is the square of the sequence length. The Softmax is improved to retain only two characteristics: the non-negativity of the attention weight generation matrix and the nonlinear weight transformation used to amplify local attention weight values.

[0135] like Figure 4 As shown in (a), PET, CT and PET / CT data are spliced ​​and input into 1×1 reparametric convolution, and the features are simply fused and the number of channels is compressed c. The expression of the output feature map Z1 is:

[0136]

[0137] In the improvement of the Softmax calculation method, in order to ensure the non-negativity and nonlinear weight conversion of the matrix, the feature map Z1 is processed with a 1×1 convolution and then processed with a Relu activation function to obtain matrices Q, K, and V respectively. The matrix dot multiplication can be transformed into matrix pixel-by-pixel multiplication to reduce the complexity of the Transformer. The associative law can also be used to change the order of calculation. The calculation formula of the second feature map Z2 of the Transformer is:

[0138]

[0139] in K j Dimension; T represents the matrix transpose. Softmax can be removed through ReLU, but the result needs to be reweighted to generate weights. Reweighting requires the use of a splittable function, such as Softmax. The split-based cosine reweighting mechanism can fit the distribution, accelerate convergence, and stabilize the training process. In this embodiment of the present invention, cosine and sin are used to split and calculate the second feature map Z2. The calculation formula of the second feature map Z2 is transformed into:

[0140]

[0141] in, The hyperparameter M represents the length of the longest sequence in Q and K; i, j = 1...N represents the position; using the associative law to change the order of calculation, the calculation formula of the second feature map Z2 is transformed into

[0142]

[0143] After the residual of the second feature map Z2 output by Transformer is added to the residual of the first feature map Z1, the third feature map Z3 is obtained through 3×3 depth convolution, BN, ReLU, 1×1 reparameter convolution and residual connection.

[0144] Z3=RC 1×1 (Relu(BN(DW 3×3 (Z1+Z3)))) (19)

[0145] The two Transformer features with c channels and the convolution features are fused and input into the 1×1 reparametric convolution. Here, the 1×1 reparametric convolution and the initial 1×1 reparametric convolution also form a bottleneck structure to reduce the number of parameters and calculations. The final output channel number is 3c, and the calculation formula for the output fourth feature map Z4 is

[0146]

[0147] The fourth feature map Z4 is cut and reorganized into six parts, and then converted into three feature maps with c channels, which are passed to the PET branch, CT branch, and PET / CT branch respectively.

[0148] While maintaining the number of parameters, the cross-modal interactive enhancement block fully utilizes the semantic correlation of cross-modal images, integrates PET functional information and CT anatomical information, and fuses and interactively enhances the complementary information of the three modalities, thereby improving the model detection performance and enhancing the model's ability to focus on features.

[0149] In the above step S3, it specifically includes:

[0150] In the feature enhancement neck of the CL-YOLOv5 model: perform 1×1 reparametric convolution on the stage 2 fusion features, stage 3 fusion features and stage 4 fusion features respectively to obtain the corresponding stage 2 features, stage 3 features and stage 4 features; use 3 layers of recursive reparametric convolution to process the stage 4 features to obtain the first convolution feature; upsample the first convolution feature to obtain the first sampling result; after splicing the first sampling result and the stage 3 features, use 3 layers of recursive reparametric convolution to process them to obtain the second convolution feature; upsample the second convolution feature to obtain the second convolution feature. Upsampling is performed to obtain a second sampling result; the second sampling result and the stage two feature are spliced ​​together, and then processed by three-layer recursive reparametric convolution to obtain a first enhanced feature; the first enhanced feature is processed by 1×1 reparametric convolution, and then spliced ​​with the second convolution feature, and the splicing result is processed by three-layer recursive reparametric convolution to obtain a second enhanced feature; the second enhanced feature is processed by 1×1 reparametric convolution, and then spliced ​​with the stage four feature, and the splicing result is processed by three-layer recursive reparametric convolution to obtain a third enhanced feature.

[0151] In the above step S4, it specifically includes:

[0152] In the prediction head of the CL-YOLOv5 model: perform 1×1 reparametric convolution on the first enhanced feature, the second enhanced feature, and the third enhanced feature respectively, and concatenate the three 1×1 reparametric convolution results to obtain the segmentation feature map.

[0153] Next, the effects of the detection method for PET / CT medical images provided by the present invention will be described through specific embodiments.

[0154] 1. Multimodal datasets:

[0155] The experimental data set of the embodiment of the present invention is the registered PET, CT and PET / CT two-dimensional images, such as Figure 5 As shown, it is difficult to distinguish the density difference between target features and normal tissues in CT images, while the target feature areas in PET images are highly metabolized and appear bright, so multimodal images can better identify and locate target features. In an embodiment of the present invention, the final number of samples in the PET / CT multimodal dataset is 1,147 for each modality. There are two types of data files in the dataset: 1,147 JPG-format image files for each of the three modalities, with diverse sizes, angles, radiation measurement conditions, and backgrounds; and 1,147 XML-format image tag files that specify the precise location of the object in the image. In an embodiment of the present invention, the public dataset is re-divided into a training set, a validation set, and a test set in a 6:2:2 ratio for experiments, with 684 images in the training set, 222 images in the validation set, and 241 images in the test set.

[0156] 2. Test environment and parameter settings:

[0157] In this example, the experimental environment was set to a 64-bit Windows 11 Professional system with an embedded Ubuntu 18.04 LTS subsystem. The computer had 40GB of memory, an AMD Ryzen 53500X CPU processor, and an NVIDIA 2070Super graphics card for image processing acceleration. The network was built using the PyTorch framework. Optimization was performed using the SGD optimizer. Different network models were trained with the same parameters on the same dataset. The weights that achieved the minimum loss in the validation set were selected for evaluation on the test set.

[0158] In terms of parameter setting, to ensure that the model parameters can be updated quickly and converge to the global optimal point, the learning rate is decayed by 0.95 every 10 training cycles, and an initial learning rate of 0.01 is used on the PET / CT multimodal dataset. The weight decay value of the SGD optimizer after each training and gradient update is set to 1e-4, and a momentum of 0.9 is used to slow down the oscillation of the gradient descent and accelerate convergence. The training cycle of the PET / CT multimodal dataset is set to 300, and the training batch size is set to 16.

[0159] 3. Evaluation indicators:

[0160] In target detection, Intersection over Union (IoU) is a criterion for object detection accuracy, which is used to measure the similarity between the predicted bounding box and the true bounding box. The formula is as follows:

[0161]

[0162] Where B_gt represents the actual ground-truth region of the target; B_p represents the predicted region. Denoting these boxes as true targets or false targets yields four potential predictions: true positive (TP), false positive (FP), true negative (TN), and false negative (FN). If the IoU between the detection box and the ground-truth box is greater than a threshold of 0.5, the detection box is labeled as TP, indicating that both the prediction and the label are the target event; otherwise, it is labeled as FP. The performance of the model can usually be evaluated by precision and recall:

[0163]

[0164]

[0165] Precision and recall affect each other and cannot be used directly to evaluate detection accuracy. Therefore, the embodiment of the present invention introduces average precision (AP) to represent detection accuracy. AP refers to the average recall of peak detections in the range of 0 to 1, that is, the area under the PR curve of the target feature is calculated. A higher AP means a higher accuracy of the network:

[0166]

[0167] Where n is the total number of images in the dataset, precision(k) is the precision at the cutoff point of k images, and ΔRecall(k) is the difference in recall between the cutoff point k-1 and the cutoff point k. mAP is calculated by averaging the area under the PR curve for all categories and averaging the AP for each category again to get mAP:

[0168]

[0169] Where C is the number of target categories. The F1 score can also consider both precision and recall, and the calculation formula is:

[0170]

[0171] FPS stands for frame rate per second, which represents the number of images that can be detected per second and is often used to evaluate the speed of object detection.

[0172] 4. Experiment on different modal detection of cross-modal semantic relevance:

[0173] The embodiment of the present invention uses four groups of experiments to verify the advantages of the auxiliary network proposed in the present invention in learning cross-modal semantic relevance. The entire network adopts dual-branch lightweight blocks and multimodal splicing, and then designs four groups of experiments based on the main network input PET / CT images. In experiment one, both branches of the auxiliary network input PET / CT images; in experiment two, the auxiliary network inputs CT and PET / CT images; in experiment three, the auxiliary network inputs PET and PET / CT images; in experiment four, the auxiliary network inputs PET and CT images. The results of the four groups of experimental detection indicators are shown in Table 1; the embodiment of the present invention selects five multimodal images containing multiple lung foreign bodies to better compare the four groups of experiments, and the visualization results are shown in Table 1. Figure 6 shown.

[0174] Table 1 Specific results of cross-modal semantic relevance detection in different modalities

[0175] Detection Model Accuracy Recall mAP F1 score Single mode 0.9566 0.9160 0.9448 0.9359 CT dual modality 0.9609 0.9186 0.9486 0.9393 PET dual modality 0.9595 0.9326 0.9507 0.9458 Trimodal 0.9652 0.9354 0.9558 0.9501

[0176] It can be clearly seen from Table 1 that in Experiment 1, both branches of the auxiliary network have the lowest precision, recall, mAP and F1 score results for PET / CT. Due to the complex characteristics of lung foreign bodies and their connection with normal tissues, Figure 6 It can be seen that there are indeed obvious defects, such as the failure to identify some foreign bodies in the third row. In Experiment 2, the accuracy of the CT and PET / CT images with anatomical information was significantly improved. Figure 6 It can be seen that the feature edges can be better identified. In Experiment 3, the recall rate of the PET and PET / CT image results with functional information was significantly improved. Figure 6 It can be seen that foreign objects can be better captured and identified, but the detection frame boundary is slightly too small, and the foreign object features cannot be accurately detected. Experiment 4, using a trimodal network with PET / CT as the backbone network and PET and CT as auxiliary networks to achieve the highest performance, can well complement multimodal semantic information, and the four indices are improved by 0.89%, 2.12%, 1.16% and 1.52% respectively compared with PET / CT single modality. Learning cross-modal complementary semantic information can improve detection performance and enhance the model's ability to focus on foreign object features. Figure 6 It can effectively identify the edges of foreign body features with a high confidence level. In medical image detection tasks, making full use of the semantic relevance of cross-modal images can effectively improve the model's recognition ability and detection accuracy.

[0177] 5. Ablation experiment:

[0178] To evaluate the effectiveness of the YOLOv5 model structure in the present invention, four sets of ablation experiments were conducted to illustrate the impact of progressive reparameterized convolution, dual-branch lightweight blocks, multimodal splicing, and cross-modal interactive enhancement blocks on the experimental results. The specific results are compared in Table 2.

[0179] Table 2 Comparison results of ablation experiments on the lung foreign body PET / CT multimodal dataset

[0180]

[0181]

[0182] In Experiment 1, resource consumption was reduced by nearly 60%, the running speed and FPS increased by 5% and 21%, and the mAP and F1 scores increased by 1.96% and 1.32%. The channel width and multi-branch re-parameterization were gradually increased to achieve lightweight, progressive self-attention spatial interaction to learn richer multimodal features, and improve the accuracy of multimodal lung foreign body detection tasks at a lower computational cost; in Experiment 2, the number of parameters and computational complexity were reduced by 83% and 86%, the running speed and FPS increased by 12% and 20%, and the mAP and F1 increased by 0.49% and 0.57%. A lightweight dual-branch structure was adopted, which greatly reduced resource consumption. The reuse of densely connected features maximized the flow of feature information. The bottleneck structure increased the network depth and ACON was used to obtain better nonlinear expression capabilities, thereby achieving a certain improvement in network performance. In Experiment 3, the three branches efficiently extracted multimodal features, leveraging the complementary semantic information of the three modalities to achieve improvements of 1.16% and 1.52% in mAP and F1. In Experiment 4, the semantic relevance of multimodal images was fully learned, using a cosine reweighted Transformer to learn the interdependencies of long-range features. This resulted in improvements of 0.97% and 1.07% in mAP and F1, respectively, with nearly 12% of resource consumption. This interactive enhancement of the network's ability to extract foreign object features effectively improved detection performance. The resulting network architecture reduced the number of parameters and computation by 7.7x and 7.8x compared to YOLOv5s, while improving precision, recall, mAP, and F1 scores by 3.34%, 5.69%, 4.66%, and 4.55%, respectively, significantly improving performance.

[0183] 6. Comparative experiment:

[0184] The CL-YOLOv5 model in the embodiment of the present invention was compared with other object detection models. To ensure fairness, the embodiment of the present invention used a unified input size of 512×512, unified training hyperparameters, and a unified training platform. To detect as many lung foreign bodies as possible, the confidence threshold was set to 0.45 and the IOU threshold was set to 0.5. The specific experimental results of the model provided by the present invention and 11 other detection models are shown in Table 3, including the R-FCN (Res101-FPN) two-stage model and 10 single-stage models. By calculating the number of parameters and computational complexity of each model, as well as its recognition accuracy and time, the recognition rate and efficiency of each model on the lung foreign body PET / CT multimodal dataset were explored.

[0185] Table 3 Comparison results of different models on the lung foreign body PET / CT multimodal dataset

[0186]

[0187] It can be seen from Table 3 that the total training time and FPS of the two-stage model R-FCN are significantly worse than those of the single-stage model. It takes a certain amount of time to calculate the candidate box using feature extraction. The single-stage model is more efficient. EfficientDet achieves good performance with less resource consumption. The FPS of the lightweight models PicoDet, NanoDet and Poly-YOLO can all significantly exceed the requirements of real-time detection. The YOLO series combines candidate box and object recognition to better improve efficiency. The model CL-YOLOv5 provided by the present invention has the smallest number of parameters and computational complexity, achieving the best efficiency, which is 5.3 times and 3.9 times less than EfficientDet-d0, and has the best accuracy, recall rate, mAP, and F1 score.

[0188] Compared with YOLOv4l, YOLOv5l adds adaptive anchor box calculation, Focus, and FPN-PAN structure for improving low-level feature propagation, which improves performance while also improving efficiency, with FPS and total time optimized by nearly 1 / 10; TPH-YOLOv5 accurately locates targets in high-density scenes through Transformer integration, and the mAP and F1 scores of the model in this invention are improved by 2.58% and 2.94%; YOLOv7l introduces model heavy parameters to achieve good performance, and the running speed is improved by 1.82% and 1.63% compared with YOLOv5l; YOLOv8 aggregates each layer of the backbone at the end in a splicing manner to ensure lightweight while obtaining richer gradient flow information, achieving higher performance. The model CL-YOLOv5 in this invention has a nearly 54-fold reduction in computational complexity and nearly 1-times faster running time than YOLOv8, with mAP and F1 scores improved by 1.44% and 1.76%

[0189] The present invention provides a detection method for PET / CT medical images, fully considering the impact of different imaging devices on the feature imaging mechanism, and proposes a cross-modal lightweight CL-YOLOv5 detection model. The semantic relevance of multimodal images is fully learned through a cross-modal interactive enhancement block. The Transformer with cosine reweighting calculation method maintains lightweight while interactively enhancing the network's ability to extract features from multimodal images. The densely connected and bottleneck-structured dual-branch lightweight block reduces model complexity and improves detection efficiency. Feature reuse, network deepening, and progressive reparameterized convolution are used to learn richer multimodal features.

[0190] Obviously, those skilled in the art may make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if such changes and modifications fall within the scope of the claims and their equivalents, the present invention is intended to include such changes and modifications.

Claims

1. A detection method for PET / CT medical images, characterized in that: The following steps are involved: S1. Acquire a target PET image, a target CT image, and a target PET / CT image; S2. A backbone network based on the CL-YOLOv5 model extracts features from the target PET image, the target CT image, and the target PET / CT image, respectively, and fuses all the extracted features to obtain fused features; S3. Perform feature enhancement processing on the fused feature through the feature enhancement neck of the CL-YOLOv5 model to obtain enhanced features; S4. Processing the enhanced features through the prediction head of the CL-YOLOv5 model to obtain a segmentation feature map; In step S2, the backbone network includes three branches and four stages; The three branches include a PET branch, a CT branch, and a PET / CT branch, corresponding to the target PET image, the target CT image, and the target PET / CT image, respectively; The four stages include: stage one, stage two, stage three and stage four; each stage is implemented by a dual-branch lightweight block and a cross-modal interactive enhancement block; the dual-branch lightweight block includes recursive reparameterized convolution and ACON loss function; Both the first and fourth stages include 3 layers of recursive reparameterized convolution, ACON loss function and cross-modal interactive enhancement block; Both the second and third stages include 8 layers of recursive reparameterized convolution, ACON loss function and cross-modal interactive enhancement block; The step S3 specifically includes: Feature enhancement neck in the CL-YOLOv5 model: Performing 1×1 re-parameterized convolution processing on the stage 2 fusion features, stage 3 fusion features, and stage 4 fusion features respectively to obtain corresponding stage 2 features, stage 3 features, and stage 4 features; The four features of the stage are processed using a three-layer recursive re-parameterized convolution to obtain the first convolution feature; Performing upsampling processing on the first convolution feature to obtain a first sampling result; After concatenating the first sampling result and the third feature of the stage, a three-layer recursive reparameterized convolution is used to obtain the second convolution feature; Performing upsampling on the second convolution feature to obtain a second sampling result; After concatenating the second sampling result and the stage two feature, a three-layer recursive reparameterized convolution is used to process the result to obtain a first enhanced feature; After performing 1×1 reparametric convolution on the first enhanced feature, concatenating it with the second convolution feature, and processing the concatenated result using 3 layers of recursive reparametric convolution to obtain a second enhanced feature; After performing 1×1 reparametric convolution on the second enhanced feature, it is concatenated with the fourth feature of the stage, and the concatenation result is processed using 3 layers of recursive reparametric convolution to obtain the third enhanced feature; The step S4 specifically includes: In the prediction head of the CL-YOLOv5 model: Performing 1×1 multi-parameter convolution processing on the first enhanced feature, the second enhanced feature, and the third enhanced feature respectively, and splicing the three 1×1 multi-parameter convolution processing results to obtain a segmentation feature map; The cross-modal interactive enhancement block specifically includes the following steps: The double-branch output features of the three branches are concatenated and then subjected to 1×1 re-parameterized convolution to obtain a feature map with a channel number of c. The Relu activation function is used to process the feature map No. 1 to obtain matrix Q, matrix K and matrix V; Multiply the matrix Q, the matrix K, and the matrix V pixel by pixel to obtain a second feature map; After residual addition of the first feature map and the second feature map, the third feature map is obtained by sequentially performing 3×3 depthwise convolution, BN function, ReLU activation function, 1×1 reparametric convolution and residual addition processing; The second feature map and the third feature map are concatenated and then subjected to 1×1 re-parameterized convolution processing to obtain a fourth feature map; The four feature maps are cut and reorganized to obtain three feature maps with c channels each, which are sequentially transmitted to the PET branch, the CT branch, and the PET / CT branch.

2. The detection method for PET / CT medical images according to claim 1, characterized in that: The first stage specifically includes the following steps: Slice the image corresponding to each branch separately to obtain the slicing results of the three branches; Using 1×1 multi-parameter convolution to extract features from the slice results of the three branches respectively, to obtain the first initial feature maps of the three branches; Performing feature extraction again on the first initial feature maps of the three branches through the three-layer recursive reparameterized convolution in the stage one to obtain the stage one three-layer convolution feature maps of the three branches; The three-layer convolution feature map of the stage one is compensated by the ACON loss function in the stage one to obtain the stage one two-branch output features of the three branches; The stage one double-branch output features of the three branches are fused through the cross-modal interactive enhancement block in the stage one to obtain the stage one fused features; The fused features of the first stage are divided into three feature maps with the same number of channels, which are respectively transmitted to the three branches of the second stage as the second initial feature maps.

3. The detection method for PET / CT medical images according to claim 2, characterized in that: The second stage specifically includes the following steps: Performing feature extraction again on the second initial feature maps of the three branches through the 8-layer recursive reparameterized convolution in the second stage to obtain the 8-layer convolution feature maps of the three branches in the second stage; The 8-layer convolutional feature map of the second stage is compensated by the ACON loss function in the second stage to obtain the two-branch output features of the third branch; The stage 2 dual-branch output features of the three branches are fused through the cross-modal interactive enhancement block in the stage 2 to obtain the stage 2 fused features; The fused features of the second stage are divided into three feature maps with the same number of channels, which are respectively transmitted to the three branches of the third stage as the third initial feature map.

4. The detection method for PET / CT medical images according to claim 3, characterized in that: The third stage specifically includes the following steps: Performing feature extraction again on the third initial feature maps of the three branches through the three-layer recursive reparameterized convolution in the third stage to obtain the three-layer convolution feature maps of the three branches in the third stage; The three-layer convolutional feature map of the stage three is compensated by the ACON loss function in the stage three to obtain the stage three dual-branch output features of the three branches; The stage three double-branch output features of the three branches are fused through the cross-modal interactive enhancement block in the stage three to obtain the stage three fused features; The fused features of the stage three are divided into three feature maps with the same number of channels, which are respectively transmitted to the three branches of the stage four as the fourth initial feature map.

5. The detection method for PET / CT medical images according to claim 4, characterized in that: The fourth stage specifically includes the following steps: Performing feature extraction again on the fourth initial feature maps of the three branches through the 8-layer recursive reparameterized convolution in the fourth stage to obtain the 8-layer convolution feature maps of the three branches in the fourth stage; The 8-layer convolutional feature map of the stage 4 is compensated by the ACON loss function in the stage 4 to obtain the stage 4 dual-branch output features of the three branches; The stage four dual-branch output features of the three branches are fused through the cross-modal interactive enhancement block in the stage four to obtain the stage four fused features.

6. The detection method for PET / CT medical images according to claim 1, characterized in that: Each layer of recursive re-parameterized convolution specifically includes the following steps: Use 1×1 multi-parameter convolution to process the input feature map with c channels and obtain an output feature map with 2c channels; Slicing the output feature map to obtain a first feature map and a second feature map; the number of channels of the first feature map and the second feature map are 1 / 4c and 7 / 4c, respectively; The second feature map is processed by a 7×7 depthwise convolution, and the convolved second feature map is sliced ​​to obtain a third feature map, a fourth feature map, and a fifth feature map; the number of channels of the third feature map, the fourth feature map, and the fifth feature map are 1 / 4c, 1 / 2c, and c, respectively; Multiplying the third feature map by the first feature map and performing a 1×1 multi-parameter convolution process to obtain a first output feature map with a channel number of 1 / 2c; Multiplying the first output feature map by the fourth feature map and performing a 1×1 multi-parameter convolution process to obtain a second output feature map with a channel number c; Multiply the second output feature map by the fifth feature map and perform a 1×1 reparametric convolution process to obtain a final output feature map with c channels.

Images