RKNN-based parathyroid real-time detection system and method

Abstract

The present application relates to a kind of based on RKNN parathyroid real-time detection system and method, including image acquisition unit, pre-processing unit, target inference unit and post-processing display unit;The image acquisition unit obtains parathyroid real-time image, and is preprocessed by pre-processing unit;The target inference unit is inferred based on RKNN model to the image after pre-processing, obtains reasonable detection frame;The post-processing display unit calls NMS non-maximum suppression filter duplicate frame, finally frames and shows the inference result of original image.The present application realizes the real-time detection of parathyroid on embedded platform.

Description

Technical Field

[0001] This invention relates to the field of real-time target detection technology, specifically to a real-time parathyroid gland detection system and method based on RKNN. Background Technology

[0002] Thyroid diseases are among the most common diseases of the endocrine system. In endoscopic thyroid surgery, early warning, continuous identification, and ischemia assessment of the parathyroid glands are crucial. Early identification of the parathyroid glands provides the surgeon with a warning, preventing unintentional damage, especially to their blood supply, allowing for more precise dissection. Continuous tracking of the parathyroid glands provides the surgeon with real-time location information during surgery, reducing the surgeon's need for constant monitoring and preventing accidental damage by inexperienced surgeons. Assessment of the parathyroid gland status plays a vital role in postoperative parathyroid function protection. Timely treatment of poorly blood-supplied parathyroid glands during surgery can effectively prevent postoperative hypoparathyroidism.

[0003] Therefore, real-time detection of the parathyroid glands is an important task, requiring a device and system capable of such detection. Summary of the Invention

[0004] In view of this, the purpose of the present invention is to provide a real-time parathyroid gland detection system and method based on RKNN, which aims to solve the above problems.

[0005] To achieve the above objectives, the present invention adopts the following technical solution:

[0006] A real-time parathyroid gland detection system based on RKNN includes an image acquisition unit, a preprocessing unit, a target inference unit, and a post-processing display unit. The image acquisition unit acquires real-time images of the parathyroid glands and preprocesses them through the preprocessing unit. The target inference unit infers the preprocessed images based on the RKNN model to obtain reasonable detection boxes. The post-processing display unit uses Non-Maximum Suppression (NMS) to filter out duplicate boxes, and finally marks the original images with bounding boxes and displays the inference results.

[0007] Furthermore, the system uses the RK3588 chip that can run Rockchip RKNN.

[0008] Furthermore, the method for constructing the RKNN model is as follows: the lightweight single-stage detection model YOLOv5s is used to train the parathyroid dataset, and the trained weight model is converted into an RKNN model in a virtual machine environment.

[0009] Furthermore, the construction of the parathyroid gland dataset is as follows:

[0010] Parathyroid glands pre-identified in the surrounding adipose tissue or thymus tissue before exposure are labeled as PTS0 and PTM0, where PTS0 is the superior parathyroid gland that has not yet been exposed and PTM0 is the thymus-associated parathyroid gland that has not yet been exposed.

[0011] From the initial partial exposure of the parathyroid glands until the parathyroid gland tissue disappeared from the field of view, the superior parathyroid glands, thyroid-dependent parathyroid glands, artery-related parathyroid glands, and thymus-related parathyroid glands were labeled as PTS1, PTT, PTA, and PTM1, respectively.

[0012] Parathyroid glands with poor blood supply requiring further evaluation are classified as PTS3, PT3, and PTM3. PTS3, PT3, and PTM3 refer to superior parathyroid glands, inferior parathyroid glands, and thymus-related parathyroid glands with poor blood supply or requiring further treatment, respectively.

[0013] Furthermore, the lightweight single-stage detection model YOLOv5s is based on the single-stage object detection network YOLOv5s, and is obtained by replacing the Sigmoid function with the ReLU function.

[0014] Furthermore, the lightweight single-stage detection model YOLOv5s is used to train the parathyroid dataset, as detailed below:

[0015] Read the JSON annotation file information from the dataset and convert the annotation information from two-point coordinate form (x1, y1, x2, y2) into center point coordinates and height and width form (x1, y1, x2, y2). o ,y o ,w,h);

[0016] Reading images from laparoscopic parathyroid surgery {x (1) ,x (2) ,...,x (r)} and tag {l (1) ,l (2) ,...,l (m) Mosaic data augmentation was used to stitch four images together to form one image Q. Letterbox was used to scale the original image Q (1920×1080×3) to a 640×640×3 image Q2 with the same aspect ratio. After shrinking, the edges that did not meet the conditions were filled with white bars. The image was then input into the CSPDarknet backbone network model and the corresponding pre-processed annotation information was read.

[0017] The image Q2 is sliced ​​using a Focus network structure, and then a convolution operation is performed to transform the feature map into a 320×320×32 channel feature map. Preliminary extraction of image feature information;

[0018] Data ψ 1 After one CBS operation, namely Conv(3,3), BatchNorm2d, and ReLU operation, the ReLU activation function, which is more suitable for the chip's NPU operator, is chosen instead of the SiLU activation function. This layer of the network outputs 320×320×64 channel data.

[0019] Data ψ 2 After several CBS and C3 operations, further feature extraction and parameter reduction are performed. The C3 module consists of three standard convolutional CBS layers and multiple BottleNeck residual modules. The number of BottleNecks is specified by parameters and used for feature extraction. The final output is a 20×20×1024 channel data set.

[0020] feature data ψ 3 After the SPPF module integrates features at different resolutions, the SPPF model, an improvement on the SPP module, is derived. SPP's role is to integrate features at different resolutions. SPPF replaces the SPP's 5×5, 9×9, and 13×13 max pooling with three 5×5 kernels. These three max pooling operations are processed sequentially and concatenated to obtain the third-layer features F = (f1, f2, ..., f...). 1024 );

[0021] The obtained features from each layer are fed into the Neck module to obtain three layers of output {O2,O1,O0}.

[0022] The three-layer output {O2,O1,O0} is decoded into corresponding location prediction boxes B={b2,b1,b0} on the image. Scores are then ranked and non-maximum suppression (NMS) is applied for filtering. The resulting predictions are used to calculate the loss function; CIoU is used to calculate the intersection-over-union ratio, as shown in the following formula:

[0023]

[0024]

[0025]

[0026] Where β is the weighting function, v is used to measure the consistency of the aspect ratio, and w and h are the width and height of the prediction box B = {b2, b1, b0}. gt h gt Given the width and height of the actual bounding box, the final loss function is defined as follows: L CIoU The Adam optimizer is used to minimize the loss function to train the network model;

[0027]

[0028] Repeat the process until the YOLOv5s network model F is trained and the parathyroid target-related features are obtained.

[0029] Furthermore, the Neck employs FPN and PAN structures; the FPN is a top-down structure that fuses high-level feature maps with low-level feature maps after upsampling; the PAN is a bottom-up structure that transmits position information to higher levels.

[0030] A real-time detection method for a parathyroid gland detection system based on RKNN includes the following steps:

[0031] (1): The system acquires an image containing parathyroid glands from the image acquisition unit, adjusts the image to a preset size proportionally, and sends it into the input queue Q. r middle;

[0032] (2): Initialize the NPU runtime environment and continuously input the data from the input queue Q. r Image acquisition is performed by the NPU, which has the highest computational performance, and the inference results are sent to the inference queue Q. i ;

[0033] (3): Read the inference results from the output queue, select the detection box D with the highest score, remove it from the output queue, and delete the candidate box if the IOU value between the remaining candidate box and D is greater than NMS.

[0034] (4): The system draws a bounding box on the original image, displays the detection results in real time, and sends them to the output queue Q. w Save;

[0035] (5): Multiple processes control (1)-(4) respectively, and the CPU and NPU work together to improve the model detection speed.

[0036] Compared with the prior art, the present invention has the following advantages:

[0037] 1. This invention uses the ReLU function instead of the Sigmoid function and prunes and reduces model parameters to adapt to chip characteristics, effectively improving detection efficiency and quality;

[0038] 2. This invention achieves parallel processing of image preprocessing, target inference, and postprocessing through system fixed frequency and multi-process processing, thereby improving the inference speed of YOLOv5s and realizing real-time detection of parathyroid glands on the embedded platform Rockchip RK3588. Attached Figure Description

[0039] Figure 1This is a diagram illustrating the model training and conversion of the RKNN-based real-time parathyroid detection system in this invention.

[0040] Figure 2 This is a flowchart of the RKNN image inference process of the real-time parathyroid gland detection system based on RKNN in this invention.

[0041] Figure 3 This is an HDMI flowchart of the RKNN-based real-time parathyroid gland detection system in this invention.

[0042] Figure 4 This is a multi-process collaborative control inference diagram of the RKNN-based real-time parathyroid detection system in this invention;

[0043] Figure 5 This is a flowchart of the image post-processing process of the RKNN-based real-time parathyroid detection system in this invention. Detailed Implementation

[0044] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0045] Please refer to Figure 1-5 This invention provides a real-time parathyroid gland detection system based on RKNN. The system uses the RK3588 chip, which is capable of running Rockchip RKNN, to construct the parathyroid gland detection system. It mainly includes preprocessing, target inference, and post-processing display and storage. The system utilizes fixed frequency, multi-process CPU and NPU collaborative work to speed up the process. The image acquisition unit acquires real-time parathyroid gland images and preprocesses them through the preprocessing unit. The target inference unit infers the preprocessed images based on the RKNN model to obtain reasonable detection boxes. The post-processing display unit calls NMS (Non-Maximum Suppression) to filter duplicate boxes, and finally marks the original image with bounding boxes and displays the inference results.

[0046] In this embodiment, the construction method of the RKNN model is as follows: the lightweight single-stage detection model YOLOv5s is used to train the parathyroid dataset, and the trained weight model is converted into an RKNN model in a virtual machine environment.

[0047] In this embodiment, the construction of the parathyroid gland dataset is as follows:

[0048] Parathyroid glands pre-identified in the surrounding adipose tissue or thymus tissue before exposure are labeled as PTS0 and PTM0, where PTS0 is the superior parathyroid gland that has not yet been exposed and PTM0 is the thymus-associated parathyroid gland that has not yet been exposed.

[0049] From the initial partial exposure of the parathyroid glands until the parathyroid gland tissue disappeared from the field of view, the superior parathyroid glands, thyroid-dependent parathyroid glands, artery-related parathyroid glands, and thymus-related parathyroid glands were labeled as PTS1, PTT, PTA, and PTM1, respectively.

[0050] Parathyroid glands with poor blood supply requiring further evaluation are classified as PTS3, PT3, and PTM3. PTS3, PT3, and PTM3 refer to superior parathyroid glands, inferior parathyroid glands, and thymus-related parathyroid glands with poor blood supply or requiring further treatment, respectively.

[0051] In this embodiment, the lightweight single-stage detection model YOLOv5s is obtained by replacing the Sigmoid function with the ReLU function, based on the single-stage object detection network YOLOv5s.

[0052] In this embodiment, the lightweight single-stage detection model YOLOv5s is used to train the parathyroid dataset, as follows:

[0053] Read the JSON annotation file information from the dataset and convert the annotation information from two-point coordinate form (x1, y1, x2, y2) into center point coordinates and height and width form (x1, y1, x2, y2). o ,y o ,w,h);

[0054] Reading images from laparoscopic parathyroid surgery {x (1) ,x (2) ,...,x (r)} and tag {l (1) ,l (2) ,...,l (m) Mosaic data augmentation was used to stitch four images together to form one image Q. Letterbox was used to scale the original image Q (1920×1080×3) to a 640×640×3 image Q2 with the same aspect ratio. After shrinking, the edges that did not meet the conditions were filled with white bars. The image was then input into the CSPDarknet backbone network model and the corresponding pre-processed annotation information was read.

[0055] The image Q2 is sliced ​​using a Focus network structure, and then a convolution operation is performed to transform the feature map into a 320×320×32 channel feature map. Preliminary extraction of image feature information;

[0056] Data ψ 1 After one CBS operation, namely Conv(3,3), BatchNorm2d, and ReLU operation, the ReLU activation function, which is more suitable for the chip's NPU operator, is chosen instead of the SiLU activation function. This layer of the network outputs 320×320×64 channel data.

[0057] Data ψ 2 After several CBS and C3 operations, further feature extraction and parameter reduction are performed. The C3 module consists of three standard convolutional CBS layers and multiple BottleNeck residual modules. The number of BottleNecks is specified by parameters and used for feature extraction. The final output is a 20×20×1024 channel data set.

[0058] feature data ψ 3 After the SPPF module integrates features at different resolutions, the SPPF model, an improvement on the SPP module, is derived. SPP's role is to integrate features at different resolutions. SPPF replaces the SPP's 5×5, 9×9, and 13×13 max pooling with three 5×5 kernels. These three max pooling operations are processed sequentially and concatenated to obtain the third-layer features F = (f1, f2, ..., f...). 1024 );

[0059] The obtained features from each layer are fed into the Neck module to obtain three layers of output {O2,O1,O0}.

[0060] The three-layer output {O2,O1,O0} is decoded into corresponding location prediction boxes B={b2,b1,b0} on the image. Scores are then ranked and non-maximum suppression (NMS) is applied for filtering. The resulting predictions are used to calculate the loss function; CIoU is used to calculate the intersection-over-union ratio, as shown in the following formula:

[0061]

[0062]

[0063]

[0064] Where β is the weighting function, v is used to measure the consistency of the aspect ratio, and w and h are the width and height of the prediction box B = {b2, b1, b0}. gt h gt Given the width and height of the actual bounding box, the final loss function is defined as follows: L CIoU The Adam optimizer is used to minimize the loss function to train the network model;

[0065]

[0066] Repeat the process until the YOLOv5s network model F is trained and the parathyroid target-related features are obtained.

[0067] In this embodiment, Neck employs FPN and PAN structures; the FPN is a top-down structure that fuses high-level feature maps with low-level feature maps after upsampling; the PAN is a bottom-up structure that transmits position information to higher levels.

[0068] In this embodiment, a real-time detection method for a real-time parathyroid gland detection system based on RKNN includes the following steps:

[0069] (1): The system acquires an image containing parathyroid glands from the image acquisition unit, adjusts the image to a preset size proportionally, and sends it into the input queue Q. r middle;

[0070] (2): Initialize the NPU runtime environment and continuously input the data from the input queue Q. r Image acquisition is performed by the NPU, which has the highest computational performance, and the inference results are sent to the inference queue Q. i ;

[0071] (3): Read the inference results from the output queue, select the detection box D with the highest score, remove it from the output queue, and delete the candidate box if the IOU value between the remaining candidate box and D is greater than NMS.

[0072] (4): The system draws a bounding box on the original image, displays the detection results in real time, and sends them to the output queue Q. w Save;

[0073] (5): Multiple processes control (1)-(4) respectively, and the CPU and NPU work together to improve the model detection speed.

[0074] In this embodiment, the specific steps of the proposed algorithm for the RKNN-based real-time parathyroid gland detection system are as follows:

[0075] 1. Senior physicians select images containing parathyroid glands from high-definition surgical videos and label them to obtain a parathyroid gland dataset and labels;

[0076] 2. Modify the single-stage object detection network YOLOv5s by replacing the Sigmoid function with the ReLU function, and train YOLOv5s using a dataset to obtain the network's parameter weights;

[0077] 3. Using the RKNN Toolkit, the network model was converted into an RKNN model in a virtual machine environment, and a parathyroid gland detection system was built on Rockchip RK3588;

[0078] 4. The system obtains the video stream to be detected through input;

[0079] 5. The system processes the input video stream {x}(1) ,x (2) ,...,x (r) Perform multi-process real-time detection, for each image x r This includes preprocessing, parathyroid target reasoning, and postprocessing;

[0080] 6. The chip transmits the detection results back, and the detection system interface displays and saves them;

[0081] In this embodiment, the training steps in step 2) regarding the real-time parathyroid gland detection method based on Rockchip RKNN are 7)-15).

[0082] 7. Read the JSON annotation file information from the dataset and convert the annotation information from two-point coordinate form (x1, y1, x2, y2) into center point coordinates and height / width form (x1, y1, x2, y2). o ,y o ,w,h);

[0083] 8. Reading images from laparoscopic parathyroid surgery {x (1) ,x (2) ,...,x (r)} and tag {l (1) ,l (2) ,...,l (m) Mosaic data augmentation was used to stitch four images together to form one image Q. Letterbox was used to scale the original image Q (1920×1080×3) to a 640×640×3 image Q2 with the same aspect ratio. After shrinking, the edges that did not meet the conditions were filled with white bars. The image was then input into the CSPDarknet backbone network model and the corresponding pre-processed annotation information was read.

[0084] 9. Use a Focus network structure to slice the Q2 image, and then perform a convolution operation to transform the feature map into a 320×320×32 channel feature map. Preliminary extraction of image feature information;

[0085] 10. Transfer the data ψ 1 After one CBS operation, namely Conv(3,3), BatchNorm2d, and ReLU operation, the ReLU activation function, which is more suitable for the chip's NPU operator, is chosen instead of the SiLU activation function. This layer of the network outputs 320×320×64 channel data.

[0086] 11. Further extract features from the data ψ 2After several CBS and C3 operations, further feature extraction and parameter reduction are performed. The C3 module consists of three standard convolutional CBS layers and multiple BottleNeck residual modules. The number of BottleNecks is specified by parameters and used for feature extraction. The final output is a 20×20×1024 channel data set.

[0087] 12. Transfer the feature data ψ 3 The SPPF module integrates features from different resolutions. The SPPF model is an improvement on the SPP module, where SPP integrates features from different resolutions. SPPF replaces the 5×5, 9×9, and 13×13 max pooling in SPP with three 5×5 kernels. By processing the three max pooling operations sequentially, SPPF reduces the number of parameters, decreases overfitting, and speeds up the network while maintaining the same performance. The concatenation function obtains the third layer features F = (f1, f2, ..., f...). 1024 );

[0088] 13. The features obtained in steps 11)-12) are fed into the Neck module to improve the network's feature extraction capability. The Neck module uses FPN and PAN structures. Higher-level feature maps have larger receptive fields due to multiple downsampling operations, while lower-level feature maps have fewer downsampling operations, preserving richer positional information. FPN is a top-down structure that merges higher-level feature maps with lower-level feature maps after upsampling. PAN is a bottom-up structure that passes features to higher levels, resulting in a three-layer output {O2, O1, O0}.

[0089] 14. Decode the three-layer output {O2,O1,O0} into corresponding location prediction boxes B={b2,b1,b0} on the image, perform score sorting and non-maximum suppression (NMS) filtering, and use the prediction results for loss function calculation; CIoU is used to calculate the intersection-over-union ratio, as shown in the following formula:

[0090]

[0091]

[0092]

[0093] Where β is the weighting function, v is used to measure the consistency of the aspect ratio, and w and h are the width and height of the prediction box B = {b2, b1, b0}. gt h gt Given the width and height of the actual bounding box, the final loss function is defined as follows: L CIoUThe Adam optimizer is used to minimize the loss function to train the network model;

[0094]

[0095] 15. Repeat steps 10)-14) until the YOLOv5s network model F is trained and the parathyroid target-related features are obtained.

[0096] In this embodiment, the training steps in step 2) regarding the real-time parathyroid detection method based on Rockchip RKNN are 16)-20).

[0097] 16. The system continuously reads images containing parathyroid glands from the HDMI interface, proportionally resizes the images to 640×640, and sends them into the input queue Q. r middle;

[0098] 17. Initialize the NPU runtime environment, continuously drawing input from the input queue Q. r Image acquisition is performed by the NPU, which has the highest computational performance, and the inference results are sent to the inference queue Q. i ;

[0099] 18. The system continuously reads inference results from the output queue, selects the detection box D with the highest score, removes it from the output queue, and deletes the candidate box if the IOU value between the remaining candidate box and D is greater than NMS.

[0100] 19. The system draws a frame on the original image, displays the detection results in real time, and sends them to the output queue Q. w Save;

[0101] 20. Multiple processes control 16)-19) respectively, with the CPU and NPU working together to improve model detection speed;

[0102] The above description is only a preferred embodiment of the present invention. All equivalent changes and modifications made within the scope of the claims of the present invention should be included in the scope of the present invention.

Claims

1. A real-time parathyroid gland detection system based on RKNN, characterized in that, The system includes an image acquisition unit, a preprocessing unit, a target inference unit, and a post-processing display unit. The image acquisition unit acquires real-time images of the parathyroid glands and preprocesses them through the preprocessing unit. The target inference unit infers the preprocessed images based on the RKNN model to obtain reasonable detection boxes. The post-processing display unit uses Non-Maximum Suppression (NMS) to filter out duplicate boxes, and finally marks the original images with bounding boxes and displays the inference results. The method for constructing the RKNN model is as follows: The parathyroid dataset was trained using the lightweight single-stage detection model YOLOv5s, and the trained weight model was converted into an RKNN model in a virtual machine environment. The construction of the parathyroid gland dataset is as follows: Parathyroid glands pre-identified in the surrounding adipose tissue or thymus tissue before exposure are labeled as PTS0 and PTM0, where PTS0 is the superior parathyroid gland that has not yet been exposed and PTM0 is the thymus-associated parathyroid gland that has not yet been exposed. From the initial partial exposure of the parathyroid glands until the parathyroid gland tissue disappeared from the field of view, the superior parathyroid glands, thyroid-dependent parathyroid glands, artery-related parathyroid glands, and thymus-related parathyroid glands were labeled as PTS1, PTT, PTA, and PTM1, respectively. Parathyroid glands with poor blood supply requiring further evaluation are classified as PTS3, PT3, and PTM3, respectively. PTS3, PT3, and PTM3 refer to superior parathyroid glands, inferior parathyroid glands, and thymus-related parathyroid glands with poor blood supply or requiring further treatment. The system adopts a multi-process collaborative control method, including the following steps: (1) The system acquires an image containing the parathyroid gland from the image acquisition unit, adjusts the image to a preset size proportionally, and sends it into the input queue. middle; (2) Initialize the NPU runtime environment and continuously input data from the input queue. Image acquisition is performed by the NPU, which has the highest computing performance, and the inference results are sent to the inference queue. ; (3) Read the inference results from the output queue and select the detection box with the highest score. Remove from the output queue if the remaining candidate boxes are... of Value greater than If so, then delete the candidate box; (4) The system draws a frame on the original image, displays the detection results in real time, and sends them to the output queue. Save; (5) Multiple processes control (1)-(4) respectively, and the CPU and NPU work together to improve the model detection speed.

2. The RKNN-based real-time parathyroid gland detection system according to claim 1, characterized in that, The system uses the RK3588 chip, which can run Rockchip's RKNN model.

3. The RKNN-based parathyroid real-time detection system of claim 1, wherein, The lightweight single-stage detection model YOLOv5s is based on the single-stage object detection network YOLOv5s, and uses... Function substitution The function is obtained.

4. The RKNN-based parathyroid real-time detection system of claim 1, wherein, The lightweight single-stage detection model YOLOv5s was used to train the parathyroid dataset, as detailed below: Read the JSON annotation file information in the data set, convert the annotation information from two-point coordinate form to center point coordinate and height-width form ; Reading images from laparoscopic parathyroid surgery and tags ,use Data augmentation: stitching four images together into one. ,use The size is Original image Scaling to the same height and width ratio image After shrinking, edges that do not meet the conditions are filled with white bars. (Input) The backbone network model is generated, and the corresponding pre-processed annotation information is read. use Network structure for images Slice the image and then perform a convolution operation to transform the feature map into... Channel characteristics Preliminary extraction of image feature information; Data After one Operation, i.e. , and For operation, select the one that best suits the chip's NPU operator. Activation function rather than Activation function, the output of this layer of the network Channel data ; Data After several times Operation and Further feature extraction and parameter reduction are performed, among which The module consists of three standard convolutional layers. and multiple The residual module consists of, The number of features is specified by a parameter and used for feature extraction; the final output size is [size missing]. Channel data ; Feature data go through The module integrates features from different resolutions. The model is made by It is an improvement on the module. Its function is to integrate features from different resolutions. Using three cores of size Max pooling replaced The sizes of the mid-cores are respectively , , Max pooling is performed by processing the three max pooling operations sequentially. Obtain the third layer of features ; The obtained features of each layer are fed into a module to obtain three-layer output ; Three-layer output Decoded into corresponding location prediction boxes on the image Score sorting and nonmaximum suppression are performed. The resulting predictions are then filtered and used for... Loss function calculation; where the loss function is used The crossover ratio (CRO) is calculated using the following formula: ， ， in For the weight function, Used to measure the consistency of aspect ratio. , For prediction boxes Width and height, , The final width and height of the actual frame The function is defined as follows: The Adam optimizer is used to minimize the loss function to train the network model; Repeat the process until the YOLOv5s network model is complete. Training is complete, and parathyroid target-related features are obtained.

5. The RKNN-based real-time parathyroid gland detection system according to claim 1, characterized in that, The Adopted and Structure; the It is a top-down structure that fuses high-level feature maps with low-level feature maps after upsampling; It is a bottom-up structure that transmits position information to higher levels.

Images