U2-Net-based image processing system and method
By using a U2-Net-based image processing system, wavelet dense residual filters and hierarchical fusion stitching modules, combined with an RSU encoder and decoder, the problem of inaccurate edge feature extraction in abdominal CT image segmentation was solved, and high-frequency details and edge information were restored, ensuring the integrity of the segmentation boundaries.
Patent Information
- Application Number
- CN202511447894.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-10-11
- Publication Date
- 2026-01-02
AI Technical Summary
Existing medical image segmentation models struggle to accurately extract edge features from abdominal CT images, often resulting in missegmentation or omissions. Furthermore, their ability to recover high-frequency details and edge information is limited, leading to incomplete segmentation boundaries.
An image processing system based on U2-Net is adopted. Through a post-stage fusion and stitching module and N RSU reconstruction units, wavelet dense residual filters and hierarchical fusion and stitching modules are used, combined with RSU encoders and decoders, to perform multi-layer convolution and upsampling to restore high-frequency details and edge information, ensuring the integrity of segmentation boundaries.
It effectively avoids artifacts and loss of detail, ensures complete segmentation boundaries, and improves the accuracy and clarity of medical image segmentation.
Smart Images
Figure CN121258818A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to medical imaging technology, and in particular to a U-based... 2 -Net's image processing system and methods. Background Technology
[0002] With the development of medical imaging equipment such as CT (computed tomography) and MRI (magnetic resonance imaging), the importance of medical image processing has become increasingly prominent. Medical segmentation technology, as a core technology in medical image processing, aims to precisely segment different organs or lesion areas from medical images such as CT, MRI, ultrasound, and X-rays. This allows doctors to more intuitively obtain the location, extent, and boundaries of lesions, thereby improving the accuracy and efficiency of diagnosis and treatment.
[0003] Currently, deep learning-based medical image segmentation methods have become the mainstream of medical image segmentation technology. The "encoder-decoder" structure, represented by U-Net (U-Squared Net, a deep learning network designed for fine image segmentation) and its improved models (such as U-Net++, AttU-Net, and TransUNet), can achieve image feature extraction and region segmentation through layer-by-layer convolution and upsampling.
[0004] However, abdominal CT images are structurally complex, have low contrast, and blurry organ boundaries, making it difficult for models to accurately extract edge features. When segmenting small lesions or locally blurred regions, missegmentation or omissions are prone to occur. Furthermore, the upsampling stage of existing models mostly relies on deconvolution or bilinear interpolation, which have limited ability to recover high-frequency details and edge information, often resulting in artifacts or loss of detail, leading to incomplete segmentation boundaries. Summary of the Invention
[0005] To address the shortcomings of the existing technologies, the technical problem to be solved by this invention is to provide a U-based method that has strong capabilities in restoring high-frequency details and edge information, effectively avoids artifacts and detail loss, and ensures complete segmentation boundaries. 2 -Net image processing system and methods.
[0006] To solve the above-mentioned technical problems, the present invention provides a U-based 2 -Net's image processing system includes a post-stage fusion and stitching module, N RSU reconstruction units, and N≥3; the RSU reconstruction unit includes a wavelet dense residual filter, a hierarchical fusion and stitching module, and U 2 -Net's RSU encoder and RSU decoder; the RSU encoder and RSU decoder in the same RSU reconstruction unit are symmetric encoder-decoder structures, and N RSU reconstruction units are sequentially arranged from front to back according to the execution order to form a sequence;
[0007] The mathematical model of the wavelet dense residual filter is as follows:
[0008] ;
[0009] Where I is the image to be processed. Let I be the low-frequency approximation component of image I in the i-th wavelet decomposition. Let be the high-frequency detail component in the horizontal direction of image I in the i-th wavelet decomposition. Let be the high-frequency detail component in the vertical direction of image I in the i-th wavelet decomposition. Let I be the high-frequency detail component in the diagonal direction of image I in the i-th wavelet decomposition. The wavelet transform function, To Low-frequency components of the layer Wavelet decomposition is performed, and the resulting low-frequency approximate components are assigned... The high-frequency detail components in the horizontal direction give it The high-frequency detail components in the vertical direction give it High-frequency detail components in the diagonal direction are given ;
[0010] Both the post-stage fusion and stitching module and the hierarchical fusion and stitching module are used to fuse and stitch multiple received images into a single image;
[0011] The height of the RSU encoders in the first N-1 RSU reconstruction units decreases sequentially from front to back, and the height of the RSU encoders in the last two RSU reconstruction units is the same. The height of the encoder refers to the number of coding layers in the encoder.
[0012] The first N-1 RSU reconstruction units each have an RSU encoder, and the data output port of the RSU encoder of the first N-2 RSU reconstruction units is connected to the data input port of the RSU encoder of the next adjacent RSU reconstruction unit.
[0013] In the first N-1 RSU reconstruction units, the data input port of the wavelet dense residual filter is connected to the data output port of the RSU decoder of the subsequent adjacent RSU reconstruction unit, and the data input port of the hierarchical fusion splicing module is connected to the data output port of the RSU encoder of this unit and the data output port of the wavelet dense residual filter of this unit respectively.
[0014] The last RSU reconstruction unit has two RSU encoders, namely the first RSUF encoder and the second RSUF encoder. The data input port of the first RSUF encoder is connected to the data output port of the RSU encoder of the preceding adjacent RSU reconstruction unit. The data output port of the first RSUF encoder is connected to the data input port of the second RSU encoder. The data output port of the second RSU encoder is connected to the data input port of the wavelet dense residual filter in this unit and the data input port of the subsequent fusion and splicing module.
[0015] In the last RSU reconstruction unit, the data input port of the hierarchical fusion and splicing module is connected to the data output port of the first RSUF encoder and the data output port of the wavelet dense residual filter of this unit, respectively.
[0016] In each RSU reconstruction unit, the data output port of the hierarchical fusion and splicing module is connected to the data input port of the RSU decoder in this unit, and the data output port of the RSU decoder is connected to the data input port of the subsequent fusion and splicing module.
[0017] The present invention also provides the use of the above-mentioned U-based 2 The image processing method of the Net image processing system includes the following steps:
[0018] 1) Obtain the target image to be processed, select the target object and target features in the target image, and input the target image into the RSU encoder of the first RSU reconstruction unit;
[0019] The target features include the texture features and edge contour features of the target object, as well as the contour features of the associated tissues between the target object and its surrounding tissues;
[0020] 2) The RSU encoder extracts target features from the received image and defines the image constructed from the extracted target features as the first feature map;
[0021] The RSU encoders of the first N-1 RSU reconstruction units output the first feature map they have constructed to the hierarchical fusion and stitching module of this unit, and then perform pooling processing on the first feature map to form a low-resolution downsampled image, which is then output to the subsequent adjacent RSU encoders.
[0022] In the last RSU reconstruction unit, the first RSUF encoder outputs its own constructed first feature map to the unit's hierarchical fusion and stitching module, and performs pooling processing on the first feature map to form a low-resolution downsampled image, which is then output to the second RSUF encoder. The second RSUF encoder outputs its own constructed first feature map to the wavelet dense residual filter and subsequent fusion and stitching module of the unit.
[0023] 3) In each RSU reconstruction unit, the wavelet dense residual filter performs wavelet decomposition on the received image to generate four target feature component maps. Then, the four target feature component maps generated by the wavelet dense residual filter are upsampled so that the resolution of the four target feature component maps is consistent with the resolution of the first feature map received by the hierarchical fusion and stitching module of this unit. Then, the hierarchical fusion and stitching module fuses and stitches the received images into a single image and outputs it to the RSU decoder of this unit.
[0024] After the RSU decoder decodes the received image, it outputs the image to the subsequent fusion and stitching module and the wavelet dense residual filter in the preceding adjacent RSU reconstruction unit.
[0025] 4) The post-stage fusion and stitching module upsamples the images input to the RSU decoder of non-first RSU reconstruction units so that the resolution of these images is consistent with the resolution of the images input to the RSU decoder of the first RSU reconstruction unit. Then, it fuses and stitches all the received images into a single image, which is then output as the final image.
[0026] The present invention provides a U-based 2 -Net image processing system and methods, through U 2 -Net's RSU encoder and decoder utilize multiple coding layers for convolution operations and downsampling to extract features. After feature recognition, downsampling reduces resolution when highlighting features. Then, the first and second RSUF encoders in the last RSU reconstruction unit dilate the convolution to ensure that more contextual information is obtained without changing the feature resolution. Frequency division is achieved through wavelet decomposition, followed by residual fusion to gradually restore spatial details and obtain multi-scale contextual information, improving the clarity of the reconstructed image. It has the characteristics of strong ability to restore high-frequency details and edge information, effectively avoiding artifacts and detail loss, and ensuring the integrity of segmentation boundaries. Attached Figure Description
[0027] Figure 1 This is an embodiment of the invention based on U 2 -Block diagram of the image processing system of Net;
[0028] Figure 2 This is an image showing the effect of segmenting lesion tissue and / or organ from a medical image using the system and method of this invention. Detailed Implementation
[0029] The embodiments of the present invention will be described in further detail below with reference to the accompanying drawings. However, these embodiments are not intended to limit the present invention. Any similar structures or variations thereof that adopt the present invention should be included in the protection scope of the present invention. The commas in the present invention all indicate the relationship between and. The English letters in the present invention are case-sensitive.
[0030] like Figure 1 As shown, the embodiment of the present invention provides a method based on U 2 -Net's image processing system includes a post-stage fusion and stitching module, N RSU reconstruction units, and N≥3; the RSU reconstruction unit includes a wavelet dense residual filter, a hierarchical fusion and stitching module, and U 2 -Net (deep learning network architecture) RSU encoder and RSU decoder; the RSU encoder and RSU decoder in the same RSU reconstruction unit are symmetric encoder-decoder structures, and N RSU reconstruction units are sequentially arranged from front to back according to the execution order to form a sequence;
[0031] The mathematical model of the wavelet dense residual filter is as follows:
[0032] ;
[0033] Where I is the image to be processed. Let I be the low-frequency approximation component of image I in the i-th wavelet decomposition. Let be the high-frequency detail component in the horizontal direction of image I in the i-th wavelet decomposition. Let be the high-frequency detail component in the vertical direction of image I in the i-th wavelet decomposition. Let I be the high-frequency detail component in the diagonal direction of image I in the i-th wavelet decomposition. The wavelet transform function, To Low-frequency components of the layer Wavelet decomposition is performed, and the resulting low-frequency approximate components are assigned... The high-frequency detail components in the horizontal direction give it The high-frequency detail components in the vertical direction give it High-frequency detail components in the diagonal direction are given ;
[0034] Both the post-stage fusion and stitching module and the hierarchical fusion and stitching module are used to fuse and stitch multiple received images into a single image;
[0035] The height of the RSU encoders in the first N-1 RSU reconstruction units decreases sequentially from front to back, and the height of the RSU encoders in the last two RSU reconstruction units is the same. The height of the encoder refers to the number of coding layers in the encoder.
[0036] The first N-1 RSU reconstruction units each have an RSU encoder, and the data output port of the RSU encoder of the first N-2 RSU reconstruction units is connected to the data input port of the RSU encoder of the next adjacent RSU reconstruction unit.
[0037] In the first N-1 RSU reconstruction units, the data input port of the wavelet dense residual filter is connected to the data output port of the RSU decoder of the subsequent adjacent RSU reconstruction unit, and the data input port of the hierarchical fusion splicing module is connected to the data output port of the RSU encoder of this unit and the data output port of the wavelet dense residual filter of this unit respectively.
[0038] The last RSU reconstruction unit has two RSU encoders, namely the first RSUF encoder and the second RSUF encoder. The data input port of the first RSUF encoder is connected to the data output port of the RSU encoder of the preceding adjacent RSU reconstruction unit. The data output port of the first RSUF encoder is connected to the data input port of the second RSU encoder. The data output port of the second RSU encoder is connected to the data input port of the wavelet dense residual filter in this unit and the data input port of the subsequent fusion and splicing module.
[0039] In the last RSU reconstruction unit, the data input port of the hierarchical fusion and splicing module is connected to the data output port of the first RSUF encoder and the data output port of the wavelet dense residual filter of this unit, respectively.
[0040] In each RSU reconstruction unit, the data output port of the hierarchical fusion and splicing module is connected to the data input port of the RSU decoder in this unit, and the data output port of the RSU decoder is connected to the data input port of the subsequent fusion and splicing module.
[0041] like Figure 1As shown in the figure, in this embodiment of the invention, there are five RSU reconstruction units (i.e., N=5). In the figure, RSU7 on the left is the RSU encoder of the first RSU reconstruction unit (height 7), and RSU7 on the right is the RSU decoder of the first RSU reconstruction unit. RSU6 on the left is the RSU encoder of the second RSU reconstruction unit (height 6), and RSU6 on the right is the RSU decoder of the second RSU reconstruction unit. RSU5 on the left is the RSU encoder of the third RSU reconstruction unit (height 5), and RSU5 on the right is the RSU decoder of the third RSU reconstruction unit. RSU4 on the left is the RSU encoder of the fourth RSU reconstruction unit (height 4), and RSU4 on the right is the RSU decoder of the fourth RSU reconstruction unit. RSU4F1 on the left is the first RSUF encoder of the fifth RSU reconstruction unit (height 4), RSU4F2 on the left is the second RSUF encoder of the fifth RSU reconstruction unit (height 4), and RSU4F on the right is the RSU decoder of the fifth RSU reconstruction unit. c in the figure represents the hierarchical fusion and splicing module. The number of RSU reconstruction units in other embodiments can also be other values.
[0042] The embodiments of the present invention also provide the use of the above-described U-based 2 The image processing method of the Net image processing system includes the following steps:
[0043] 1) Obtain the target image to be processed, select the target object and target features in the target image, and input the target image into the RSU encoder of the first RSU reconstruction unit;
[0044] The target image is a medical image, the target object is a lesion tissue and / or organ, and the target features include the texture features and edge contour features of the target object, as well as the contour features of the associated tissues between the target object and the surrounding tissues.
[0045] 2) The RSU encoder extracts target features from the received image and defines the image constructed from the extracted target features as the first feature map;
[0046] The RSU encoders of the first N-1 RSU reconstruction units output the first feature map they have constructed to the hierarchical fusion and stitching module of this unit, and then perform pooling processing on the first feature map to form a low-resolution downsampled image, which is then output to the subsequent adjacent RSU encoders.
[0047] In the last RSU reconstruction unit, the first RSUF encoder outputs its own constructed first feature map to the unit's hierarchical fusion and stitching module, and performs pooling processing on the first feature map to form a low-resolution downsampled image, which is then output to the second RSUF encoder. The second RSUF encoder outputs its own constructed first feature map to the wavelet dense residual filter and subsequent fusion and stitching module of the unit.
[0048] 3) In each RSU reconstruction unit, the wavelet dense residual filter performs wavelet decomposition on the received image to generate four target feature component maps. Then, the four target feature component maps generated by the wavelet dense residual filter are upsampled so that the resolution of the four target feature component maps is consistent with the resolution of the first feature map received by the hierarchical fusion and stitching module of this unit. Then, the hierarchical fusion and stitching module fuses and stitches the received images (the first feature map and the four target feature component maps after upsampling) into a single image and outputs it to the RSU decoder of this unit.
[0049] After the RSU decoder decodes the received image, it outputs the image to the subsequent fusion and stitching module and the wavelet dense residual filter in the preceding adjacent RSU reconstruction unit.
[0050] 4) The post-stage fusion and stitching module upsamples the images input to the RSU decoder of non-first RSU reconstruction units so that the resolution of these images is consistent with the resolution of the images input to the RSU decoder of the first RSU reconstruction unit. Then, it fuses and stitches all the received images into a single image, which is then output as the final image.
[0051] In this embodiment of the invention, the method for extracting target features from an image by the RSU encoder, the method for decoding an image by the RSU decoder, and the method for performing wavelet decomposition on an image are all existing technologies and are described in many patent documents. Therefore, they will not be repeated in this embodiment.
[0052] In this embodiment of the invention, when the RSU encoder of the first N-1 RSU reconstruction units extracts target features from the image, it performs convolution operations through multiple coding layers and downsamples layer by layer from top to bottom to achieve feature extraction. Then, it performs upsampling layer by layer from bottom to top through multiple coding layers to restore the image to its initial resolution (this method is existing technology and is a conventional technique for RSU encoders to extract target features). When the RSU encoder of the last RSU reconstruction unit extracts target features from the image, it performs convolution operations through multiple coding layers to achieve feature extraction.
[0053] The system and method of this invention are used to segment images of lesions and / or organs from medical images. They can be applied to medical electronic devices such as CT and MRI, as well as to other forms of 2D medical images.
[0054] Figure 2 This is an image showing the effect of segmenting lesion tissue and / or organ from a medical image using the system and method of this invention.
[0055] Figure 2 The first column contains four original medical images, which are labeled as image 1, image 2, image 3, and image 4 from top to bottom. The second column contains four actual images of the target features (lesion tissue and / or organ) in image 1, image 2, image 3, and image 4 from top to bottom.
[0056] The four images in the third column are target feature images segmented from four medical images using the system and method of this invention (the final images output in step 4 above). The four images in the third column, from top to bottom, are the target feature segmentation images in image 1, image 2, image 3, and image 4, respectively.
[0057] from Figure 2 It can be seen that the target feature images segmented from the four medical images using the system and method of this invention are basically consistent with the actual target feature images.
Claims
1. A U 2 image processing system based on U The RSU reconstruction unit comprises a wavelet residual filter, a hierarchical fusion splicing module, and U 2 The RSU encoder and the RSU decoder of Net, the RSU encoder and the RSU decoder in the same RSU reconstruction unit are symmetric encoder-decoder structures, and the N RSU reconstruction units sequentially form a sequence from front to back according to the execution order. The mathematical model of the wavelet secret residual filter is: ； wherein I is the image to be processed, is the low frequency approximation component of the image I in the i-th wavelet decomposition, is the horizontal high frequency detail component of the image I in the i-th wavelet decomposition, is the vertical high frequency detail component of the image I in the i-th wavelet decomposition, is the diagonal high frequency detail component of the image I in the i-th wavelet decomposition, is the wavelet transform function, is the inverse wavelet transform function, is the low frequency component of the layer, is the low frequency approximation component of the image I in the i-th wavelet decomposition, is the horizontal high frequency detail component of the image I in the i-th wavelet decomposition, is the vertical high frequency detail component of the image I in the i-th wavelet decomposition, is the diagonal high frequency detail component of the image I in the i-th wavelet decomposition, ; The post-stage fusion splicing module and the hierarchical fusion splicing module are used for fusing and splicing the received multiple images into a single image; The heights of the RSU encoders of the first N-1 RSU reconstruction units decrease from front to back, and the heights of the RSU encoders of the last two RSU reconstruction units are the same, where the height of the encoder refers to the number of encoding layers in the encoder; Each of the first N-1 RSU reconstruction units has one RSU encoder, and the data output port of the RSU encoder of each of the first N-2 RSU reconstruction units is connected to the data input port of the RSU encoder of the next RSU reconstruction unit; In the first N-1 RSU reconstruction units, the data input port of the wavelet secret residual filter is connected to the data output port of the RSU decoder of the next RSU reconstruction unit, and the data input port of the hierarchical fusion splicing module is connected to the data output port of the RSU encoder of the unit and the data output port of the wavelet secret residual filter of the unit, respectively; The last RSU reconstruction unit has two RSU encoders, which are a first RSU encoder and a second RSU encoder, the data input port of the first RSU encoder is connected to the data output port of the RSU encoder of the previous RSU reconstruction unit, the data output port of the first RSU encoder is connected to the data input port of the second RSU encoder, and the data output port of the second RSU encoder is connected to the data input port of the wavelet secret residual filter of the unit and the data input port of the post-stage fusion splicing module, respectively; In the last RSU reconstruction unit, the data input port of the hierarchical fusion splicing module is connected to the data output port of the first RSU encoder and the data output port of the wavelet secret residual filter of the unit, respectively; In each RSU reconstruction unit, the data output port of the hierarchical fusion splicing module is connected to the data input port of the RSU decoder of the unit, and the data output port of the RSU decoder is connected to the data input port of the post-stage fusion splicing module.
2. The U 2 An image processing method of an image processing system based on U The method comprises the following steps: 1) obtaining a target image to be processed, selecting a target object and a target feature in the target image, and inputting the target image into the RSU encoder of the first RSU reconstruction unit; The target feature includes the texture feature, the edge contour feature of the target object, and the contour feature of the associated tissue between the target object and the surrounding tissue; 2) the RSU encoder extracts the target feature from the received image, and defines the image constructed by the extracted target feature as a first feature map; The RSU encoder of the first N-1 RSU reconstruction units outputs the first feature map constructed by itself to the hierarchical fusion splicing module of the unit, and forms a low-resolution down-sampled image by performing pooling processing on the first feature map and outputs the low-resolution down-sampled image to the next RSU encoder; In the last RSU reconstruction unit, the first RSUF encoder outputs the first feature map constructed by itself to the unit-level fusion splicing module, and adopts pooling processing on the first feature map to form a low-resolution down-sampled image output to the second RSUF encoder, and the second RSUF encoder outputs the first feature map constructed by itself to the wavelet dense residual filter and the subsequent fusion splicing module; 3) In each RSU reconstruction unit, the wavelet dense residual filter generates four target feature component maps after wavelet decomposition on the received image, and then performs up-sampling on the four target feature component maps generated by the wavelet dense residual filter, so that the resolution of the four target feature component maps is consistent with the resolution of the first feature map received by the unit-level fusion splicing module, and then the level fusion splicing module fuses and splices the received images into a single image and outputs to the RSU decoder in the unit; The RSU decoder decodes the received image and outputs to the subsequent fusion splicing module and the wavelet dense residual filter in the previous adjacent RSU reconstruction unit; 4) The subsequent fusion splicing module performs up-sampling on the images input by the RSU decoder of the non-first RSU reconstruction unit, so that the resolution of these images is consistent with the resolution of the image input by the RSU decoder of the first RSU reconstruction unit, and then fuses and splices all the received images into a single image as the final image.