A long-short exposure tumor image-based tumor image denoising and deblurring method

Abstract

The present application relates to a kind of based on long short exposure tumor image tumor image denoising deblurring method, it is related to image processing technical field.The present application in the process of collecting tumor image, according to the first exposure length acquisition first tumor image, according to the second exposure length acquisition second tumor image;First tumor image and second tumor image are aligned, and first exposure length is less than second exposure length;The first tumor image and second tumor image of alignment are carried out uniform sampling splicing fusion and obtain fusion image;Denoising deblurring model is constructed, and the denoising deblurring model is according to the fusion image and carries out denoising deblurring operation and obtains high-quality tumor image;Wherein, the denoising deblurring model includes: multiscale downsampling module, the inverse rearrangement encoder formed by the pixel inverse rearrangement module and multiscale downsampling module of cascade, multiscale coding feature extraction network, the pixel rearrangement decoder formed by the pixel rearrangement module and convolution layer of cascade.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of tumor image processing, and in particular to a tumor image denoising and deblurring method based on long and short exposure tumor images. BACKGROUND

[0002] Tumor medical images are important basis for diagnosing and analyzing tumors. In the process of tumor medical image acquisition, the longer the imaging exposure time, the clearer the edge imaging, but the more noise it contains. The longer the imaging exposure time, the smaller the overall noise, but the longer the exposure time, the more obvious the influence of patient movement, and the imaging will produce edge blur. Neither edge blur nor imaging noise is conducive to subsequent tumor diagnosis and analysis. In the actual tumor acquisition process, medical image technicians need to grasp the exposure time to obtain high-quality tumor images. A tumor image denoising and deblurring method based on long and short exposure tumor images is needed. SUMMARY

[0003] In order to solve the above technical problems or at least partially solve the above technical problems, the present application provides a tumor image denoising and deblurring method based on long and short exposure tumor images.

[0004] In the first aspect, the present application provides a tumor image denoising and deblurring method based on long and short exposure tumor images, comprising:

[0005] In the process of acquiring tumor images, a first tumor image is acquired according to a first exposure time, and a second tumor image is acquired according to a second exposure time; the first tumor image and the second tumor image are aligned, and the first exposure time is less than the second exposure time;

[0006] The aligned first tumor image and the second tumor image are uniformly sampled, spliced and fused to obtain a fused image;

[0007] A denoising and deblurring model is constructed, which performs denoising and deblurring operations according to the fused image to obtain a high-quality tumor image; wherein the denoising and deblurring model comprises: a multi-scale down-sampling module, a pixel inverse rearrangement module and a multi-scale down-sampling module forming an inverse rearrangement encoder, a multi-scale coding feature extraction network, a pixel rearrangement module and a convolution layer forming a pixel rearrangement decoder.

[0008] Further, the uniform sampling, splicing and fusion of the aligned first tumor image and the second tumor image to obtain the fused image comprises:

[0009] A blank image equal in size to the first tumor image and the second tumor image is constructed;

[0010] The first tumor image and the second tumor image are aligned in brightness;

[0011] For the first tumor image, starting from position (0, 0), every other row and every other column of pixels is filled into the corresponding position of the empty image; starting from position (1, 1), every other row and every other column of pixels is filled into the corresponding position of the empty image;

[0012] For the second tumor image, starting from position (0, 1), every other row and every other column of pixels is filled into the corresponding position of the empty image; starting from position (1, 0), every other row and every other column of pixels is filled into the corresponding position of the empty image;

[0013] The fused image is subjected to gamma correction.

[0014] Further, the process of brightness alignment of the first tumor image and the second tumor image includes: converting the first tumor image and the second tumor image to the LAB color space, extracting the brightness channel of the LAB color space, and calculating the ratio of the brightness channel values in the range of 0.5-2 as the brightness alignment coefficient through the clip operation: ratio = np.clip(l_long / (l_short + 0.1), 0.5, 2.0); wherein l_short is the brightness channel value of the first tumor image in the LAB color space, l_long is the brightness channel value of the second tumor image in the LAB color space, and the brightness alignment coefficient is multiplied by the brightness channel of the first tumor image and then restored to the color space.

[0015] Further, the inverse rearrangement encoder includes two cascaded pixel inverse rearrangement modules and a multi-scale down-sampling module.

[0016] The spatial dimension data of the fused image is rearranged to the channel dimension by the two cascaded pixel inverse rearrangement modules, the spatial dimension scale is down-sampled, and the channel dimension scale is expanded to obtain the first rearranged feature map and the second rearranged feature map.

[0017] The multi-scale down-sampling module realizes 2-fold, 4-fold and 8-fold down-sampling of the fused image through interpolation reduction to obtain the multi-scale down-sampled feature map of the fused image.

[0018] Further, the pixel inverse rearrangement module arranges the input spatial dimension (height, width) into the form of (height, down-sampling coefficient, width, down-sampling coefficient), and then moves the dimension corresponding to the down-sampling coefficient to the channel dimension.

[0019] Further, the multi-scale encoding feature extraction network includes: an input enhancement module as an encoder unit, and the input enhancement module cascades an encoder convolution layer; the encoder convolution layer is also an encoder unit.

[0020] The input enhancement module and the corresponding decoder convolution layer are cascaded; a bottleneck composed of a convolution layer and a residual convolution is arranged between the input enhancement module and the corresponding decoder convolution layer and between the corresponding encoder convolution layer and the decoder convolution layer, wherein the depth of the residual convolution in the bottleneck increases with the increase of the depth of the encoding; the input of the bottleneck is the interpolation up-sampling result of the output feature map of the middle part of the previous level bottleneck and the output splicing of the encoder unit at the same level; the output of any level bottleneck is spliced with the interpolation up-sampling result of the previous level decoder convolution layer and then input to the decoder convolution layer at the current level.

[0021] Further, the inverse rearrangement encoder provides a second rearranged feature map and multi-scale down-sampled feature maps to the input enhancement module, the input enhancement module aligns each multi-scale down-sampled feature map in the spatial dimension with the second rearranged feature map through up-sampling and down-sampling, splices all the down-sampled feature maps aligned in the spatial dimension with the second rearranged feature map, performs convolution fusion and global average pooling after splicing, compresses and restores the channels through a linear layer to generate attention weights, normalizes the attention weights through Sigmoid, and weights the convolution fusion result; after weighting, the result is spliced with the result of the convolution layer processing of the second rearranged feature map, convolution fusion is performed to obtain an enhanced input feature map.

[0022] Further, the pixel rearrangement decoder includes two groups of cascaded pixel rearrangement decoding units, each group of pixel rearrangement decoding units including a pixel rearrangement module processing the convolution layer output by the corresponding level pixel inverse rearrangement module, splicing and convolution layer.

[0023] Further, one pixel rearrangement decoding unit uses the pixel rearrangement module to rearrange the pixels of the multi-scale aggregated feature map, splices the one-time pixel rearrangement result of the multi-scale aggregated feature map with the convolution result output by the corresponding level pixel inverse rearrangement module, and then fuses through a convolution layer after splicing; another pixel rearrangement decoding unit uses the pixel rearrangement module to continue to rearrange the pixels of the fused feature map, splices the two-time pixel rearrangement result of the multi-scale aggregated feature map with the convolution result output by the corresponding level pixel inverse rearrangement module, and then maps into a denoising and deblurring image through a convolution layer.

[0024] Further, the spatial and frequency domain LI distances between the generated denoising and deblurring image and the real denoising and deblurring image are taken as a loss function, and the parameters of the denoising and deblurring model are adjusted to minimize the loss function.

[0025] In a second aspect, the present application provides a device for de-noising and de-blurring a tumor image based on long and short exposure tumor images, comprising: at least one processing unit, the processing unit being connected to a storage unit through a bus unit, the storage unit storing a computer program, the computer program being executed by the processing unit to implement the method for de-noising and de-blurring a tumor image based on long and short exposure tumor images.

[0026] In a third aspect, the present application provides a computer readable storage medium storing a computer program, the computer program being executed by a processor to implement the method for de-noising and de-blurring a tumor image based on long and short exposure tumor images.

[0027] The above technical solutions provided by the embodiments of the present application have the following advantages compared with the prior art:

[0028] The present application provides a de-noising and de-blurring method based on long and short exposure tumor image fusion. The first tumor image and the second tumor image are collected in the same collection process to ensure that the spatial positions of the two images are completely matched, avoiding misalignment in subsequent fusion. The exposure time of the first tumor image is short, the motion blur is less, the clear edge is retained, but the noise is more. The exposure time of the second tumor image is long, the noise is reduced, and the color is more accurate, but the blur may be caused by target movement or device shaking. By combining the advantages of tumor images with different exposure times, the deep learning model uses the clear edge information of the first tumor image with short exposure to suppress motion blur and uses the low noise information of the second tumor image with long exposure to suppress noise, solves the limitation of single exposure, and realizes high-quality tumor medical image reconstruction.

[0029] In the present application, the brightness of the first tumor image with short exposure is adjusted by the ratio of long exposure brightness (l_long) to short exposure brightness (l_short), avoiding brightness difference caused by exposure difference.

[0030] In the present application, the first tumor image and the second tumor image are fused to form a fused image. The first tumor image with short exposure provides high-frequency details such as clear edges, and the second tumor image with long exposure supplements low-frequency information such as smooth areas with less noise. The resolution of the fused image remains unchanged, and gamma correction is performed adaptively to enhance dark details or suppress overexposed areas, improving the fusion quality.

[0031] In the present application, the noise reduction deblurring model; wherein, the noise reduction deblurring model includes: the inverse rearrangement encoder formed by the cascaded pixel inverse rearrangement module and the multi-scale down-sampling module, the multi-scale coding feature extraction network, the pixel rearrangement decoder formed by the cascaded pixel rearrangement module and the convolution layer; through the inverse rearrangement coding and the multi-scale down-sampling feature combination, the multi-scale coding feature extraction, the local noise and the global blur are effectively captured, so as to provide the high-quality multi-scale aggregation feature map for generating the high-quality noise reduction deblurring tumor image. And then, the pixel rearrangement decoder effectively generates the high-quality noise reduction deblurring tumor image according to the multi-scale aggregation feature map. BRIEF DESCRIPTION OF DRAWINGS

[0032] The accompanying drawings, which are incorporated herein and constitute part of the specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the application.

[0033] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or the prior art description. Obviously, for those skilled in the art, other drawings can also be obtained from these drawings without any creative labor.

[0034] Figure 1 A flowchart of a tumor image noise reduction deblurring method based on long-short exposure tumor images is provided for the embodiments of the present application;

[0035] Figure 2 A flowchart of uniformly sampling and splicing and fusing the aligned first tumor image and the second tumor image to obtain a fused image is provided for the embodiments of the present application;

[0036] Figure 3 A schematic diagram of a noise reduction deblurring model is provided for the embodiments of the present application;

[0037] Figure 4 A schematic diagram of a visual language model is provided for the embodiments of the present application;

[0038] Figure 5 A schematic diagram of a tumor image noise reduction deblurring device based on long-short exposure tumor images is provided for the embodiments of the present application. DETAILED DESCRIPTION

[0039] In order to make the objects, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without any creative labor are within the protection scope of the present application.

[0040] It should be noted that, in this document, the terms "comprising", "including", or any other variant thereof are intended to cover a non-exclusive inclusion, such that processes, methods, articles, or apparatuses that comprise a list of elements are not limited to those elements, but can include other elements not expressly listed or inherent to such processes, methods, articles, or apparatuses. Without further limitation, an element defined by the statement "comprising a" does not exclude the presence of additional identical elements in the process, method, article, or apparatus that includes the element.

[0041] Embodiment 1

[0042] As shown in the long-short exposure tumor image-based tumor image denoising and deblurring method includes: Figure 1

[0043] In the process of collecting tumor images, a first tumor image is collected according to a first exposure time, and a second tumor image is collected according to a second exposure time. In the specific implementation process, the first tumor image and the second tumor image are collected in the same collection process to ensure that the first tumor image and the second tumor image are aligned; in this collection process, the first exposure time of the first tumor image is less than the second exposure time of the second tumor image.

[0044] The aligned first tumor image and the second tumor image are uniformly sampled and spliced and fused to obtain a fused image; as shown in the long-short exposure tumor image-based tumor image denoising and deblurring method includes: Figure 2

[0045] An empty image as large as the first tumor image and the second tumor image is constructed;

[0046] The first tumor image and the second tumor image are aligned in brightness, as shown in the long-short exposure tumor image-based tumor image denoising and deblurring method includes: Figure 4 The process includes: converting the first tumor image and the second tumor image to the LAB color space, and extracting the brightness channel of the LAB color space; calculating the ratio of the brightness channel values as the brightness alignment coefficient through the clip operation in the range of 0.5-2: ratio = np.clip(l_long / (l_short + 0.1), 0.5, 2.0); wherein l_short is the brightness channel value of the first tumor image in the LAB color space, and l_long is the brightness channel value of the second tumor image in the LAB color space; the brightness alignment coefficient ratio is multiplied by the brightness channel of the first tumor image, and then the color space is restored to align the brightness information difference caused by different exposure times;

[0047] ​​After the brightness alignment, for the first tumor image, starting from position (0, 0), every other row and every other column of pixels are filled into the corresponding positions of the empty image, and the pixels at positions (0, 0), (0, 2), (0, 4) … (2, 0), (2, 2), (2, 4) … are added to the empty image; starting from position (1, 1), every other row and every other column of pixels are filled into the corresponding positions of the empty image, and the pixels at positions (1, 1), (1, 3), (1, 5) … (3, 1), (3, 3), (3, 5) … are added to the empty image;

[0048] For the second tumor image, starting from position (0, 1), every other row and every other column of pixels are filled into the corresponding positions of the empty image, and the pixels at positions (0, 1), (0, 3), (0, 5) … (2, 1), (2, 3), (2, 5) … are added to the empty image; starting from position (1, 0), every other row and every other column of pixels are filled into the corresponding positions of the empty image, and the pixels at positions (1, 0), (1, 2), (1, 4) … (3, 0), (3, 2), (3, 4) … are added to the empty image.

[0049] The fusion image is subjected to gamma correction. In a specific implementation process, a trainable fully connected network is provided as a gamma correction network, which generates gamma correction parameters.

[0050] The fusion image is a mixed image, part of the pixels come from the first tumor image with short exposure, which retains clear edges but has more noise information, and part of the pixels come from the second tumor image with long exposure, which has smaller noise and more accurate color; the fusion image fuses the information of the first tumor image and the second tumor image with two exposures, so that noise reduction can be performed through the information provided by the second tumor image, and deblurring can be performed using the information provided by the first tumor image.

[0051] A denoising and deblurring model is constructed and trained; as shown in Figure 3 the denoising and deblurring model includes: an inverse rearrangement encoder formed by cascading a pixel inverse rearrangement module and a multi-scale downsampling module, a multi-scale coding feature extraction network, and a pixel rearrangement decoder formed by cascading a pixel rearrangement module and a convolution layer;

[0052] The pre-trained denoising and deblurring model performs denoising and deblurring operations on the fusion image to obtain a denoised and deblurred tumor image.

[0053] The multi-scale down-sampling module realizes 2 times, 4 times and 8 times down-sampling of the fusion image through interpolation zooming to obtain multi-scale down-sampling feature maps of the fusion image; in the specific implementation process, the dimension of the fusion image is (H, W, C), the dimension of the down-sampling feature map of 2 times down-sampling is (H / 2, W / 2, C), the dimension of the down-sampling feature map of 4 times down-sampling is (H / 4, W / 4, C), and the dimension of the down-sampling feature map of 8 times down-sampling is (H / 8, W / 8, C).

[0054] The inverse rearrangement encoder comprises two cascaded pixel inverse rearrangement modules and a multi-scale down-sampling module; the spatial dimension data of the fusion image is rearranged to the channel dimension, the spatial dimension scale is down-sampled, and the channel dimension scale is expanded through the two cascaded pixel inverse rearrangement modules to obtain a first rearranged feature map and a second rearranged feature map; wherein the dimension of the first rearranged feature map is (H / 2, W / 2, 4C), the second rearranged feature map is obtained by further inverse rearranging the first rearranged feature map, and the dimension of the second rearranged feature map is (H / 4, W / 4, 16C); in the specific implementation process, the pixel inverse rearrangement module arranges the input spatial dimension (height, width) into the form of (height, down-sampling coefficient, width, down-sampling coefficient), and then moves the dimension corresponding to the down-sampling coefficient to the channel dimension.

[0055] The multi-scale encoding feature extraction network comprises an input enhancement module as an encoder unit, and the input enhancement module is cascaded with an encoder convolution layer; the encoder convolution layer is also an encoder unit. The specific structure of the multi-scale encoding feature extraction network is as follows:

[0056] The input enhancement module and the encoder convolution layer are correspondingly provided with cascaded decoder convolution layers; the input enhancement module and the corresponding decoder convolution layer, and the corresponding encoder convolution layer and decoder convolution layer are all provided with a bottleneck composed of a convolution layer and a residual convolution, wherein the depth of the residual convolution in the bottleneck increases with the increase of the depth of the encoding; the input of the bottleneck is the interpolation up-sampling result of the output feature map of the middle part of the previous level bottleneck and the output of the same level encoder unit (input enhancement module or encoder convolution layer) spliced together, and the output of any level bottleneck is input into the decoder convolution layer of the current level after splicing the interpolation up-sampling result of the previous level decoder convolution layer.

[0057] In the implementation process, the inverse rearrangement encoder provides the second rearrangement feature map and the multi-scale down-sampling feature map to the input enhancement module, the input enhancement module uses interpolation to up-sample the 8 times down-sampled down-sampling feature map, uses interpolation to down-sample the 2 times down-sampled down-sampling feature map, and aligns the spatial dimensions of the second rearrangement feature map; the 4 times down-sampled down-sampling feature map is originally aligned with the second rearrangement feature map, and then the spatial dimensions are unified; the all down-sampling feature maps with a spatial dimension of (H / 4, W / 4) are spliced with the second rearrangement feature map, convolution fusion and global average pooling are performed after splicing, the channel is compressed and restored through a linear layer to generate attention weights, the attention weights are normalized through Sigmoid, and then weighted with the convolution fusion result; after weighting, the result of convolution layer processing is spliced with the result of convolution layer processing of the second rearrangement feature map, convolution fusion is performed, and an enhanced input feature map is obtained. The enhanced input feature map combines multi-scale down-sampling feature map information and second rearrangement feature map information, introduces pixel continuity information through the multi-scale down-sampling feature map, and can eliminate artifacts caused by pixel discontinuity of the second rearrangement feature map. The input enhancement module is a multi-scale feature enhancement module for processing input features and second rearrangement features of different down-sampling scales, and improving input representation ability through channel attention mechanism and feature fusion.

[0058] The multi-scale coding feature extraction network transmits and fuses information of the enhanced input feature map containing short exposure and long exposure information from different scales, models noise and blur, and obtains a multi-scale aggregation feature map.

[0059] The pixel rearrangement decoder includes two groups of cascaded pixel rearrangement decoding units, each group of pixel rearrangement decoding units including a pixel rearrangement module processing a convolution layer output by a corresponding level pixel inverse rearrangement module, splicing and a convolution layer.

[0060] In the specific implementation process, one pixel rearrangement decoding unit uses a pixel rearrangement module to rearrange pixels of the multi-scale aggregation feature map, and the first pixel rearrangement result of the multi-scale aggregation feature map is spliced with the convolution result output by the corresponding level pixel inverse rearrangement module, i.e., the convolution result of the first rearrangement feature map, and then fused through a convolution layer after splicing; another pixel rearrangement decoding unit uses a pixel rearrangement module to continue to rearrange pixels of the fused feature map, and the second pixel rearrangement result of the multi-scale aggregation feature map is spliced with the convolution result output by the corresponding level pixel inverse rearrangement module, i.e., the convolution result of the fused image, and then mapped into a denoising and deblurring image through a convolution layer.

[0061] In the specific implementation process, the pixel rearrangement module rearranges data in the channel dimension to the spatial dimension; the pixel inverse rearrangement module splits the input channel dimension into (original channel / up-sampling coefficient square, up-sampling coefficient, up-sampling coefficient), and then adjusts the dimension order to interleave the dimensions corresponding to the up-sampling coefficient and the height and width.

[0062] As a preferred embodiment, the dynamic depth separable convolution is adopted for the convolution layer involving the multi-channel feature map in the present application to reduce the calculation amount, and the spatial dimension convolution kernel of each channel of the dynamic depth separable convolution adopts the adaptive average pooling and the linear layer to construct a dynamic convolution kernel. After the spatial dimension convolution kernel convolves each channel of the feature map, the dynamic depth separable convolution utilizes the 1x1 convolution to perform the cross-channel information fusion. The calculation cost is reduced, and the complex tumor image characteristics are adapted.

[0063] In order to enable the denoising deblurring model to effectively perform denoising deblurring, the present application takes the spatial and frequency domain L1 distance between the generated denoising deblurring image and the real denoising deblurring image as the loss function, takes the minimum loss function as the goal, adjusts the parameters of the denoising deblurring model, and makes the generated denoising deblurring tumor image close to the real denoising deblurring tumor image. The spatial L1 distance loss directly constrains the pixel-level difference between the generated tumor image and the real denoising deblurring tumor image, and retains the details. The calculation steps of the frequency domain L1 distance loss: Fourier transform is performed on the generated tumor image and the real denoising deblurring tumor image, and the L1 distance of the frequency domain amplitude spectrum is calculated. The frequency domain L1 distance loss strengthens the repair ability of the model to high-frequency noise and blur, and improves the overall structural consistency. The joint optimization of the spatial domain and the frequency domain can balance the detail restoration and the global smoothing.

[0064] Embodiment 2

[0065] Referring to Figure 5 As shown in the figure, the embodiment of the present application provides a tumor image denoising deblurring device based on long-short exposure tumor images, which comprises at least one processing unit connected with a storage unit through a bus unit, and the storage unit is a computer readable storage medium, which can be used to store software programs, computer executable programs and modules, such as the software programs, computer executable programs and modules corresponding to the tumor image denoising deblurring method based on long-short exposure tumor images in the embodiment of the present application. The processing unit realizes the above-mentioned tumor image denoising deblurring method based on long-short exposure tumor images by running the software programs, computer executable programs and modules stored in the storage unit, which comprises:

[0066] In the process of collecting tumor images, a first tumor image is collected according to a first exposure time, and a second tumor image is collected according to a second exposure time; the first tumor image and the second tumor image are aligned, and the first exposure time is less than the second exposure time;

[0067] The aligned first tumor image and the second tumor image are uniformly sampled and spliced to obtain a fused image;

[0068] construct and train a denoising deblurring model; wherein the denoising deblurring model comprises: an inverse rearrangement encoder formed by cascading a pixel inverse rearrangement module and a multi-scale down-sampling module, a multi-scale coding feature extraction network, and a pixel rearrangement decoder formed by cascading a pixel rearrangement module and a convolution layer;

[0069] The pre-trained denoising deblurring model performs denoising deblurring operation on the fused image to obtain a denoising deblurred tumor image.

[0070] Of course, the computer program stored in the storage unit of the tumor image denoising deblurring device based on long-short exposure tumor images provided by the embodiments of the present application is not limited to the method operations described above, and can also perform related operations in the tumor image denoising deblurring method based on long-short exposure tumor images provided by any embodiment of the present application.

[0071] Embodiment 3

[0072] The embodiments of the present application provide a computer readable storage medium storing a computer program, which, when executed, implements the tumor image denoising deblurring method based on long-short exposure tumor images, comprising:

[0073] In the process of collecting the tumor image, a first tumor image is collected according to a first exposure time, and a second tumor image is collected according to a second exposure time; the first tumor image and the second tumor image are aligned, and the first exposure time is less than the second exposure time;

[0074] The aligned first tumor image and the second tumor image are uniformly sampled and spliced to obtain a fused image;

[0075] construct and train a denoising deblurring model; wherein the denoising deblurring model comprises: an inverse rearrangement encoder formed by cascading a pixel inverse rearrangement module and a multi-scale down-sampling module, a multi-scale coding feature extraction network, and a pixel rearrangement decoder formed by cascading a pixel rearrangement module and a convolution layer;

[0076] The pre-trained denoising deblurring model performs denoising deblurring operation on the fused image to obtain a denoising deblurred tumor image.

[0077] The computer program stored in the computer readable storage medium provided by the embodiments of the present application is not limited to the method operations described above, and can also perform related operations in the tumor image denoising deblurring method based on long-short exposure tumor images provided by any embodiment of the present application.

[0078] In the embodiments of the present application, it should be understood that the disclosed structure and method can be implemented in other manners. For example, the described embodiments of the structure are merely schematic, and the division of the units is merely a logical function division, and there can be another division manner in actual implementation. For example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the displayed or discussed mutual couplings or direct couplings or communication connections can be indirect couplings or communication connections through some interfaces, structures or units, and can be in electrical, mechanical or other forms.

[0079] The units described as separated components can or can not be physically separated, and the components displayed as units can or can not be physical units, i.e., can be located in one place, or can be distributed on a plurality of network units. In actual implementation, some or all of the units can be selected according to actual needs to achieve the purposes of the embodiments of the present application.

[0080] In addition, each function unit in the various embodiments of the present application can be integrated in a processing unit, or each unit can be a physically independent unit, or two or more units can be integrated in a unit. The integrated unit can be implemented in a form of hardware, or in a form of software functional unit.

[0081] The above descriptions are merely specific embodiments of the present application, enabling a person skilled in the art to understand or implement the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to these embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for denoising and deblurring tumor images based on long-short exposure tumor images, characterized in that, The method comprises the following steps: In the process of collecting tumor images, a first tumor image is collected according to a first exposure time, and a second tumor image is collected according to a second exposure time; the first tumor image and the second tumor image are aligned, and the first exposure time is less than the second exposure time; The aligned first tumor image and the second tumor image are uniformly sampled and spliced to obtain a fused image, comprising: constructing an empty image with the same size as the first tumor image and the second tumor image; aligning the brightness of the first tumor image and the second tumor image; for the first tumor image, starting from position (0, 0), every 2 rows and every 2 columns of a pixel is taken to fill the corresponding position of the empty image; starting from position (1, 1), every 2 rows and every 2 columns of a pixel is taken to fill the corresponding position of the empty image; for the second tumor image, starting from position (0, 1), every 2 rows and every 2 columns of a pixel is taken to fill the corresponding position of the empty image, and starting from position (1, 0), every 2 rows and every 2 columns of a pixel is taken to fill the corresponding position of the empty image; the fused image is gamma corrected; A denoising and deblurring model is constructed and trained; wherein the denoising and deblurring model comprises: an inverse rearrangement encoder formed by cascading a pixel inverse rearrangement module and a multi-scale down-sampling module, a multi-scale coding feature extraction network, and a pixel rearrangement decoder formed by cascading a pixel rearrangement module and a convolution layer; Wherein, the inverse rearrangement encoder comprises two cascaded pixel inverse rearrangement modules and a multi-scale down-sampling module; the spatial dimension data of the fused image is rearranged to the channel dimension by the two cascaded pixel inverse rearrangement modules, the spatial dimension scale is down-sampled, and the channel dimension scale is expanded to obtain a first rearranged feature map and a second rearranged feature map; the multi-scale down-sampling module realizes 2 times, 4 times and 8 times down-sampling of the fused image through interpolation reduction to obtain a multi-scale down-sampling feature map of the fused image; The multi-scale coding feature extraction network comprises: an input enhancement module as an encoder unit, and the input enhancement module cascades an encoder convolution layer; the encoder convolution layer is also an encoder unit; the input enhancement module and the encoder convolution layer correspondingly set cascaded decoder convolution layers; between the input enhancement module and its corresponding decoder convolution layer, and between the mutually corresponding encoder convolution layer and the decoding convolution layer, a bottleneck composed of a convolution layer and a residual convolution is arranged, wherein the depth of the residual convolution in the bottleneck increases with the increase of the depth of the coding; the input of the bottleneck is the interpolation up-sampling result of the output feature map of the middle part of the upper level bottleneck and the splicing of the output of the same level encoder unit, and the output of any level bottleneck is input into the decoder convolution layer of the current level after splicing the interpolation up-sampling result of the upper level decoder convolution layer; The inverse rearrangement encoder provides the second rearranged feature map and the multi-scale down-sampling feature map to the input enhancement module, the input enhancement module aligns each multi-scale down-sampling feature map in the spatial dimension to the second rearranged feature map in the form of up-sampling and down-sampling, splices all the spatially aligned down-sampling feature maps with the second rearranged feature map, performs convolution fusion and global average pooling after splicing, compresses and restores the channel through a linear layer to generate attention weights, normalizes the attention weights through Sigmoid, and then weights the convolution fusion result; after weighting, the result is spliced with the result of the convolution layer processing of the second rearranged feature map, convolution fusion is performed, and an enhanced input feature map is obtained; The pixel rearrangement decoder comprises two groups of cascaded pixel rearrangement decoding units, each group of pixel rearrangement decoding units comprising a pixel rearrangement module for processing the convolution layer output by the corresponding level pixel inverse rearrangement module, splicing and convolution layer; one pixel rearrangement decoding unit uses the pixel rearrangement module to rearrange the pixels of the multi-scale aggregation feature map, splices the one-time pixel rearrangement result of the multi-scale aggregation feature map with the convolution result output by the corresponding level pixel inverse rearrangement module, and then fuses through a convolution layer for fusion; the other pixel rearrangement decoding unit uses the pixel rearrangement module to continue to rearrange the pixels of the fused feature map, splices the two-time pixel rearrangement result of the multi-scale aggregation feature map with the convolution result output by the corresponding level pixel inverse rearrangement module, and then maps it into a denoised and deblurred image through a convolution layer. The pre-trained denoising and deblurring model performs denoising and deblurring operations on the fused image to obtain a denoised and deblurred tumor image.

2. The method of claim 1, wherein the method is a long-short-exposure tumor image based denoising and deblurring method for tumor images. The process of brightness alignment of the first tumor image and the second tumor image includes: converting the first tumor image and the second tumor image to the LAB color space, extracting the brightness channel of the LAB color space, and calculating the ratio of the brightness channel values in the range of 0.5-2 as the brightness alignment coefficient through the clip operation: ratio = np.clip(l_long / (l_short + 0.1), 0.5, 2.0); wherein l_short is the brightness channel value of the first tumor image in the LAB color space, l_long is the brightness channel value of the second tumor image in the LAB color space, and the brightness alignment coefficient is multiplied by the brightness channel of the first tumor image and then restored to the color space. 3.The method of claim 1, wherein the method further comprises: determining a first blur kernel for the first image; determining a second blur kernel for the second image; and determining a third blur kernel for the third image based on the first blur kernel and the second blur kernel. The pixel inverse rearrangement module arranges the input spatial dimensions (height, width) into the form of (height, down-sampling coefficient, width, down-sampling coefficient), and then moves the dimensions corresponding to the down-sampling coefficient to the channel dimension.

4. The method of claim 1, wherein the method is a long-short-exposure tumor image based denoising and deblurring method for tumor images. The spatial and frequency domain LI distances between the generated denoised and deblurred image and the real denoised and deblurred image are taken as the loss function, and the parameters of the denoising and deblurring model are adjusted to minimize the loss function.

Images