CT image noise reduction method, device and equipment based on regional BM3D algorithm and deep learning and medium

Abstract

The invention discloses a CT image noise reduction method, device and equipment based on a regional BM3D algorithm and deep learning, and a medium, and the method comprises the steps: obtaining an original workpiece image set collected through a CT system, and dividing each original workpiece image in the original workpiece image set into at least two image regions; performing image noise reduction on each image area in the original workpiece image by adopting a BM3D algorithm to generate a label workpiece image for each original workpiece image; training a preset noise reduction model based on the original workpiece image and the label workpiece image to obtain a CT image noise reduction model; and performing image noise reduction on a workpiece image acquired by a CT system by using the CT image noise reduction module. According to the method, deep learning is utilized to fit the noise reduction process of regional BM3D, the defects of an original BM3D algorithm are eliminated, adaptive noise reduction can be carried out on different regions, the overall noise reduction effect is improved, and meanwhile, the noise reduction speed is extremely high.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, in particular to a CT image denoising method and device based on a sub-region BM3D algorithm and deep learning, equipment and medium. BACKGROUND

[0002] As a kind of non-destructive testing, industrial CT uses CT equipment to scan the detected object, and the rays are attenuated by the object, and the penetrating rays are received by the detector, and then converted into digital signals (projection) by digital processing, and then reconstructed to obtain two-dimensional tomographic images or three-dimensional images that can show the internal structure characteristics, material composition and defect condition of the object. Its application is almost in all industrial fields. However, due to the influence of various factors such as CT device hardware noise (such as thermal noise, dark current noise, etc.), quantum noise and reconstruction algorithm error, the quality of the initial reconstructed image is generally low, so image denoising is needed to improve the image quality.

[0003] The existing CT denoising algorithm mainly includes traditional CT image denoising algorithm and deep learning CT image denoising algorithm. The commonly used denoising algorithm in traditional denoising algorithm is BM3D algorithm, which has good denoising performance in CT image denoising, but it cannot adaptively denoise different regions of the image. Its operation speed is slow due to the need for time-consuming block matching and two-step filtering.

[0004] Therefore, the prior art still needs to be improved and improved. SUMMARY

[0005] The technical problem to be solved by the present application is to provide a CT image denoising method, device, equipment and medium based on a sub-region BM3D algorithm and deep learning to solve the problems of the prior art.

[0006] To solve the above technical problems, the first aspect of the present application provides a CT image denoising method based on a sub-region BM3D algorithm and deep learning, wherein the CT image denoising method based on a sub-region BM3D algorithm and deep learning specifically comprises:

[0007] Obtain the original workpiece image set collected by the CT system, and divide each original workpiece image in the original workpiece image set into at least two image regions;

[0008] Using the BM3D algorithm to denoise each image region in the original workpiece image to generate a labeled workpiece image for each original workpiece image;

[0009] The preset denoising model is trained based on the original workpiece image and the labeled workpiece image to obtain a CT image denoising model.

[0010] The CT image denoising model is used to perform image denoising on a workpiece image collected by a CT system.

[0011] The CT image denoising method based on the regional BM3D algorithm and deep learning, wherein the original workpiece image is divided into at least two image regions specifically includes:

[0012] The workpiece type corresponding to the original workpiece image is obtained, and a region template corresponding to the workpiece type is obtained, wherein the region template includes at least two regions.

[0013] The region template is applied to the original workpiece image to divide the original workpiece image into at least two image regions.

[0014] The CT image denoising method based on the regional BM3D algorithm and deep learning, wherein the BM3D algorithm is used to perform image denoising on each image region in the original workpiece image to generate a labeled workpiece image for each original workpiece image specifically includes:

[0015] The denoising intensity of each image region is obtained.

[0016] Based on the denoising intensity of each image region, the BM3D algorithm is used to perform image denoising on each image region.

[0017] The denoised image regions are integrated to generate a labeled workpiece image for each original workpiece image.

[0018] The CT image denoising method based on the regional BM3D algorithm and deep learning, wherein the denoising intensity of each image region is obtained specifically includes:

[0019] For each image region in the at least two image regions, a Laplacian operator is used to perform a convolution operation on the image region, and the average pixel value of the image region after the convolution operation is calculated to obtain an initial denoising intensity.

[0020] A plurality of adjacent denoising intensities of the initial denoising intensity are obtained, and the image region is denoised using the initial denoising intensity and the plurality of adjacent denoising intensities, respectively.

[0021] According to the denoising results corresponding to the initial denoising intensity and each adjacent denoising intensity, the denoising intensity corresponding to the image region is selected from the initial denoising intensity and the plurality of adjacent denoising intensities.

[0022] The CT image denoising method based on the sub-region BM3D algorithm and deep learning, wherein any two adjacent image regions in the at least two image regions have an overlapping region; and the integrating the denoised image regions to generate a labeled workpiece image for each original workpiece image specifically comprises:

[0023] obtaining a weight transition change matrix corresponding to the overlapping region;

[0024] integrating the denoised image regions according to the weight transition change matrix to obtain a denoised CT image.

[0025] The CT image denoising method based on the sub-region BM3D algorithm and deep learning, wherein the CT image denoising model is based on a RED-CNN network, a NAF module is added before each convolution layer in the RED-CNN network and after each deconvolution layer in the RED-CNN network, and each convolution layer is used to perform a down-sampling operation.

[0026] The CT image denoising method based on the sub-region BM3D algorithm and deep learning, wherein the loss function used in the training process of the CT image denoising model comprises a prediction loss term, a perception loss term, an edge weight loss term and an image visual loss term.

[0027] The second aspect of the present application provides a CT image denoising device based on a sub-region BM3D algorithm and deep learning, wherein the CT image denoising device based on the sub-region BM3D algorithm and deep learning specifically comprises:

[0028] an acquisition module, configured to acquire an original workpiece image set collected by a CT system, and divide each original workpiece image in the original workpiece image set into at least two image regions;

[0029] a generation module, configured to perform image denoising on each image region in the original workpiece image by using a BM3D algorithm to generate a labeled workpiece image for each original workpiece image;

[0030] a training module, configured to train a preset denoising model based on the original workpiece image and the labeled workpiece image to obtain a CT image denoising model;

[0031] a denoising module, configured to perform image denoising on a workpiece image collected by the CT system by using the CT image denoising model.

[0032] The third aspect of the present application provides a computer readable storage medium, the computer readable storage medium stores one or more programs, the one or more programs can be executed by one or more processors to implement the steps of the CT image denoising method based on the sub-region BM3D algorithm and deep learning as any one of the above.

[0033] The fourth aspect of the present application provides a terminal device, comprising: a processor and a memory.

[0034] The memory stores a computer readable program that can be executed by the processor;

[0035] The processor executes the computer readable program to implement the steps of the CT image denoising method based on the sub-region BM3D algorithm and deep learning as any one of the above.

[0036] Advantages: compared with the prior art, the present application provides a CT image denoising method, device, equipment and medium based on sub-region BM3D algorithm and deep learning, the method comprises: acquiring a set of original workpiece images collected by a CT system, and dividing each original workpiece image in the set of original workpiece images into at least two image regions; using a BM3D algorithm to perform image denoising on each image region in the original workpiece image to generate a labeled workpiece image for each original workpiece image; training a preset denoising model based on the original workpiece image and the labeled workpiece image to obtain a CT image denoising model; and using the CT image denoising model to perform image denoising on a workpiece image collected by the CT system. The present application uses deep learning to fit the denoising process of sub-region BM3D, eliminates the shortcomings of the original BM3D algorithm, can not only adaptively denoise different regions, but also improve the overall denoising effect, and has extremely fast denoising speed. BRIEF DESCRIPTION OF DRAWINGS

[0037] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0038] Figure 1 The flowchart of the CT image denoising method based on sub-region BM3D algorithm and deep learning provided by the embodiments of the present application.

[0039] Figure 2 The overall flowchart of the CT image denoising method based on sub-region BM3D algorithm and deep learning provided by the embodiments of the present application.

[0040] Figure 3An example diagram for an overlap region.

[0041] Figure 4 An example diagram for a weight transition change matrix.

[0042] Figure 5 An example diagram for a weight transition.

[0043] Figure 6 An example diagram for a network structure of a CT image denoising model.

[0044] Figure 7 An example diagram for a network structure of a NAF module.

[0045] Figure 8 An example diagram for a principle block diagram of a CT image denoising device based on a sub-region BM3D algorithm and deep learning provided by an embodiment of the present application.

[0046] Figure 9 An example diagram for a principle block diagram of a terminal device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0047] The embodiment of the present application provides a CT image denoising method, device, equipment and medium based on a sub-region BM3D algorithm and deep learning. In order to make the purpose, technical scheme and effect of the present application more clear and definite, the present application is further described in detail below with reference to the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application, and are not used to limit the present application.

[0048] Those skilled in the art of the present technology can understand that, unless specifically stated, the singular form "a", "an" and "the" used herein also includes the plural form. It should be further understood that the phrase "comprising" used in the specification of the present application means that the features, integers, steps, operations, elements and / or components exist, but do not exclude the existence or addition of one or more other features, integers, steps, operations, elements, components and / or their groups. It should be understood that when we say that an element is "connected" or "coupled" to another element, it can be directly connected or coupled to the other element, or there can be intermediate elements. In addition, the "connection" or "coupling" used herein can include wireless connection or wireless coupling. The phrase "and / or" used herein includes all or any single unit and all combinations of the associated listed items.

[0049] Those skilled in the art can understand that, unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. It should also be understood that terms such as those defined in commonly used dictionaries should be interpreted as having a meaning that is consistent with their meaning in the context of the relevant art and will not be interpreted in an idealized or overly formal sense unless expressly so defined herein.

[0050] It should be understood that the sequence numbers and sizes of the steps in the embodiments do not mean the order of execution, and the execution order of the processes is determined by their functions and inherent logic, and should not constitute any limitation on the implementation process of the embodiments of the application.

[0051] The application will be further described by the description of the embodiments in conjunction with the drawings.

[0052] The embodiment provides a CT image denoising method based on a sub-region BM3D algorithm and deep learning, as shown in Figure 1 and Figure 2 The method comprises the following steps.

[0053] S10, acquiring an original workpiece image set collected by a CT system, and dividing each original workpiece image in the original workpiece image set into at least two image regions.

[0054] Specifically, the original workpiece image set comprises a plurality of original workpiece images, and each original workpiece image in the plurality of original workpiece images is a workpiece CT image reconstructed based on projection data collected by the CT system. For example, the original workpiece image can be reconstructed based on multi-projection data, and the multi-projection data is obtained by scanning a scanned workpiece at multiple scanning angles by a CT imaging system.

[0055] It should be noted that the original workpiece image set can include one workpiece type or multiple workpiece types, and each workpiece type corresponds to a plurality of original workpiece images.

[0056] Further, after acquiring the original workpiece image set, each original workpiece image in the original workpiece image set is partitioned, so that each original workpiece image contains at least two image regions. Each image region in the at least two image regions contains a partial image region of the original workpiece image, and adjacent two image regions in the at least two image regions can be provided with an overlapping region. For example, as shown in Figure 3 The original workpiece image contains two image regions, and there is an overlapping region between the two image regions. The region width of the overlapping region can be adjusted according to actual requirements, which is not limited here.

[0057] In one embodiment, the dividing the original workpiece image into at least two image regions specifically comprises:

[0058] obtaining a workpiece type corresponding to the original workpiece image, and obtaining a region template corresponding to the workpiece type, wherein the region template comprises at least two regions;

[0059] applying the region template to the original workpiece image to divide the original workpiece image into at least two image regions.

[0060] Specifically, the workpiece type is the type of the workpiece contained in the original workpiece image, the structures of workpieces of the same workpiece type are basically the same, and the structures of workpieces of different workpiece types will differ. Therefore, when dividing the original workpiece image into regions, the original workpiece image can be divided according to the workpiece type of the workpiece contained in the original workpiece image, so that image points containing different noise intensities can be accurately divided into different image regions, and then different noise reduction intensities are used for denoising different image regions, thereby improving the image denoising effect.

[0061] Based on this, all workpiece types contained in the original workpiece image set can be obtained, and a region template is preset for each workpiece type. Then, the region template of each workpiece type is used to divide each original workpiece image in the original workpiece image set into regions. The construction process of the region template can obtain the structure of the workpiece, and then the structure of the workpiece divides a original workpiece image corresponding to the workpiece type into a plurality of image regions to obtain a partitioned original workpiece image, each image region comprising part of the workpiece structure. Then, a blank image is constructed, and the blank image is partitioned according to the partitioned original workpiece image to obtain a region template corresponding to the workpiece type, the region template comprising at least two regions. Of course, in actual application, other ways can also be used to construct the region template, for example, manually constructing the region template for each workpiece type.

[0062] The embodiment of the application preconstructs a region module, which can then be applied to original workpiece images of the corresponding workpiece type in batches, thereby improving the partitioning speed of the original workpiece image. At the same time, by dividing the original tool image into different image regions, image denoising can be performed on different image regions respectively, so as to achieve fine denoising of different tools and different details, thereby improving the denoising effect.

[0063] S20, using a BM3D algorithm to perform image denoising on each image region in the original workpiece image to generate a labeled workpiece image for each original workpiece image.

[0064] Specifically, the BM3D algorithm is an image and video denoising algorithm, which includes a block matching collaborative filtering (BM) and a basic estimation (3D filtering), the block matching collaborative filtering stage is used to search a three-dimensional group (3D Group) by block matching, and the basic estimation is used to transform the three-dimensional group obtained by the block matching collaborative filtering, and then suppress high-frequency noise in the transform domain.

[0065] After the original workpiece image is divided into a plurality of image regions, the BM3D algorithm is used for image denoising for each image region respectively. That is, each of the plurality of image regions is independently subjected to image denoising by the BM3D algorithm, and the image denoising of one image region does not affect the image denoising of other image regions. For the overlapping region of adjacent image regions, the overlapping region will be subjected to image denoising as the region content of two image regions respectively, and two denoising results are obtained. That is, for the image points contained in the overlapping region, there are multiple denoising results, and the number of the multiple denoising results is the same as the number of the image regions to which the image points belong. For example, the original workpiece image includes an image region A and an image region B, the image region A and the image region B can be extracted from the original workpiece image, the image region A and the image region B are respectively taken as images to be denoised, and then the BM3D algorithm is used to perform image denoising on the image region A and the image region B respectively, to obtain a denoised image region corresponding to the image region A and a denoised image region corresponding to the image region B. For the image points belonging to the overlapping region of the image region A and the image region B, there is one denoising result in the denoised image region corresponding to the image region A, and there is one denoising result in the denoised image region corresponding to the image region B. For the image points in the image region A except the overlapping region, there is one denoising result in the denoised image region corresponding to the image region A, and for the image points in the image region B except the overlapping region, there is one denoising result in the denoised image region corresponding to the image region B.

[0066] Further, since each image region includes different workpiece structures, in order to finely denoise each workpiece structure, a denoising strength can be configured for each image region respectively, to realize adaptive image denoising of each image region, and then the effect of finely denoising different workpiece details can be realized. Based on this, the BM3D algorithm is used to perform image denoising on each image region in the original workpiece, to generate a labeled workpiece image for each original workpiece image, specifically including:

[0067] S21, obtaining a denoising strength of each image region;

[0068] S22, based on the noise reduction intensity of each image region, performing image noise reduction on each image region by using a BM3D algorithm;

[0069] S23, integrating the image regions after noise reduction to generate a labeled workpiece image for each original workpiece image.

[0070] In step S21, the noise reduction intensity is used when performing image noise reduction on each image region by using the BM3D algorithm. The noise reduction intensity of each image region can be different, or the noise reduction intensity of part of the image regions can be different, and the noise reduction intensity of part of the image regions can be the same. That is, when the at least two image regions include two image regions, the noise reduction intensities of the two image regions are different, and when the at least two image regions include three or more image regions, the noise reduction intensities of at least two of the three or more image regions are different.

[0071] In an embodiment, the obtaining of the noise reduction intensity of each image region specifically includes:

[0072] For each of the at least two image regions, performing a convolution operation on the image region by using a Laplacian operator, and calculating an average pixel value of the image region after the convolution operation to obtain an initial noise reduction intensity;

[0073] Obtaining a plurality of adjacent noise reduction intensities of the initial noise reduction intensity, and performing image noise reduction on the image region by using the initial noise reduction intensity and the plurality of adjacent noise reduction intensities respectively;

[0074] According to the noise reduction results corresponding to the initial noise reduction intensity and each adjacent noise reduction intensity, selecting the noise reduction intensity corresponding to the image region from the initial noise reduction intensity and the plurality of adjacent noise reduction intensities.

[0075] Specifically, the convolution operation on the image region by using the Laplacian operator is used to suppress the influence of image structure on the calculation of the noise reduction intensity, and the initial noise reduction intensity is equal to the average pixel value of the image region after the convolution operation.

[0076] The plurality of adjacent noise reduction intensities are determined according to the initial noise reduction intensity, and the intensity difference between each adjacent noise reduction intensity in the plurality of adjacent noise reduction intensities and the initial noise reduction intensity is within a preset range. Wherein, the obtaining process of the plurality of adjacent noise reduction intensities can be that the initial noise reduction intensity is taken as an initial value, the initial noise reduction intensity is moved by a preset step number according to a preset fixed step to obtain a preset step number of adjacent noise reduction intensities, for example, if the preset fixed step is 1 / 100 of the initial noise intensity and the preset step number is 10, then 1 / 100 of the initial noise intensity is taken as a sliding window, and the initial noise reduction intensity is slid by 10 steps to obtain 10 adjacent noise reduction intensities.

[0077] Of course, in practical applications, other ways of obtaining several adjacent denoising strengths can also be used, for example, a neighborhood radius can be preset, and then a neighborhood interval of the initial denoising strength can be determined according to the neighborhood radius and the initial denoising strength, and several neighborhood denoising strengths can be randomly selected or equidistantly selected in the neighborhood interval.

[0078] After obtaining the several neighborhood denoising strengths, the BM3D algorithm is respectively performed on the original workpiece image based on the initial denoising strength and each neighborhood denoising strength, to obtain a denoised image corresponding to the initial denoising strength and each neighborhood denoising strength, and a denoising strength is selected from the initial denoising strength and each neighborhood denoising strength according to the denoising effect of the denoised image, for example, the initial denoising strength or the neighborhood denoising strength corresponding to the denoised image with the best denoising effect is selected as the denoising strength of the image region. The denoising effect can be determined according to a preset denoising index (edge smoothness or artifact removal degree, etc.), or can be manually selected by a user.

[0079] It should be noted that in practical applications, a fixed denoising strength can also be directly set for each image region, and when image denoising is performed on the image region, the fixed denoising strength corresponding to the image region can be directly read, etc. Alternatively, the initial denoising strength described above can be directly used as the denoising strength corresponding to the image region, etc.

[0080] In step S22, after obtaining the denoising strength corresponding to each image region, image denoising is performed on each image region by using the BM3D algorithm based on the denoising strength of each image region, to obtain a denoising result corresponding to each image region. The denoising process of using the BM3D algorithm to perform image denoising on each image region can use the existing process, which will not be described here.

[0081] In step S23, the denoised image regions are integrated, which means that the denoised image regions are combined according to their image positions on the original workpiece image to obtain a denoised workpiece image. Then, the denoised workpiece image is used as a labeled workpiece image of the original workpiece image, and the denoising process of the preset denoising model learning region-based BM3D is supervised through the labeled workpiece image, so that the preset denoising model learns the denoising process of the region-based BM3D, and can perform adaptive image denoising on the workpiece CT image.

[0082] Further, when the denoised image regions are integrated, for the image regions in each image region except the overlapping regions, the denoising result corresponding to the image points in the image region is used as the denoising result of the image points in the labeled workpiece image. For the overlapping regions, since the image points in the overlapping regions correspond to at least two denoising results, the denoising results corresponding to the image points in the overlapping regions are integrated, and the integrated denoising result is used as the denoising result of the image points in the labeled workpiece image.

[0083] For example, any two adjacent image regions in the at least two image regions have an overlap region; and the integrating the denoised image regions to generate a labeled workpiece image for each original workpiece image specifically comprises:

[0084] obtaining a weight transition change matrix corresponding to the overlap region;

[0085] integrating the denoised image regions according to the weight transition change matrix to obtain a denoised CT image.

[0086] Specifically, the weight transition change matrix is used to weight and fuse two denoised structures of the image points in the overlap region, so as to effectively ensure the smooth transition of the boundary, thereby avoiding the boundary effect caused by directly integrating the denoised results of the regions. The weight transition change matrix is determined according to the region linear attenuation direction of the image region. Specifically, the weight in the weight transition change matrix attenuates along the region linear attenuation direction, and the region linear attenuation direction is a direction from the image region to an image region that has the overlap region with the image region. For example, the image region A and the image region B have an overlap region, and the image region B is on the left side of the image region A. Then, the region linear attenuation direction of the image region A is a direction from right to left.

[0087] Further, when obtaining the weight transition change matrix corresponding to the overlap region, the weight transition change matrix corresponding to one image region containing the overlap region can be obtained first, and then the weight transition change matrix corresponding to another image region is calculated according to the weight transition change matrix corresponding to the image region. Specifically, for one image region containing the overlap region, a pre-set weight linear attenuation relationship, an initial weight, and a region width of the overlap region are obtained, and the weight corresponding to each image column in the image region is determined according to the region width, the weight linear attenuation relationship, and the initial weight. Then, each image column corresponding weight is normalized to obtain the weight transition change matrix corresponding to the image region. Then, the weight transition change matrix corresponding to another image region is calculated according to the weight transition change matrix corresponding to the image region, so that the sum of all weights corresponding to each image point in the overlap region is equal to 1. The weight linear attenuation relationship is a function of the region width and the initial weight, and the specific form of the function is not specifically limited here.

[0088] For example, the overlap region is a 5x5 matrix, the region linear attenuation direction is from right to left, and the initial weight is set as the region width. The weight linear attenuation relationship = (initial weight - column number of image column + 1) / region width. For example, Figure 4As shown, the weight of each image point is determined by the linear decay relationship of the weights, and the weight transition matrix decays from right to left. For the weight matrix of the left region, it is only necessary to subtract the weight transition matrix from 1.

[0089] It should be noted that the above method can be used to obtain other overlapping cases, which will not be repeated here. However, for the convenience of illustrating the transition between pixels in the weight transition matrix, a schematic diagram of the weight transition is provided below. Figure 5 The text mainly lists the weighting scenarios for right transitions, left transitions, lower left transitions, and upper left transitions. Similarly, the weighting transition matrices for other scenarios can be derived.

[0090] Furthermore, in practical applications, the image region division can be tailored to the specific circumstances, and the selection criteria for overlapping regions can be pre-defined. For example, the original workpiece image can be divided into five image regions with different noise intensities, named the left image region, right image region, upper image region, lower image region, and middle image region. For the left image region, a right overlapping region is defined; for the right image region, a left overlapping region is defined; for the upper image region, left, right, and lower overlapping regions are defined; for the lower image region, left, right, and upper overlapping regions are defined; and for the middle image region, upper left, upper right, lower left, and lower right overlapping regions are defined. After dividing the image regions and their overlapping regions, the weighted transition matrix can be obtained using the above process. Then, the overlapping regions can be weighted and fused using the weighted transition matrix to obtain a smooth boundary image, effectively solving the boundary effect problem.

[0091] S30. The preset denoising model is trained based on the original workpiece image and the labeled workpiece image to obtain the CT image denoising model.

[0092] Specifically, the preset denoising model is pre-set and used to learn the denoising process of regional BM3D. This ensures that training the preset denoising model enables regional BM3D image denoising of workpiece images acquired through a CT system. The preset denoising model and the CT image denoising model have the same model structure; the difference lies in that the CT image denoising model uses trained model parameters, while the preset denoising model uses initial model parameters. The model structure will be explained here using the CT image denoising model as an example.

[0093] The CT image denoising model is based on the RED-CNN network. A NAF module is added before each convolutional layer and after each deconvolutional layer in the RED-CNN network, and each convolutional layer is used to perform a downsampling operation. Specifically, as...Figure 6 As shown in the figure, the CT image denoising model introduces a NAF module on the basis of the RED-CNN network, the RED-CNN network includes a plurality of convolution layers Conv (such as 5 convolution layers), the step of each convolution layer Conv is 2, in order to perform convolution downsampling (reduce the amount of calculation), the NAF module is arranged before each convolution layer Conv, and each convolution layer is connected with an activation function layer (such as a ReLu activation function). At the same time, in order to restore the image size, the CT image denoising model contains a plurality of deconvolution layers Deconv (the number of which is the same as the number of convolution layers, such as 5 convolution layers and 5 deconvolution layers), and the NAF module is added after each deconvolution layer Deconv, and the NAF module is connected with an activation function layer (such as a ReLu activation function). Among them, the last convolution layer Conv and the first deconvolution layer Deconv, the first deconvolution layer Deconv is connected with the NAF module through an adder, and the adder is connected with the second last convolution layer Conv; the third deconvolution layer Deconv is connected with the NAF module through an adder, and the adder is connected with the fourth last convolution layer Conv, and so on. The last deconvolution layer Deconv is connected with the NAF module through an adder, and the adder is connected with the input item Input (C, H, W) of the CT image denoising model, and the NAF module and the activation function layer are used to obtain the output item Output (C, H, W), and the adder is connected with the input item Input (C, H, W) of the CT image denoising model.

[0094] Further, as shown in the figure, Figure 7 The NAF module contains SCA (Simplified Channel Attention) and SG (Simple Gate) structures. SCA is a simplified version of channel attention mechanism. First, the feature map is compressed through the average pooling layer AvgPooling, then the channel weight information is learned through the 1x1 convolution, and finally multiplied with the original input. It has the functions of aggregating global information and utilizing different channel information, and does not bring precision loss in simplification. The SG structure realizes it by splitting the channel into two parts and multiplying them, which is a substitute for the GELU activation function. Compared with the complex implementation of GELU, its implementation is very simple, and at the same time it also brings the improvement of the precision of the model. The forward process of the whole NAF can be described as follows: the input feature layer first passes through the layer normalization (stabilize the training process and accelerate the convergence), then passes through the convolution layer and the SG structure, then aggregates the information of different channels through the SCA structure, and then passes through the convolution layer + Dropout (avoid overfitting) to output the preliminary result. Subsequently, a residual structure is used to integrate key information (β is updated and learned with training), and the same as above, the features are further extracted through the above structure LayerNorm, Conv, SG, Dropout, and residual structure (γ update) to obtain the final output.

[0095] Further, the output item Output(C, H, W) of the preset denoising model is used to construct a loss function with the label workpiece image Label. The loss function includes a prediction loss item, a perception loss item, an edge weight loss item, and an image visual loss item, which are obtained by weighting the prediction loss item, the perception loss item, the edge weight loss item, and the image visual loss item. Correspondingly, the loss function can be represented as:

[0096] L total =w1L1+w2L P +w3L Edge +w4L TV

[0097] wherein L total represents the loss function, L1 represents the prediction loss item, L P represents the perception loss item, L Edge represents the edge weight loss item, L TV represents the image visual loss item, and w1, w2, w3, and w4 all represent weighting coefficients.

[0098] The prediction loss item L1 is used to avoid the case that the output image is excessively smoothed, wherein the expression of the prediction loss item L1 is as follows:

[0099]

[0100] wherein y represents the label workpiece image, represents the output item of the preset denoising model, n represents the pixel number of the original workpiece image, and i represents the i-th pixel of the original workpiece image.

[0101] The expression of the perception loss item L P is as follows:

[0102]

[0103] wherein Φ represents a pre-trained network used for extracting deep features, l represents a feature layer position used, and C, H, and W respectively represent the channel, height, and width of the current feature layer.

[0104] It can be seen that the perception loss term is to extract the high-level semantic features of the label workpiece image and the output term of the preset denoising model through the pre-training network, and the difference between the two high-level semantic features is calculated as a loss term, so that the denoising result is closer to the perception of the human eye. In actual use of the pre-training network, since the existing pre-training network is generally for three-channel image data, and the CT image is single-channel, a pre-training network for perception loss needs to be retrained. The strategy adopted here is to use grayscale image data to train VGG16. Here, the cifar10 dataset is used, which has 60,000 color images. The color images can be converted to grayscale images by the grayscale formula. Then, the input channel of the VGG16 network is set to 1, and the L1 loss is used for iterative training, so that a pre-training network for single-channel images can be obtained. In calculating the perception loss term, the pre-training network is used for high-level semantic feature extraction, and the preset feature layer position is selected, for example, assuming that the preset feature layer position is the 13th feature layer, i.e. l = 13, then the output of Φ is the feature map extracted by the 13th feature layer.

[0105] Edge weight loss term L Edge The expression is:

[0106]

[0107] Where f represents a filter function (such as Gaussian filtering). The label workpiece image is filtered before edge extraction to reduce noise interference to some extent. G h represents edge extraction in the horizontal direction using the sobel operator, G v represents edge extraction in the vertical direction using the sobel operator. The edge weight loss term performs filtering operation on the label workpiece image, then integrates the edges in the two directions to obtain the edge at the corresponding position, and performs maximum value normalization to obtain the final edge weight coefficient. This is used as the product term of the output term of the preset denoising model and the absolute error of the label workpiece image to constrain the edge.

[0108] Image visual loss term L TV The expression is:

[0109]

[0110] Where, represents the output term of the preset denoising model, m represents the mth image row, n represents the nth image column, and β is a constant. The image visual loss term uses the difference between each pixel and the adjacent next pixel in the horizontal and vertical directions to limit the fluctuation between pixels, which can avoid excessive noise in the output image and ensure the smoothness between pixels.

[0111] It should be noted that the weighting coefficients of the prediction loss term, the perception loss term, the edge weight loss term and the image visual loss term can be set according to actual conditions. In the embodiments of the present application, the prediction loss term is the main loss, so the maximum weighting coefficient is set for the prediction loss term, and the perception loss term, the edge weight loss term and the image visual loss term are set with small weighting coefficients relative to the prediction loss term. Through the perception loss term, the edge weight loss term and the image visual loss term, the generation of the image is adjusted in real time, and more details are retained.

[0112] The embodiments of the present application use the label workpiece image obtained by the partitioned BM3D denoising as a label, train a preset denoising model using the original workpiece image and the label workpiece image, and use a loss function including a prediction loss term, a perception loss term, an edge weight loss term and an image visual loss term to optimize the model parameters of the preset denoising model, so that the preset denoising model can learn the forward process of the partitioned BM3D denoising. The CT image denoising model obtained by training not only has the ability to adapt to different intensity noises, but also has an extremely fast denoising processing speed.

[0113] S40, using the CT image denoising model to perform image denoising on the workpiece image collected by the CT system.

[0114] Specifically, the CT image denoising model is a neural network model that learns the forward process of the partitioned BM3D denoising. Through the forward process of the partitioned BM3D denoising, the workpiece image collected by the CT system can be partitioned and BM3D denoised to obtain a workpiece image after image denoising.

[0115] In summary, the embodiment provides a CT image denoising method based on a sub-region BM3D algorithm and deep learning, which comprises: acquiring an original workpiece image set collected by a CT system, and dividing each original workpiece image in the original workpiece image set into at least two image regions; performing image denoising on each image region in the original workpiece by using a BM3D algorithm to generate a labeled workpiece image for each original workpiece image; training a preset denoising model based on the original workpiece image and the labeled workpiece image to obtain a CT image denoising model; and performing image denoising on a workpiece image collected by the CT system by using the CT image denoising model. The embodiment divides the original workpiece image into a plurality of image regions, and performs BM3D denoising with different denoising intensities on each image region to obtain a labeled workpiece image, thereby improving the denoising effect of each image region of the original workpiece image, avoiding problems such as loss of key details and insufficient denoising intensity, and improving the denoising effect of the original workpiece image. Meanwhile, the embodiment introduces a weight transition change matrix when performing sub-region BM3D denoising on the original workpiece image, and performs weighted integration on the overlapping regions of the image regions by using the weight transition change matrix, thereby avoiding the boundary problem introduced by sub-region denoising.

[0116] After the labeled workpiece image obtained by sub-region BM3D denoising is acquired, the embodiment constructs a training image data set by using the original workpiece image and the labeled workpiece image, trains the preset denoising model by using the training image data set, makes the preset denoising model learn the forward process of sub-region BM3D denoising, that is, fits the forward process of sub-region BM3D denoising into the preset denoising model, and obtains a trained CT image denoising model. In this way, the cumbersome BM3D algorithm can be directly bypassed, the forward process of sub-region BM3D denoising can be quickly inferred by using deep learning, the problem of slow image denoising speed caused by complex operation of the BM3D algorithm can be avoided, and the image denoising efficiency is improved. Meanwhile, the NAF module is introduced into the CT image denoising model, the ability of the CT image denoising model to aggregate global information and channel information is improved. In the training process, the loss function including the prediction loss term, the perception loss term, the edge weight loss term and the image visual loss term can strengthen the learning of the network on the edge details, and the perception loss of single-channel pre-training can improve the fidelity of detail generation.

[0117] Based on the CT image denoising method based on the sub-region BM3D algorithm and deep learning, the embodiment provides a CT image denoising device based on the sub-region BM3D algorithm and deep learning, as shown in Figure 8 The CT image denoising device based on the sub-region BM3D algorithm and deep learning specifically comprises:

[0118] The acquisition module 100 is configured to acquire a set of original workpiece images collected by a CT system, and divide each original workpiece image in the set of original workpiece images into at least two image regions.

[0119] The generation module 200 is configured to perform image denoising on each image region in the original workpiece image by using a BM3D algorithm, to generate a labeled workpiece image for each original workpiece image.

[0120] The training module 300 is configured to train a preset denoising model based on the original workpiece images and the labeled workpiece images, to obtain a CT image denoising model.

[0121] The denoising module 400 is configured to perform image denoising on a workpiece image collected by the CT system by using the CT image denoising model.

[0122] Based on the CT image denoising method based on the region-based BM3D algorithm and deep learning, the embodiment provides a computer-readable storage medium storing one or more programs, which can be executed by one or more processors to implement the steps in the CT image denoising method based on the region-based BM3D algorithm and deep learning.

[0123] Based on the CT image denoising method based on the region-based BM3D algorithm and deep learning, the embodiment provides a terminal device, as shown in the accompanying drawings, which includes at least one processor 20, a display screen 21, and a memory 22, and can further include a communications interface 23 and a bus 24. The processor 20, the display screen 21, the memory 22, and the communications interface 23 can communicate with each other through the bus 24. The display screen 21 is configured to display a user guide interface preset in an initial setting mode. The communications interface 23 can transmit information. The processor 20 can call logical instructions in the memory 22 to execute the method in the above embodiment. Figure 9

[0124] In addition, the logical instructions in the memory 22 described above can be implemented in the form of a software functional unit and sold or used as an independent product, which can be stored in a computer-readable storage medium.

[0125] The memory 22 as a computer-readable storage medium can be configured to store software programs, computer executable programs, program instructions or modules corresponding to the method in the embodiment of the disclosure. The processor 20 executes the functions of the application and data processing by running the software programs, instructions or modules stored in the memory 22, that is, implements the method in the above embodiment. ​

[0126] The memory 22 can include a program storage area and a data storage area, wherein the program storage area can store an operating system, application programs required by at least one function; the data storage area can store data created according to the use of the terminal device, etc. In addition, the memory 22 can include a high-speed random access memory, and can also include a non-volatile memory. For example, various media that can store program codes, such as a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc., can also be a temporary storage medium.

[0127] In addition, the specific processes of the above-mentioned storage medium and the plurality of instruction processors in the terminal device load and execute have been described in detail in the above method, and will not be described one by one here.

[0128] Finally, it should be pointed out that: the above examples are only used to illustrate the technical solutions of the present application, and not to limit them; although the present application has been described in detail with reference to the foregoing examples, those skilled in the art should understand that: it can still modify the technical solutions recorded in the foregoing examples, or make equivalent replacement for part of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A CT image denoising method based on the region-based BM3D algorithm and deep learning, characterized in that, The CT image denoising method based on the region-based BM3D algorithm and deep learning specifically includes: Acquire a set of original workpiece images acquired by a CT system, and divide each original workpiece image in the set into at least two image regions; The BM3D algorithm is used to denoise each image region in the original workpiece image to generate a labeled workpiece image for each original workpiece image. The preset denoising model is trained based on the original workpiece image and the labeled workpiece image to obtain the CT image denoising model; The CT image noise reduction module is used to reduce the noise of workpiece images acquired by the CT system.

2. The CT image denoising method based on the regional BM3D algorithm and deep learning according to claim 1, characterized in that, The step of dividing the original workpiece image into at least two image regions specifically includes: Obtain the workpiece type corresponding to the original workpiece image, and obtain the region template corresponding to the workpiece type, wherein the region template includes at least two regions; The region template is applied to the original workpiece image to divide the original workpiece image into at least two image regions.

3. The CT image denoising method based on the regional BM3D algorithm and deep learning according to claim 1, characterized in that, The step of using the BM3D algorithm to perform image noise reduction on each image region in the original workpiece image to generate a labeled workpiece image for each original workpiece image specifically includes: Obtain the noise reduction intensity for each image region; Based on the noise reduction intensity of each image region, the BM3D algorithm is used to perform image noise reduction for each image region; The denoised image regions are integrated to generate labeled workpiece images for each original workpiece image.

4. The CT image denoising method based on the regional BM3D algorithm and deep learning according to claim 3, characterized in that, The specific steps for obtaining the noise reduction intensity of each image region include: For each of at least two image regions, the image region is convolved using the Laplacian operator, and the average pixel value of the convolved image region is calculated to obtain the initial noise reduction intensity. Obtain several neighboring noise reduction intensities of the initial noise reduction intensity, and use the initial noise reduction intensity and several neighboring noise reduction intensities to perform image noise reduction on the image region respectively; Based on the initial noise reduction intensity and the noise reduction results corresponding to each neighboring noise reduction intensity, the noise reduction intensity corresponding to the image region is selected from the initial noise reduction intensity and several neighboring noise reduction intensities.

5. The CT image denoising method based on the regional BM3D algorithm and deep learning according to claim 2, characterized in that, The at least two image regions have overlapping areas between any two adjacent image regions; the step of integrating the denoised image regions to generate a labeled workpiece image for each original workpiece image specifically includes: Obtain the weight transition matrix corresponding to the overlapping region; The image regions after denoising are integrated according to the weighted transition matrix to obtain a denoised CT image.

6. The CT image denoising method based on the regional BM3D algorithm and deep learning according to claim 1, characterized in that, The CT image denoising model is based on the RED-CNN network. A NAF module is added before each convolutional layer and after each deconvolutional layer in the RED-CNN network, and each convolutional layer is used to perform a downsampling operation.

7. The CT image denoising method based on the regional BM3D algorithm and deep learning according to claim 1, characterized in that, The loss function used in the training process of the CT image denoising model includes a prediction loss term, a perception loss term, an edge weight loss term, and an image visual loss term.

8. A CT image noise reduction device based on the regional BM3D algorithm and deep learning, characterized in that, The CT image noise reduction device based on the region-based BM3D algorithm and deep learning specifically includes: The acquisition module is used to acquire the original workpiece image set acquired by the CT system and divide each original workpiece image in the original workpiece image set into at least two image regions. The generation module is used to perform image noise reduction on each image region in the original workpiece image using the BM3D algorithm, so as to generate a labeled workpiece image for each original workpiece image. The training module is used to train a preset denoising model based on the original workpiece image and the labeled workpiece image to obtain a CT image denoising model. The noise reduction module is used to perform image noise reduction on the workpiece image acquired by the CT system using the CT image noise reduction module.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores one or more programs, which can be executed by one or more processors to implement the steps in the CT image denoising method based on the regional BM3D algorithm and deep learning as described in any one of claims 1-7.

10. A terminal device, characterized in that, include: Processor and memory; The memory stores a computer-readable program that can be executed by the processor; When the processor executes the computer-readable program, it implements the steps in the CT image denoising method based on the regional BM3D algorithm and deep learning as described in any one of claims 1-7.

Images