Industrial ct image reconstruction method and device, electronic equipment and storage device

Abstract

The application discloses an industrial CT image reconstruction method and device, electronic equipment and storage equipment, comprising obtaining projection image data and preprocessing to obtain target image data; based on the pre-trained U-Net variant lightweight network, the target image data is processed to obtain fine gradient direction prediction data and structure level similarity data, the U-Net variant lightweight network is trained based on the projection image data and the true value image data, and the loss function of the U-Net variant lightweight network comprises a structure similarity loss; based on the target image data, the fine gradient direction prediction data and the structure level similarity data, the target function is iteratively updated by using an alternating direction multiplier method to obtain reconstructed image data, and the target function is a high-order regularization function constructed based on the second-order total generalized variation and the structure level non-local mean of the iterative image data. The application can inhibit the streak artifact and photon noise of sparse images, so as to meet the requirements of fast scanning and sparse image reconstruction quality.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of automatic detection technology, and in particular to an industrial CT image reconstruction method and device, an electronic device and a storage device. BACKGROUND

[0002] In industrial non-destructive testing, fast CT (Computed Tomography) and high-quality image reconstruction are the key to improving detection efficiency. In related technologies, fast scanning can be achieved by reducing the projection angle (sparse image) and shortening the X-ray exposure time, but this will introduce serious streak (striped) artifacts and photon noise, reducing the quality of CT image reconstruction. SUMMARY

[0003] The present application aims to at least solve one of the technical problems existing in the prior art. To this end, the present application provides an industrial CT image reconstruction method and device, an electronic device and a storage device, which can suppress the streak artifacts and photon noise of sparse images, thereby meeting the requirements of fast scanning and sparse image reconstruction quality.

[0004] In a first aspect, an embodiment of the present application provides an industrial CT image reconstruction method, comprising:

[0005] Obtaining projection image data, which is image data obtained by an industrial CT scanning device in a fast scanning mode;

[0006] Pretreating the projection image data to obtain target image data;

[0007] Processing the target image data based on a pre-trained U-Net variant lightweight network to obtain fine gradient direction prediction data and structure-level similarity data, the U-Net variant lightweight network being trained based on projection image data and ground truth image data, and the loss function of the U-Net variant lightweight network including a structure similarity loss;

[0008] Based on the target image data, the fine gradient direction prediction data and the structure-level similarity data, iteratively updating a target function using an alternating direction multiplier method to obtain reconstructed image data, the target function being a high-order regularization function constructed based on a second-order total generalized variation and a structure-level non-local mean of iterative image data.

[0009] According to some embodiments of the present application, the target function is a high-order regularization function constructed based on a second-order total generalized variation and a structure-level non-local mean of iterative image data, comprising:

[0010] The target function is: u = argmin(||u||TGV + λ · u * NLM ), s.t. ||f - W · u||2≤ξ, wherein u is iterative image data, ||u||2 TGV is a second-order total generalized variation of the iterative image data, u * NLM is a structure-level non-local mean of the iterative image data, f is projection image data, W is a system matrix, λ is a regularization coefficient, and ξ is a preset tolerance.

[0011] According to some embodiments of the present application, the second-order total generalized variation of the iterative image data is: ||u|| TGV = ||D1u||1+α||D2u||1, wherein D1u is a first-order derivative of the iterative image data, D2u is a second-order derivative of the iterative image data, and α is a weight coefficient.

[0012] According to some embodiments of the present application, the iterative updating of the target function based on the target image data, the fine gradient direction prediction data and the structure-level similarity data by using an alternating direction multiplier method to obtain reconstructed image data comprises:

[0013] taking the target image data as an initial value of iterative image data;

[0014] updating a second-order total generalized variation of the target function based on the iterative image data and the fine gradient direction prediction data to obtain first update data;

[0015] updating a structure-level non-local mean of the target function based on the iterative image data and the structure-level similarity data to obtain second update data;

[0016] determining reconstructed image data according to the first update data and the second update data;

[0017] performing data consistency constraint processing according to the reconstructed image data;

[0018] determining an image difference between reconstructed image data of adjacent two iterations, and taking the reconstructed image data as new iterative image data to repeat the above iteration steps until the image difference is less than a preset tolerance or the number of iterations reaches a preset number.

[0019] According to some embodiments of the present application, the updating of the second-order total generalized variation of the target function based on the iterative image data and the fine gradient direction prediction data to obtain first update data comprises:

[0020] generating a direction mask according to the fine gradient direction prediction data;

[0021] determining a first derivative and a second derivative of the iterative image data;

[0022] modifying the first derivative and the second derivative of the iterative image data according to the direction mask respectively, to obtain a directional first derivative and a directional second derivative;

[0023] determining a gradient of a second total generalized variation according to the directional first derivative and the directional second derivative;

[0024] updating the iterative image data along a direction opposite to the gradient of the second total generalized variation to obtain first update data.

[0025] According to some embodiments of the present application, the second update data is obtained by updating the structure-level non-local mean of the target function based on the iterative image data and the structure-level similarity data, comprising:

[0026] determining a target pixel of the iterative image data;

[0027] determining a search window of the target pixel, and determining pixels in the search window as candidate pixels;

[0028] determining a weight value of each of the candidate pixels according to the structure-level similarity data;

[0029] performing weighted summation according to the weight value and the gray value of each of the candidate pixels to obtain the second update data.

[0030] According to some embodiments of the present application, the data consistency constraint processing according to the reconstructed image data comprises:

[0031] generating theoretical simulation projection image data based on the reconstructed image data;

[0032] determining image residual data according to the target image data and the theoretical simulation projection image data;

[0033] adjusting the iterative image data according to the image residual data.

[0034] In a second aspect, embodiments of the present application provide an industrial CT image reconstruction device, comprising:

[0035] a data acquisition module configured to acquire projection image data, the projection image data being image data obtained by an industrial CT scanning device in a fast scanning mode;

[0036] a preprocessing module configured to preprocess the projection image data to obtain target image data;

[0037] The first processing module is configured to process the target image data based on a pre-trained U-Net variant lightweight network to obtain fine gradient direction prediction data and structure level similarity data, wherein the U-Net variant lightweight network is trained based on projection image data and ground truth image data, and a loss function of the U-Net variant lightweight network comprises a structure similarity loss.

[0038] The second processing module is configured to iteratively update a target function based on the target image data, the fine gradient direction prediction data and the structure level similarity data by using an alternating direction multiplier method to obtain reconstructed image data, wherein the target function is a high-order regularization function constructed based on a second-order total generalized variation and a structure level non-local mean of iterative image data.

[0039] In a third aspect, an embodiment of the present application provides an electronic device, comprising a processor and a memory, wherein the memory stores a computer program, and the processor is configured to implement the industrial CT image reconstruction method described above when running the computer program.

[0040] In a fourth aspect, an embodiment of the present application provides a storage device, wherein the storage device stores a computer program, and the computer program is configured to implement the industrial CT image reconstruction method described above when running.

[0041] The embodiments of the present application have at least the following beneficial effects:

[0042] The target image data is processed based on the pre-trained U-Net variant lightweight network to obtain the fine gradient direction prediction data and the structure level similarity data, the fine gradient direction prediction data can assist the second-order total generalized variation of the high-order regularization to capture the complex edge of the target image data, the structure level similarity data can assist the structure level non-local mean of the high-order regularization function to realize the matching of the local structure, the target function is the high-order regularization function constructed based on the second-order total generalized variation and the structure level non-local mean of the iterative image data, the second-order total generalized variation controls the gradient and curvature of the image to suppress the noise and artifacts while maintaining the edge and details of the image, and the structure level non-local mean utilizes the information of the similar areas in the image to reduce the noise and suppress the blocking effect in the image reconstruction process, so that when the image data obtained in the fast scanning mode is reconstructed, the streak artifacts and photon noise of the sparse image can be suppressed, and the requirements of the fast scanning and the sparse image reconstruction quality are taken into account.

[0043] Additional aspects and advantages of the present application will be given in part in the following description, become apparent from the following description, or be understood through practice of the present application. BRIEF DESCRIPTION OF DRAWINGS

[0044] The above and / or additional aspects and advantages of the present application will become apparent and more readily appreciated from the following description, taken in conjunction with the following drawings in which:

[0045] Figure 1 A flow chart of steps of an industrial CT image reconstruction method according to an embodiment of the present application;

[0046] Figure 2 A principle block diagram of an industrial CT image reconstruction apparatus according to an embodiment of the present application;

[0047] Figure 3 A principle block diagram of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0048] Embodiments of the present application are described in detail below with reference to the attached drawings, wherein the same or similar components have the same or similar designations throughout the several views. The embodiments described below are examples of the present application, and are not intended to limit the present application.

[0049] In the description of the present application, the meaning of "several" is one or more, the meaning of "multiple" is two or more, greater than, less than, more than, etc. are understood as not including the number, "above", "below", "within", etc. are understood as including the number. If there is a description of "first", "second", etc. is only used to distinguish technical features for the purpose, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of indicated technical features or implicitly indicating the order of the indicated technical features.

[0050] Please refer to Figure 1 The embodiment discloses an industrial CT image reconstruction method, comprising steps S100-S400. It should be noted that the step numbers in the embodiment are only for the convenience of understanding, and do not limit the execution order of the steps. The contents of each step are described in detail as follows:

[0051] S100, acquiring projection image data, the projection image data is obtained by using an industrial CT scanning device in a fast scanning mode;

[0052] The projection image data is image data obtained by scanning a target object by a CT scanning device. The scanning device in the embodiment is an industrial CT scanning device, and the scanning object is an industrial product. The purpose of the scanning is to perform nondestructive testing of defects in the industrial product. In the scanning process, the more the scanning angles, the more the projection angles of the image data, and the more conducive to three-dimensional image reconstruction. However, the more the scanning angles, the longer the scanning time, which is not conducive to improving the efficiency of industrial production. Therefore, the related art realizes fast scanning by reducing the projection angle and shortening the X-ray exposure time to obtain sparse views. Although sparse views can shorten the scanning time, they can introduce serious streak artifacts and photon noise. In extreme cases, for example, when the projection angle is less than 50 and the exposure time is less than 20 milliseconds, there are structure distortions and blocking effects. In view of the defects of sparse views, the embodiment obtains projection image data so as to facilitate subsequent processing of the projection image data to suppress artifacts and noise. The projection image data in the embodiment is image data obtained by the industrial CT scanning device in a fast scanning mode, and the projection image data contains artifacts and noise.

[0053] S200, pre-processing the projection image data to obtain target image data;

[0054] The projection image data is single-channel grayscale image data, and the number of pixels of the projection image data is determined according to the detector parameters of the industrial CT scanning device, for example, the number of pixels is 3040*2400. Generally, the data storage format of the projection image data is a raw format. In the pre-processing process, the projection image data is subjected to penalized weighted least-squares (PWLS) denoising processing to make the image smoother and improve the signal-to-noise ratio. In order to be compatible with more image data formats, if the format of the projection image data is not a raw format, the format of the projection image data is converted to a raw format in the pre-processing process.

[0055] S300, processing the target image data based on a pre-trained U-Net variant lightweight network to obtain fine gradient direction prediction data and structure-level similarity data, the U-Net variant lightweight network being trained based on the projection image data and ground truth image data, and the loss function of the U-Net variant lightweight network including a structure similarity loss;

[0056] For example, the U-Net variant lightweight network is based on the symmetrical structure of the classic encoder and decoder of U-Net, and is optimized for the rapid reconstruction requirements of industrial CT. The optimization directions include simplifying the number of convolution layers and using lightweight convolution layers in the convolution units of the down-sampling stage and the up-sampling stage, such as using depth separable convolution or reducing the number of convolution kernels to reduce the amount of calculation. In this way, the U-Net variant lightweight network improves the calculation speed while ensuring the feature extraction capability by simplifying the number of convolution layers, using lightweight convolution operations, and reducing the number of channel expansions, which is conducive to adapting to the rapid reconstruction requirements of industrial CT. It should be noted that the use and performance of the neural network are related to the network architecture and the training data. The training process of the U-Net variant lightweight network in the embodiment is based on end-to-end learning of projection image data and ground truth image data, and uses an industrial CT dataset containing different material and defect samples, and the loss function includes a structural similarity loss. The core of the structural similarity loss is to measure the similarity of the local structure of the image (including brightness, contrast, and structural consistency), rather than simply the pixel gray difference. Therefore, the U-Net variant lightweight network will learn to capture local structural features in the image, such as defect edges, material interface shapes, and gradient information, during training. The output structural similarity data is used to quantify the degree of similarity of the local structure at different positions in the target image data, and the numerical value of the structural similarity data directly reflects the matching degree of the two regions at the structural level. The dimension of the fine gradient direction prediction data is the same as that of the target image data, and each pixel value of the fine gradient direction prediction data reflects the edge gradient direction at the corresponding position. In industrial CT images, the edge of an object is an important feature, and accurate capture of complex edges is crucial for image reconstruction quality. The fine gradient direction prediction data can provide edge direction information for the total generalized variation of the second order (referred to as TGV term) in the following, so that the total generalized variation of the second order can more accurately constrain the edge region. For example, when reconstructing a CT image of an industrial part containing multiple materials and complex shapes, the fine gradient direction prediction data can accurately indicate the edge direction between different materials, so that the total generalized variation of the second order can optimize the edge according to the edge direction, avoiding edge blurring or distortion.

[0057] S400, based on the target image data, the fine gradient direction prediction data, and the structural similarity data, the target function is iteratively updated using the alternating direction multiplier method to obtain the reconstructed image data, and the target function is a high-order regularization function constructed based on the total generalized variation of the second order and the structural non-local mean of the iterative image data.

[0058] Image denoising is an essential problem in image processing, aiming to recover a clear and true image from a noisy image. The presence of noise can seriously affect the quality of the image, reducing the performance of subsequent image analysis, recognition and application. Among the many denoising methods, the model based on variational method is concerned due to its good denoising performance and interpretability. Total Variation (TV) regularization is one of the most widely used regularization terms in variational methods, which constrains the smoothness of the solution by penalizing the total sum of image gradients. However, the total variation regularization has the defect of staircase effect, which leads to the unnatural block structure of the recovered image. In order to overcome this defect, the total generalized variation (TGV) regularization is proposed as a high-order extension of the total variation regularization, which can effectively suppress the staircase effect, preserve the detail information of the image, and obtain a smoother denoising result.

[0059] The non-local means (NL-means, referred to as NLM) filtering algorithm is a denoising technology. The core of the non-local means algorithm is to calculate the gray similarity of the pixel neighborhood at different positions in the image, and to realize denoising by weighted average of the target pixel. The similarity measurement of non-local means mainly relies on the direct comparison of pixel-level gray values, such as calculating the Euclidean distance of pixel gray in the neighborhood window, only focusing on the consistency of local gray distribution, and ignoring the local structural features (such as edge direction, texture pattern, defect morphology, etc.) of the pixels. In industrial CT images, due to the existence of streak artifacts, noise and material differences, simple pixel-level gray matching may misjudge the regions with similar structures but slightly different gray levels as dissimilar, or misjudge the regions with similar gray levels but different structures caused by noise as similar, resulting in problems such as blurring, blocking effect or loss of details in the denoised image. The non-local means of the embodiment is a structure-level non-local means, which assigns weights according to the structure-level similarity data (output by the U-Net variant lightweight network) of the local regions where the target pixel and the candidate pixel are located. That is, the more similar the structure (such as consistent edge direction or same texture pattern, etc.), the greater the weight, and the greater the structural difference, the smaller the weight. Compared with the traditional non-local means which relies on gray difference, the similarity judgment standard of the structure-level non-local means of the embodiment is upgraded from "whether the pixel gray is close" to "whether the local structure matches".

[0060] The high-order regularization function constructed based on the second-order total generalized variation and the structure-level non-local means is an optimization problem with constraints. The Alternating Direction Method of Multipliers (ADMM) can be used to decompose the complex optimization problem into multiple sub-problems and update the variables alternately to gradually approach the optimal solution.

[0061] In the process of updating the target function by using the alternating direction multiplier method, the target image data is taken as the initial value of the iterative image data, and the gradient direction information of the fine gradient direction prediction data provides an explicit optimization direction for each iteration to update the TGV term based on the fine gradient direction prediction data. When updating the TGV term, the regions in the target image data that may have streak artifacts and blocking effects are adjusted according to the gradient direction of the fine gradient direction prediction data. If the edge gradient direction of the fine gradient direction prediction data is complex in a certain region, the TGV term of the region is strengthened to suppress the artifacts and blocking effects, so that the edges of the intermediate image are clearer and closer to the true structure. Through continuous iteration, the TGV term is gradually optimized under the guidance of the fine gradient direction prediction data, so that the quality of the reconstructed image is continuously improved. In the structure-level non-local mean filtering, the structure-level similarity data output by the U-Net variant lightweight network is used to realize dynamic updating based on the current iterative image data, so that the continuity of the edge is preserved during noise reduction.

[0062] Thus, the target image data is processed based on the pre-trained U-Net variant lightweight network to obtain fine gradient direction prediction data and structure-level similarity data. The fine gradient direction prediction data can assist the second-order total generalized variation of the high-order regularization to capture the complex edge of the target image data. The structure-level similarity data can assist the structure-level non-local mean of the high-order regularization function to realize the matching of the local structure. The target function is a high-order regularization function constructed based on the second-order total generalized variation and the structure-level non-local mean of the iterative image data. The second-order total generalized variation suppresses noise and artifacts while preserving the edges and details of the image by controlling the gradient and curvature of the image. The structure-level non-local mean reduces noise and suppresses blocking effects in the image reconstruction process by using the information of similar regions in the image. Thus, when reconstructing the image data obtained in the fast scanning mode based on deep learning and high-order regularization, the streak artifacts and photon noise of the sparse image can be suppressed, so as to meet the requirements of fast scanning and image reconstruction quality.

[0063] In step S400, the target function is a high-order regularization function constructed based on the second-order total generalized variation and the structure-level non-local mean of the iterative image data, including:

[0064] The target function is: u = argmin(||u||2+λ·||u||2), s.t.||f-W·u||2≤ξ, where u is the iterative image data, ||u||2 is the second-order total generalized variation of the iterative image data, u is the target function, f is the target image data, W is a weight matrix, λ is a regularization parameter, and ξ is a noise threshold. TGV +λ∙u * NLM ), s.t.||f-W·u||2≤ξ, where u is the iterative image data, ||u||2 is the second-order total generalized variation of the iterative image data, u is the target function, f is the target image data, W is a weight matrix, λ is a regularization parameter, and ξ is a noise threshold. TGV * NLM ​For the structure-level non-local mean of the iterative image data, f is the projection image data, W is the system matrix, λ is the regularization coefficient, and ξ is the preset tolerance. It should be known that argmin is used to represent the value of the independent variable that makes the function minimum. S.t. is the abbreviation of such that, which is used to represent that a certain variable or condition must meet a certain requirement or constraint, and ||·||2 is used to represent the L2 norm. The embodiment fuses the second-order total generalized variation and the structure-level non-local mean of the iterative image data through the objective function, and constrains the data consistency through the constraint condition, so as to ensure that the image reconstruction result matches the target image data.

[0065] wherein the second-order total generalized variation of the iterative image data is: ||u|| TGV = ||D1u||1+α||D2u||1, D1u is the first-order derivative of the iterative image data, D2u is the second-order derivative of the iterative image data, and α is a weight coefficient. It should be known that ||·||1 is used to represent the L1 norm.

[0066] In order to facilitate understanding of the application concept of the second-order total generalized variation in the embodiment, the following describes each part of the second-order total generalized variation:

[0067] The first-order derivative is used to describe the local change rate of the pixel gray scale in the iterative image data, and its physical meaning is to capture the edge information of the image, such as including the object contour, defect boundary, etc. In the regularization constraint, the L1 norm of the first-order derivative is constrained, which is realized through the "sparsity assumption" to achieve the following effects: retaining key edges and suppressing high-frequency noise, wherein the sparsity assumption is an assumption that the edges of a natural image are sparsely distributed in the gradient domain. Retaining key edges means to strengthen the edge structure that really exists in the image, so as to avoid edge blurring caused by noise or artifacts; suppressing high-frequency noise means to punish the region with rapid fluctuation of gray scale value (such as noise), so as to reduce its interference with the overall structure of the image; but there is a limitation in relying only on the first-order derivative constraint: over-emphasizing the sparsity of the first-order gradient may cause the local region of the image to appear step-shaped distortion (a manifestation of the blocking effect), especially in the region with flat gray scale but complex structure (such as the internal fine texture of the material), which is easy to lose details due to the over-hard gradient constraint, therefore, the second-order derivative is introduced.

[0068] The second derivative describes the rate of change of the first derivative, reflects the "curvature" or "smoothness" of the image gray scale change, and is used to depict the structure smoothness and reduce the staircase effect. The constraint on the L1 norm of the second derivative is the core to control the continuity of the gradient change and achieve the following functions: depicting the smooth transition of the structure and reducing the staircase distortion. The depiction of the smooth transition of the structure refers to that in the edge area, the second derivative can distinguish "real edge" (gradient mutation but continuous) and "pseudo edge" (gradient mutation and discontinuous, such as noise or block effect), and the latter is punished; the reduction of the staircase distortion refers to that in the gray flat area, the second derivative constraint (||D2u||1) can avoid the "segmented flattening" phenomenon (i.e., adjacent pixel gray levels are forced to be the same level, forming block partitions) caused by the first derivative alone, so that the gray scale change is more natural and continuous. In short, the second derivative constraint, by focusing on "whether the gradient change is reasonable", makes up for the deficiency of the first derivative that only focuses on "whether the gradient exists", and constrains the rationality of the image structure from a higher dimension.

[0069] The core function of the weight coefficient a is to dynamically adjust the proportion of the first derivative and the second derivative in the total constraint, so as to balance the action strength of the two, and finally reduce the block effect. The specific logic is as follows:

[0070] When a takes a small value, the first derivative constraint (||D1u||1) dominates, at this time the image edge retention ability is strong, but block effect may be caused by excessive suppression of high frequency components; when a takes a large value, the proportion of the second derivative constraint (||D2u||1) is improved, at this time the smoothness of the image structure is enhanced, which can effectively alleviate the staircase distortion, but excessive emphasis on the second constraint may cause edge blur. By setting a reasonable a value (which needs to be optimized through experiments according to the material characteristics, defect types, etc. in specific industrial CT scenes), the following can be achieved: in the edge area, the first derivative constraint dominates to ensure clear edges; in the flat area, the second derivative constraint dominates to ensure continuous gray scale change and avoid block effect; overall, the image retains the edge information of the key structure and ensures the smooth transition of the local area, thereby reducing the structure distortion caused by the first or second constraint alone.

[0071] In the high-order regularization function, u * NLM The improved non-local mean filtered image data is u * NLM The structure level similarity data of the U-Net variant lightweight network is introduced, wherein u * NLM The relationship is:

[0072]

[0073] In the formula, i and j are respectively used to represent the row index and the column index of the target pixel in the image, and are used to locate the pixel position that currently needs to be denoised. For example, in an image with a pixel size of N*N, the values of i and j are in the range of 1≤i≤N and 1≤j≤N, which correspond to the coordinates of each specific pixel in the image.

[0074] m and s are respectively used to represent the row index and the column index of the candidate pixel in the image, and are used to locate the other pixel position for similarity comparison of the target pixel i and j. In the NLM algorithm, the candidate pixel usually comes from a search window (i.e., a local area within a certain range) around the target pixel, and the structural similarity between the candidate pixel and the target pixel is calculated to provide a weighted basis for denoising of the target pixel. Similarly, in an image with a pixel size of N*N, the values of m and s are related to the size of the search window, and m and s take values within a small range around i and j;

[0075] h represents a filter coefficient, which is used to control the decay rate of the similarity weight and determines the degree of influence of the similarity distance on the weight. The physical meaning of the filter coefficient is that when the structural level similarity distance between the candidate pixel and the target pixel is small, the value of the exponential term is large, and the candidate pixel has a higher weighted contribution to the target pixel; on the contrary, when the structural level similarity distance is large, the exponential term decays rapidly and the contribution is significantly reduced. The value of the filter coefficient needs to be adjusted according to the image noise level. A larger filter coefficient will make the weight distribution more gentle (the denoising effect is stronger but the details may be blurred), and a smaller filter coefficient will focus more on preserving local structures.

[0076] wherein the structural level similarity distance is the structural level similarity data output by the U-Net variant lightweight network, which is used to improve the similarity measurement of the non-local mean to realize the upgrade from the pixel level to the matching based on the local structure. It is worth mentioning that the structural level similarity data output by the U-Net variant lightweight network can be weight data or distance data. For example, the U-Net variant lightweight network outputs structural level similarity weight, calculates the structural level similarity distance based on the structural level similarity weight, or adjusts the structural level similarity distance based on the structural level similarity weight.

[0077] In step S400, the target function is iteratively updated based on the target image data, the fine gradient direction prediction data and the structural level similarity data by using the alternating direction multiplier method to obtain the reconstructed image data, including:

[0078] In S410, the target image data is taken as the initial value of the iterative image data.

[0079] S420, based on the iterative image data and the fine gradient direction prediction data, updating the second-order total generalized variation of the target function to obtain first updated data;

[0080] S430, based on the iterative image data and the structure-level similarity data, updating the structure-level non-local mean of the target function to obtain second updated data;

[0081] S440, determining the reconstructed image data according to the first updated data and the second updated data;

[0082] S450, performing data consistency constraint processing according to the reconstructed image data;

[0083] S460, determining the image difference between the reconstructed image data of two adjacent iterations, and taking the reconstructed image data as new iterative image data to repeat the above iteration steps until the image difference is less than a preset tolerance or the number of iterations reaches a preset number.

[0084] In step S420, based on the iterative image data and the fine gradient direction prediction data, the second-order total generalized variation of the target function is updated to obtain first updated data, including:

[0085] S421, generating a direction mask according to the fine gradient direction prediction data;

[0086] For example, as described above, the U-Net variant lightweight network can output fine gradient direction prediction data to provide prior edge information, and each pixel value of the fine gradient direction prediction data corresponds to the edge gradient direction at that position in the image, such as the direction of the object contour or the defect boundary. This prediction result is learned based on the projection image data and the ground truth image data in the training data, and can capture complex edges that are easily ignored by traditional gradient calculation, such as fine structures with large curvature changes or multiple material junctions. Based on the fine gradient direction prediction data, a direction mask M is generated, in which the pixel values of the real edge direction area are 1 and the pixel values of the artifact or noise area are 0, thereby providing a directional basis for updating the TGV term.

[0087] S422, determining the first derivative and the second derivative of the iterative image data;

[0088] For example, the first derivative and the second derivative of the current iterative image data are determined by calculation, and the calculation method of the derivative is a routine technical means for those skilled in the art, which will not be described in this embodiment. The iterative image data is denoted as u prev , the first derivative is denoted as D1u prev , and the second derivative is denoted as D2u prev .

[0089] S423、According to the direction mask, the first-order derivative and the second-order derivative of the iterative image data are modified respectively to obtain a directional first-order derivative and a directional second-order derivative;

[0090] For example, the directional first-order derivative D1u' = M*D1u prev Only the gradient component of the real edge direction is retained, and the false gradient caused by the artifact is weakened; the directional second-order derivative D2u' = M*D2u prev Only the continuous region of the real edge is applied to the smoothing constraint to avoid excessive smoothing of the non-edge region.

[0091] S424、According to the directional first-order derivative and the directional second-order derivative, the gradient of the second-order total generalized variation is determined;

[0092] For example, the gradient of the second-order total generalized variation is denoted as ▽||u|| TGV Then ▽||u|| TGV = sign(D1u') + a*sign(D2u'), where sign represents a sign function, returns 1 for positive input, 0 for zero, and -1 for negative input. The weight a is determined by experiment optimization and needs to adapt to the material properties and defect types of the industrial CT scene. For example, in the edge region, the value of a is small, the first-order derivative constraint dominates to ensure sharp edges; in the flat region, the value of a is large, the second-order derivative constraint dominates to ensure continuous gray scale change and avoid step-shaped blocking effects.

[0093] S425, update the iterative image data along the opposite direction of the gradient of the second-order total generalized variation to obtain first updated data.

[0094] For example, the data is updated along the opposite direction of the gradient, and the updated data is denoted as u TGV Then u TGV= u prev -β*▽||u|| TGV Where β is the iteration step size. In this way, the image retains clear gradient characteristics in the real edge region and weakens false gradient characteristics in the artifact or noise region, achieving the dual effects of streak artifact suppression and blocking effect reduction.

[0095] Step S430, based on the iterative image data and the structure-level similarity data, the structure-level non-local mean of the target function is updated to obtain second updated data, including:

[0096] S431, determine the target pixel of the iterative image data;

[0097] S432, determine the search window of the target pixel, and determine the pixels in the search window as candidate pixels;

[0098] For example, for a target pixel (i, j), candidate pixels (m, s) are selected in a preset search window, which are potential structure similar regions. The size of the search window can be adjusted adaptively according to actual application, for example, the size of the search window is 5x5 or 7x7.

[0099] S433, determine the weight value of each candidate pixel according to the structure level similarity data;

[0100] For example, as described above, the similarity measure of traditional NLM is based on the gray Euclidean distance of pixel neighborhood, only focusing on the numerical difference of pixel gray, and easily misjudging the gray fluctuation caused by noise as structural difference, or misjudging the region with similar structure but slightly different gray as dissimilar. The present embodiment upgrades the pixel level similarity measure of traditional NLM to local structure based matching by introducing structure level similarity distance, thereby better preserving the edges and detailed structures of the image while suppressing noise. The structure level similarity distance corresponding to the target pixel (i, j) is denoted as In some application examples, the structure level similarity distance can be the structure similarity data output by the U-Net variant lightweight network, or in other application examples, the structure similarity data output by the U-Net variant lightweight network is structure level similarity weight data, and the structure level similarity distance is determined according to the structure level similarity weight data. The structure level similarity distance is a distance calculated based on local structure features (such as gradient direction and texture consistency, etc.), and the smaller the value of the structure level similarity distance, the more similar the local structure of the target pixel and the candidate pixel. In the defect edge region of industrial CT image, traditional non-local mean value may determine that the pixels on both sides of the edge are dissimilar due to the gray mutation on both sides of the edge, but the structure level non-local mean value of the present embodiment can identify that both sides belong to the edge structure through the structure level similarity distance, and determine that they are highly similar, thereby preserving the continuity of the edge when denoising. According to the u * NLM According to the relationship, the weight value of the candidate pixel (m, s) is:

[0101] ,

[0102] Wherein, the filter coefficient h can be dynamically adjusted according to the noise level or optimized by network training.

[0103] S434, weighted sum according to the weight value and the gray value of each candidate pixel to obtain second update data.

[0104] Exemplarily, the gray values of all candidate pixels in the search window are weighted and summed to obtain the denoised value of the target pixel (i, j), achieving the effect of "smoothing noise while preserving structural features with the pixel values of the structurally similar region", wherein the gray value of the candidate pixel is .

[0105] It is worth mentioning that the structural level non-local mean filter in the embodiment is not executed independently, but is dynamically updated based on the current iteration image data u n-1 and the structural level similarity data output by the U-Net variant lightweight network. For example, by using the local structural information of the current iteration image data, combining the structural level similarity data, the similarity weight value of each target pixel is recalculated to ensure that the weight value adapts to the changes of the image in iteration. Then, the current iteration image data u n-1 is denoised by weighted summation to obtain new iteration image data u n . The new iteration image data u n is taken as the input of the data consistency constraint processing to ensure that the denoised image still matches the projection image data, avoiding over-smoothing leading to structural distortion.

[0106] Step S450, performing data consistency constraint processing according to the reconstructed image data, comprising:

[0107] S451, generating theoretical simulation projection image data based on the reconstructed image data;

[0108] Exemplarily, in the process of iteration update, the reconstructed image data is constantly changing until the final reconstructed image data is output after meeting the condition. Wherein, the current reconstructed image data is taken as the input when performing data consistency constraint processing. After updating in steps S420 and S430, the TGV term (||u TGV ) and the NLM term (u * NLM ) can be obtained, then the reconstructed image data can be determined according to the objective function in the above, that is, the implementation manner of step S440 is: substituting the first update data and the second update data into the objective function to determine the image reconstruction data. Based on the reconstructed image data (denoted as u), the theoretical simulation projection data is generated through the system matrix W, denoted as .

[0109] S452, determining image residual data according to the target image data and the theoretical simulation projection image data;

[0110] Exemplarily, the image residual data is denoted as r, and the target image data is denoted as f, then the image residual data is used to measure the deviation between the target image data and the theoretical simulation projection image data.

[0111] S453, adjusting the iterative image data according to the image residual data.

[0112] Exemplarily, the back-projection matrix W T The image residual data is mapped back to the image domain by the relationship u n = u n-1 + γW T The iterative image data is adjusted to reduce the bias to within the tolerance ξ, where γ is the step size. It should be noted that the tolerance ξ and the step size γ are adjusted according to the projection noise level to avoid over-tight or over-tight constraints, otherwise structural distortion or noise will be introduced. The data consistency constraint processing cooperates with the TGV term regularization of step S420 and the structure-level NLM noise reduction of step S430, balances the structural rationality and physical reality, and ensures that the reconstruction result not only suppresses artifacts and noise, but also fits the actual data.

[0113] In the process of iterative updating, the preset number of iterations or the image difference between the previous and next iterations can be used as the iteration termination condition. The image difference between the previous and next iterations can be realized by quantifying the pixel-level difference between the adjacent two iterations, for example, by calculating the L2 norm or mean square error. After the iteration is completed, the final reconstructed image is saved in a standard format for subsequent defect analysis or visualization processing.

[0114] The architecture of the U-Net variant lightweight network (hereinafter referred to as the network) will be described below. The network is based on the classic "encoder-decoder" symmetric structure of U-Net, and is optimized for lightweight for the needs of industrial CT fast reconstruction. The overall architecture can be divided into five parts: input layer, encoder module, bottleneck layer, decoder module and output layer. Each module is connected by a skip connection to realize cross-layer feature fusion, and the structure is compact and has high computational efficiency.

[0115] 1. Input layer

[0116] Input content: receive pre-processed target image data, where the target image data is a single-channel grayscale image in raw format, with a pixel size of 3040x2400, and the pixel size corresponds to the CT detector parameters.

[0117] Function: convert the target image data into a tensor format that can be processed by the U-Net variant lightweight network, providing initial input for subsequent feature extraction without changing the spatial size and basic information of the data.

[0118] 2. Encoder module (downsampling stage)

[0119] The encoder module includes 3-4 down-sampling stages (the number of layers is simplified according to the lightweight requirement), each stage including a convolution unit and a down-sampling layer:

[0120] Convolution unit: Each unit contains 1-2 3x3 lightweight convolution layers (using depthwise separable convolution or reducing the number of convolution kernels to reduce computation), combined with batch normalization (BN) and ReLU activation function.

[0121] Function: Extract multi-scale features of projection data layer by layer, from low-level features (such as noise texture, local edge) to intermediate features (such as regional structure, defect outline), enhance feature expression ability.

[0122] Down-sampling layer: Down-sampling is achieved by using 2x2 max pooling or convolution layer with step size 2.

[0123] Function: By reducing the spatial size of the feature map (size is halved after each down-sampling), increasing the number of feature channels (modest increase in channel number under lightweight design, avoiding redundancy), focusing on global structure information extraction.

[0124] 3. Bottleneck layer

[0125] Located between the encoder module and the decoder module, it is the deepest feature processing unit in the network:

[0126] Structure: Contains 1-2 3x3 convolution units (same as the lightweight design of the encoder module), without down-sampling operation.

[0127] Function: Integrates the deep abstract features output by the encoder module, captures the most critical global structure information in the target image data (such as the overall shape of the workpiece, the distribution of large-scale defects), and provides semantic anchor points for feature restoration of the decoder module.

[0128] 4. Decoder module (up-sampling stage)

[0129] Symmetrical with the encoder module, it contains 3-4 up-sampling stages, each stage including an up-sampling layer, a feature fusion unit, and a convolution unit:

[0130] Up-sampling layer: Up-sampling is achieved by using transpose convolution unit (1x1 or 3x3) or interpolation unit (such as bilinear interpolation).

[0131] Function: Gradually restore the spatial size of the feature map (size doubles after each up-sampling), reduce the number of channels, and match the feature scale of the corresponding stage of the encoder module.

[0132] Feature fusion unit: The shallow features (containing detailed information) of the corresponding stage of the encoder module are spliced or weighted fused with the current deep features (containing semantic information) of the decoder module through jump connection.

[0133] Function: Supplement the detailed information (such as small edges, local texture) lost in the up-sampling process, balance the feature expression of global structure and local details.

[0134] Convolutional unit: Same as the design of the encoder module, each convolutional unit contains 1-2 3x3 lightweight convolutional layers, with batch normalization (BN) and ReLU activation function.

[0135] Function: Further processing of the fused features, eliminating redundant information and strengthening effective features (such as defect edges and structural textures).

[0136] 5. Output layer

[0137] Structure: Contains 1 1x1 convolutional layer (without activation function), and the output channel number is 2.

[0138] Function: Convert the feature map output by the decoder module into two target outputs:

[0139] Fine gradient direction prediction data: Single-channel image, pixel value reflects the edge gradient direction at the corresponding position, which helps subsequent high-order regularization (TGV term) to accurately capture complex edges and suppress streak artifacts.

[0140] Structural similarity data: Single-channel image, pixel value is structural similarity distance or structural similarity weight, used to improve the similarity measure of non-local mean (NLM), upgrade from pixel-level matching to local structure-based matching, and reduce blocking artifacts.

[0141] Core design features: Lightweight is reflected in simplifying the number of convolutional layers, using lightweight convolutional operations, and reducing the number of channel expansions, which improves the calculation speed while ensuring the feature extraction capability, and adapts to the rapid reconstruction requirements of industrial CT; Through the design of skip connection and double output, the dual functions of "detail capture-structure modeling" are realized, providing key auxiliary information for subsequent iterative reconstruction.

[0142] Network training is performed using a small-scale industrial CT dataset (containing samples of different materials and defects) in an end-to-end manner with "projection data-true value image" pairs, and the loss function is mean square error (MSE) + structural similarity loss (SSIMLoss).

[0143] Please refer to Figure 2 , the embodiment of the present application provides an industrial CT image reconstruction device, comprising:

[0144] The data acquisition module 110 is configured to acquire projection image data, wherein the projection image data is obtained by using an industrial CT scanning device in a fast scanning mode.

[0145] The preprocessing module 120 is configured to preprocess the projection image data to obtain target image data.

[0146] The first processing module 130 is configured to process the target image data based on a pre-trained U-Net variant lightweight network to obtain fine gradient direction prediction data and structure-level similarity data, the U-Net variant lightweight network is trained based on the projection image data and the ground truth image data, and the loss function of the U-Net variant lightweight network includes a structure similarity loss.

[0147] The second processing module 140 is configured to perform iterative updating on a target function based on the target image data, the fine gradient direction prediction data and the structure-level similarity data by using an alternating direction multiplier method to obtain reconstructed image data, the target function is a high-order regularization function constructed based on a second-order total generalized variation and a structure-level non-local mean of iterative image data.

[0148] The inventive concept of the industrial CT image reconstruction device embodiment is the same as that of the above-described industrial CT image reconstruction method embodiment, and the content not involved in the industrial CT image reconstruction device embodiment can refer to the above-described industrial CT image reconstruction method embodiment, which will not be described here. The target image data is processed based on the pre-trained U-Net variant lightweight network to obtain fine gradient direction prediction data and structure-level similarity data, the fine gradient direction prediction data can assist the second-order total generalized variation of the high-order regularization to capture the complex edge of the target image data, and the structure-level similarity data can assist the structure-level non-local mean of the high-order regularization function to realize the matching of the local structure, the target function is a high-order regularization function constructed based on the second-order total generalized variation and the structure-level non-local mean of the iterative image data, the second-order total generalized variation suppresses noise and artifacts by controlling the gradient and curvature of the image while maintaining the edge and details of the image, and the structure-level non-local mean reduces noise and suppresses the blocking effect in the image reconstruction process by using the information of similar regions in the image, so that when the image data obtained in the fast scanning mode is reconstructed, the streak artifacts and photon noise of the sparse image can be suppressed, thereby meeting the requirements of fast scanning and sparse image reconstruction quality.

[0149] Please refer to Figure 3The embodiment of the present application provides an electronic device, which comprises a processor 210 and a memory 220, the memory 220 stores a computer program, and the processor 210 is used to realize the industrial CT image reconstruction method described above when the computer program is run. The details of the industrial CT image reconstruction method can be referred to the above, and will not be repeated here. The target image data is processed based on the pre-trained U-Net variant lightweight network to obtain fine gradient direction prediction data and structure level similarity data. The fine gradient direction prediction data can assist the second-order total generalized variation of the high-order regularization to capture the complex edge of the target image data. The structure level similarity data can assist the structure level non-local mean of the high-order regularization function to realize the matching of the local structure. The target function is a high-order regularization function constructed based on the second-order total generalized variation and the structure level non-local mean of the iterative image data. The second-order total generalized variation controls the gradient and curvature of the image to suppress noise and artifacts, while maintaining the edge and details of the image. The structure level non-local mean uses the information of the similar areas in the image to reduce noise and suppress the blocking effect in the image reconstruction process. Therefore, when the image data obtained in the fast scanning mode is reconstructed, the streak artifacts and photon noise of the sparse image can be suppressed, so as to meet the requirements of fast scanning and sparse image reconstruction quality.

[0150] The embodiment of the present application provides a storage device, which stores a computer program, and the computer program realizes the industrial CT image reconstruction method described above when the computer program is run. The details of the industrial CT image reconstruction method can be referred to the above, and will not be repeated here. The target image data is processed based on the pre-trained U-Net variant lightweight network to obtain fine gradient direction prediction data and structure level similarity data. The fine gradient direction prediction data can assist the second-order total generalized variation of the high-order regularization to capture the complex edge of the target image data. The structure level similarity data can assist the structure level non-local mean of the high-order regularization function to realize the matching of the local structure. The target function is a high-order regularization function constructed based on the second-order total generalized variation and the structure level non-local mean of the iterative image data. The second-order total generalized variation controls the gradient and curvature of the image to suppress noise and artifacts, while maintaining the edge and details of the image. The structure level non-local mean uses the information of the similar areas in the image to reduce noise and suppress the blocking effect in the image reconstruction process. Therefore, when the image data obtained in the fast scanning mode is reconstructed, the streak artifacts and photon noise of the sparse image can be suppressed, so as to meet the requirements of fast scanning and sparse image reconstruction quality.

[0151] The above describes the embodiments of the present application in detail in combination with the drawings, but the present application is not limited to the above embodiments, and various changes can be made within the knowledge range of those skilled in the art without departing from the purpose of the present application.

Claims

1. An industrial CT image reconstruction method, characterized by, The method comprises the following steps: acquiring projection image data obtained by an industrial CT scanning device in a fast scanning mode; preprocessing the projection image data to obtain target image data; processing the target image data based on a pre-trained U-Net variant lightweight network to obtain fine gradient direction prediction data and structure-level similarity data, wherein the U-Net variant lightweight network is trained based on projection image data and true value image data, and a loss function of the U-Net variant lightweight network comprises a structure similarity loss; Based on the target image data, the fine gradient direction prediction data and the structure level similarity data, a target function is iteratively updated by using an alternating direction multiplier method to obtain reconstructed image data, the target function being: u = argmin(||u||1+λ||u||2), s.t.||f-Wu||2≤ξ, wherein u is iterative image data, ||u||1 is a first-order total generalized variation of the iterative image data, ||u||2 is a second-order total generalized variation of the iterative image data, u is a structure level non-local mean of the iterative image data, f is projection image data, W is a system matrix, λ is a regularization coefficient, and ξ is a preset tolerance, ||u||1 = ||D1u||1+α||D2u||1, wherein D1u is a first-order derivative of the iterative image data, D2u is a second-order derivative of the iterative image data, and α is a weight coefficient. TGV +λ u * NLM ), s.t.||f W u||2≤ξ, wherein u is iterative image data, ||u|| TGV is a first-order total generalized variation of the iterative image data, ||u|| * NLM is a second-order total generalized variation of the iterative image data, u TGV is a structure level non-local mean of the iterative image data, f is projection image data, W is a system matrix, λ is a regularization coefficient, and ξ is a preset tolerance, ||u||1 = ||D1u||1+α||D2u||1, wherein D1u is a first-order derivative of the iterative image data, D2u is a second-order derivative of the iterative image data, and α is a weight coefficient. in the process of iteratively updating a target function, based on the iterative image data and the fine gradient direction prediction data, the second-order total generalized variation of the target function is updated to obtain first update data, comprising: generating a direction mask according to the fine gradient direction prediction data; determining the first derivative and the second derivative of the iterative image data; correcting the first derivative and the second derivative of the iterative image data according to the direction mask to obtain directional first derivative and directional second derivative; determining the gradient of the second-order total generalized variation based on the directional first derivative and the directional second derivative; updating the iterative image data in the opposite direction of the gradient of the second-order total generalized variation to obtain first update data.

2. The industrial CT image reconstruction method of claim 1, wherein, based on the target image data, the fine gradient direction prediction data and the structure-level similarity data, the target function is iteratively updated by using an alternating direction multiplier method to obtain reconstructed image data, comprising: taking the target image data as the initial value of the iterative image data; updating the second-order total generalized variation of the target function based on the iterative image data and the fine gradient direction prediction data to obtain first update data; updating the structure-level non-local mean of the target function based on the iterative image data and the structure-level similarity data to obtain second update data; determining the reconstructed image data according to the first update data and the second update data; performing data consistency constraint processing according to the reconstructed image data; determining the image difference between the reconstructed image data of two adjacent iterations, and taking the reconstructed image data as new iterative image data to repeat the above iteration steps until the image difference is less than a preset tolerance or the number of iterations reaches a preset number.

3. The industrial CT image reconstruction method of claim 2, wherein, updating the structure-level non-local mean of the target function based on the iterative image data and the structure-level similarity data to obtain second update data, comprising: determining the target pixel of the iterative image data; determining the search window of the target pixel, and determining the pixels in the search window as candidate pixels; determining the weight value of each candidate pixel according to the structure-level similarity data; performing weighted summation according to the weight value and the gray value of each candidate pixel to obtain second update data.

4. The method of industrial CT image reconstruction of claim 2, wherein, the data consistency constraint processing according to the reconstructed image data, comprising: generating theoretical simulation projection image data based on the reconstructed image data; determine image residual data according to the target image data and the theoretical simulation projection image data; adjust the iterative image data according to the image residual data.

5. An industrial CT image reconstruction apparatus characterized by comprising: The method comprises: a data acquisition module configured to acquire projection image data, the projection image data being image data obtained by an industrial CT scanning device in a fast scanning mode; a preprocessing module configured to preprocess the projection image data to obtain target image data; a first processing module configured to process the target image data based on a pre-trained U-Net variant lightweight network to obtain fine gradient direction prediction data and structure-level similarity data, the U-Net variant lightweight network being trained based on projection image data and true value image data, and a loss function of the U-Net variant lightweight network comprising a structure similarity loss; a second processing module configured to perform iterative update on a target function by using an alternating direction method of multipliers based on the target image data, the fine gradient direction prediction data and the structure level similarity data, to obtain reconstructed image data, the target function being: u = argmin(||u||2+λ||f-Wu||2), s.t.||f-Wu||2≤ξ, where u is the iterative image data, ||u||2 is a second order total generalized variation of the iterative image data, u is a structure level non-local mean of the iterative image data, f is the projection image data, W is a system matrix, λ is a regularization coefficient, and ξ is a preset tolerance, and ||u||2 = ||D1u||1+α||D2u||1, where D1u is a first order derivative of the iterative image data, D2u is a second order derivative of the iterative image data, and α is a weight coefficient. TGV * NLM TGV * NLM TGV ​​​​​​​​ in the process of iterative updating of the target function, first update data is obtained by updating a second-order total generalized variation of the target function based on the iterative image data and the fine gradient direction prediction data, comprising: generating a direction mask according to the fine gradient direction prediction data; determining a first-order derivative and a second-order derivative of the iterative image data; correcting the first-order derivative and the second-order derivative of the iterative image data respectively according to the direction mask to obtain a directional first-order derivative and a directional second-order derivative; determining a gradient of the second-order total generalized variation according to the directional first-order derivative and the directional second-order derivative; updating the iterative image data in the opposite direction of the gradient of the second-order total generalized variation to obtain the first update data.

6. An electronic device comprising a processor and a memory, said memory having stored therein a computer program, characterized in that, The processor, when running the computer program, is configured to implement the industrial CT image reconstruction method according to any one of claims 1 to 4.

7. A storage device having stored therein a computer program, characterized by The computer program, when being run, is configured to implement the industrial CT image reconstruction method according to any one of claims 1 to 4.

Images