Terahertz image super-resolution reconstruction method, computer program product and computing device

Abstract

The invention provides a terahertz image super-resolution reconstruction method and computing equipment, and the method comprises the steps: obtaining a first image from a terahertz imaging system, and enabling the first image to have a resolution lower than a first threshold value; performing data reconstruction on the first image through a compressed sensing technology to form a second image; and carrying out super-resolution reconstruction on the second image by utilizing a pre-training diffusion model which is finely adjusted through transfer learning. According to the technical scheme provided by the invention, the resolution and the overall quality of the terahertz image can be improved by applying an image processing technology, and a high-resolution path image is obtained, so that the defects of the terahertz imaging technology in a safety detection scene are overcome, and efficient and comprehensive safety detection is realized.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to a method for super-resolution reconstruction of terahertz images, a computer program product and a computing device. Background Art

[0002] Terahertz waves, also known as far infrared rays, refer to electromagnetic waves with frequencies between 0.1 and 10 THz (wavelengths between 30 μm and 3 mm). This waveband is between microwaves and infrared rays, and belongs to the category of far infrared and submillimeter waves. Compared with microwaves, terahertz waves have shorter wavelengths and can provide higher resolution; compared with infrared rays, they show stronger penetrating power, for example: they can easily penetrate everyday objects such as clothing, but will be absorbed or reflected by materials such as metals and liquids. Terahertz waves have low photon energy and will not cause harm to the human body, so they are very suitable for human security inspections.

[0003] Terahertz imaging technology is particularly advantageous in the field of human security inspection, as it can detect objects hidden under clothing, including non-metallic objects. Existing security inspection technologies, such as optical / infrared imagers, X-ray systems, CT detection systems, and metal detectors, all have their own limitations. Among them, optical and infrared imaging cannot penetrate coverings; X-rays and CT have good penetration but high radiation; and metal detectors cannot detect non-metallic objects. Terahertz technology can effectively circumvent the defects of these technologies and improve the accuracy and efficiency of security inspections.

[0004] However, the existing terahertz imaging technology also has the defect of low image quality. The images obtained by terahertz imaging technology are often blurred and have artifacts, and the contrast is low, making it difficult to identify hidden objects, which greatly increases the possibility of misdetection or missed detection. Therefore, the improvement of image quality is the focus of current research, aiming to solve this problem and improve the application effect and reliability of terahertz imaging in security inspection.

[0005] To this end, a technical solution is needed that can apply image processing technology to improve the resolution and overall quality of terahertz images and obtain high-resolution road images, thereby solving the defects of terahertz imaging technology in security detection scenarios and achieving efficient and comprehensive security detection. Summary of the invention

[0006] The present invention aims to provide a method and computing device for super-resolution reconstruction of terahertz images, which can apply image processing technology to improve the resolution and overall quality of terahertz images and obtain high-resolution road images, thereby solving the defects of terahertz imaging technology in security detection scenarios and realizing efficient and comprehensive security detection.

[0007] According to one aspect of the present invention, a method for super-resolution reconstruction of a terahertz image is provided, comprising:

[0008] A first image obtained from the terahertz imaging system has a resolution below a first threshold;

[0009] Reconstructing the first image using compressed sensing technology to form a second image;

[0010] The second image is super-reconstructed using a pre-trained diffusion model fine-tuned by transfer learning.

[0011] According to some embodiments, the method further comprises collecting and processing data for training the pre-trained diffusion model:

[0012] Acquire a data set of high-definition images, wherein the high-definition images have a resolution higher than a second threshold;

[0013] Preprocessing the high-definition image to form a low-resolution image set, wherein the low-resolution image set has a resolution lower than the first threshold;

[0014] The low-resolution image is fitted using a Lasso regression model to determine the features of non-zero coefficients and form a training image set for training.

[0015] According to some embodiments, preprocessing the high-definition image includes:

[0016] Using a downsampling technique to reduce the resolution of the high-definition image to simulate the resolution loss in terahertz imaging, thereby obtaining a sampled image;

[0017] The downsampled image is interpolated using a bicubic interpolation method to form a first low-resolution image.

[0018] According to some embodiments, preprocessing the data set of the high-definition image further includes:

[0019] Selecting a transform domain to perform sparse representation on the high-definition image;

[0020] The Bernoulli matrix is ​​used as the measurement matrix to randomly sample the sparsely represented high-definition image to obtain a compressed measurement vector;

[0021] A second low-resolution image is formed based on the measurement vectors using matching pursuit.

[0022] According to some embodiments, fitting the low-resolution image using a Lasso regression model to determine the characteristics of non-zero coefficients includes:

[0023] The first low-resolution image and the second resolution image are respectively fitted using a Lasso regression model, and an intersection of features of non-zero coefficients is determined to form a training image.

[0024] According to some embodiments, further comprising:

[0025] Pre-train the Stable Diffusion model on a color image dataset to learn texture, edge, and high-frequency information;

[0026] The low-resolution image set used for training is used to perform transfer learning fine-tuning on the pre-trained Stable Diffusion model, thereby obtaining the pre-trained diffusion model fine-tuned by transfer learning.

[0027] According to some embodiments, obtaining a first image from a terahertz imaging system using a compressed sensing technique includes:

[0028] Determine a transform domain in which the imaging signal is in a sparse state, and decompose and represent the imaging signal in the transform domain;

[0029] The imaging signal after decomposition representation is randomly sampled using a Bernoulli matrix as a measurement matrix to obtain the first image.

[0030] According to some embodiments, reconstructing data of the first image by using a compressed sensing technology includes:

[0031] The image data is fitted using the Lasso regression model to determine the features of the non-zero coefficients for data reconstruction. According to another aspect of the present invention, a computer program product is provided, comprising: a computer program, wherein when the computer program is executed by a processor, the method described in any one of the above items is implemented.

[0032] According to another aspect of the present invention, there is provided a computing device, comprising:

[0033] Processor; and

[0034] A memory storing a computer program, which, when executed by the processor, enables the processor to execute any of the methods described above.

[0035] According to another aspect of the present invention, there is provided a non-transitory computer-readable storage medium having computer-readable instructions stored thereon, and when the instructions are executed by a processor, the processor is caused to execute any of the above methods.

[0036] According to an embodiment of the present invention, by utilizing compressed sensing technology, high-quality images are restored from data with a sampling rate lower than the traditional Nyquist rate, which greatly reduces the amount of data required in the data acquisition process, thereby reducing the bandwidth and storage requirements of the imaging device, so that efficient image acquisition can be performed even in resource-limited environments; selecting the most influential features from a large number of potential image features through the Lasso regression model is crucial to improving the quality of super-resolution reconstruction. It not only provides a sparse solution, but also avoids overfitting through regularization, ensuring the generalization ability of the model, and can effectively reduce and denoise during the diffusion process, thereby reliably and efficiently generating high-resolution images; by combining the transfer learning of the StableDiffusion model, after pre-training on color images, it is adjusted to the terahertz image processing task through transfer learning, thereby improving the model's ability to process single-channel grayscale images, overcoming the difficulty of applying the model directly from color to grayscale, and realizing the effective transfer of model knowledge, so that the pre-trained model can better adapt to the characteristics of terahertz images.

[0037] It is to be understood that the foregoing general description and the following detailed description are exemplary only and are not restrictive of the invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for describing the embodiments are briefly introduced below.

[0039] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for describing the embodiments are briefly introduced below.

[0040] Figure 1 A flow chart of a method for super-resolution reconstruction of a terahertz image according to an example embodiment is shown.

[0041] Figure 2 A schematic diagram of a method for super-resolution reconstruction of a terahertz image according to an example embodiment is shown.

[0042] Figure 3 A low-resolution image input diagram is shown according to an experimental method for terahertz image super-resolution reconstruction.

[0043] Figure 4 A high-resolution image output diagram is shown according to an experimental method for terahertz image super-resolution reconstruction.

[0044] Figure 5 A LOSS fitting curve diagram based on an experimental method for super-resolution reconstruction of terahertz images is shown.

[0045] Figure 6 A block diagram of a computing device is shown according to an exemplary embodiment. DETAILED DESCRIPTION

[0046] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in many forms and should not be construed as limited to the embodiments set forth herein; rather, these embodiments are provided so that the present invention will be comprehensive and complete and fully convey the concepts of the example embodiments to those skilled in the art. The same reference numerals in the figures represent the same or similar parts, and thus their repeated description will be omitted.

[0047] In addition, the described features, structures or characteristics may be combined in one or more embodiments in any suitable manner. In the following description, many specific details are provided to provide a full understanding of the embodiments of the present invention. However, those skilled in the art will appreciate that the technical solution of the present invention can be practiced without one or more of the specific details, or other methods, components, devices, steps, etc. may be adopted. In other cases, known methods, devices, implementations or operations are not shown or described in detail to avoid blurring various aspects of the present invention.

[0048] The block diagrams shown in the accompanying drawings are merely functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities may be implemented in software form, or in one or more hardware modules or integrated circuits, or in different networks and / or processor devices and / or microcontroller devices.

[0049] The flowcharts shown in the accompanying drawings are only exemplary and do not necessarily include all the contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps can be decomposed, and some operations / steps can be combined or partially combined, so the actual execution order may change according to actual conditions.

[0050] It should be understood that although the terms first, second, third, etc. may be used herein to describe various components, these components should not be limited by these terms. These terms are used to distinguish one component from another component. Therefore, the first component discussed below can be referred to as the second component without departing from the teachings of the present inventive concept. As used herein, the term "and / or" includes any one of the associated listed items and all combinations of one or more.

[0051] The user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in the present invention are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0052] Those skilled in the art will appreciate that the drawings are merely schematic diagrams of example embodiments, and the modules or processes in the drawings are not necessarily necessary for implementing the present invention, and therefore cannot be used to limit the protection scope of the present invention.

[0053] Image super-resolution (SR) technology is a key technology in the field of computer vision. It is dedicated to reconstructing high-resolution (HR) images from low-resolution (LR) images to restore the lost details and textures in the image. It is widely used in various fields, such as satellite imaging, medical imaging, video enhancement, and security monitoring, and has high research and commercial value.

[0054] At the end of the 20th century, research on image super-resolution technology began to emerge. The initial methods mainly used interpolation techniques, such as bilinear interpolation and bicubic interpolation. Although the methods are simple and fast, the image quality obtained is difficult to meet the high-quality requirements. Subsequently, more complex algorithms were proposed, such as reconstruction based on learning. By learning a large number of low-resolution and high-resolution image pairs, a mapping relationship is established, thereby achieving more accurate high-resolution image reconstruction.

[0055] With the rise of machine learning, especially deep learning in the 21st century, image super-resolution technology has ushered in a revolutionary development. Deep learning models, especially convolutional neural networks (CNN), have become the main method for super-resolution reconstruction due to their powerful feature extraction capabilities. In 2014, the SRCNN (Super-Resolution Convolutional Neural Network) model was proposed. This model learns end-to-end mapping relationships through a three-layer convolutional network, significantly improving the quality of reconstructed images. Subsequently, more super-resolution models based on deep learning were proposed, such as ESPCN, VDSR, SRGAN, etc. These models have made a lot of innovations and optimizations in network structure, loss function and training strategy, which has promoted the update and further development of super-resolution technology.

[0056] At present, although image super-resolution technology has made significant progress, it still faces many challenges, such as processing super-resolution of real-time video, improving the versatility of the algorithm, and optimizing the computational efficiency during large-scale deployment. Future research may focus more on improving the practicality and efficiency of the algorithm, as well as exploring more innovative neural network architectures and learning strategies to further push the boundaries of technology. In short, image super-resolution technology is an active and rapidly developing research field both domestically and internationally. Continuous technological innovation and broad application prospects make it an important branch in the field of computer vision.

[0057] To this end, the present invention proposes a method, a computer program and a computing device for super-resolution reconstruction of terahertz images, which can apply image processing technology to improve the resolution and overall quality of terahertz images and obtain high-resolution road images, thereby solving the defects of terahertz imaging technology in security detection scenarios and realizing efficient and comprehensive security detection. According to the embodiments, by utilizing compressed sensing technology, high-quality images are recovered from data with a sampling rate lower than the traditional Nyquist rate, which greatly reduces the amount of data required in the data acquisition process, thereby reducing the bandwidth and storage requirements of the imaging device, so that efficient image acquisition can be performed even in resource-limited environments; selecting the most influential features from a large number of potential image features through the Lasso regression model is crucial to improving the quality of super-resolution reconstruction. It not only provides a sparse solution, but also avoids overfitting through regularization, ensuring the generalization ability of the model, and can effectively reduce and denoise during the diffusion process, thereby reliably and efficiently generating high-resolution images; by combining transfer learning with the Stable Diffusion model, after pre-training on color images, it is adjusted to the terahertz image processing task through transfer learning, thereby improving the model's ability to process single-channel grayscale images, overcoming the difficulty of applying the model directly from color to grayscale, and realizing the effective transfer of model knowledge, so that the pre-trained model can better adapt to the characteristics of terahertz images.

[0058] Exemplary embodiments of the present invention are described below with reference to the accompanying drawings.

[0059] Figure 1 A flow chart of a method for super-resolution reconstruction of a terahertz image according to an example embodiment is shown.

[0060] See also Figure 1 , the figure shows a method for super-resolution reconstruction of terahertz images, the specific steps include:

[0061] At S101 , a first image is obtained from a terahertz imaging system, wherein the first image has a resolution lower than a first threshold.

[0062] According to some embodiments, a first image is selected from a terahertz imaging system, the image having a resolution below a predetermined threshold.

[0063] In S103, data of the first image is reconstructed by using a compressed sensing technology to form a second image.

[0064] According to some embodiments, the low-resolution first image is reconstructed using compressed sensing technology to generate a second image. Compressed sensing is a signal processing technique that allows the acquisition of signals at a sampling rate far below the Nyquist rate and the recovery of the original signal from a small number of measurements through an optimization algorithm. This method is particularly suitable for sparse signals or signals that can be converted to sparse representations.

[0065] At S105 , super-resolution reconstruction is performed on the second image using the pre-trained diffusion model fine-tuned by transfer learning.

[0066] According to some embodiments, a pre-trained diffusion model that has been fine-tuned by transfer learning is used to further reconstruct the second image in super-resolution. The diffusion model is a type of generative model based on deep learning that can retain the authenticity and details of the image while improving the image resolution. Transfer learning can apply the knowledge learned by a model on one task to another related task. In the solution of the present invention, a diffusion model that has been trained on a large number of images is used and fine-tuned according to the characteristics of the terahertz image to better adapt to specific application scenarios.

[0067] According to some embodiments, before this, data for training the pre-trained diffusion model is collected and processed: a data set of high-definition images is obtained, the high-definition images have a resolution higher than a second threshold; the high-definition images are pre-processed to form low-resolution images, the low-resolution images have a resolution lower than the first threshold. Afterwards, the low-resolution images are fitted using a Lasso regression model to determine the characteristics of non-zero coefficients to form a training image set for training. High-quality terahertz images are collected as training and testing data sets. A high-definition terahertz image data set can be obtained through experimental acquisition, downloading from a public database, or cooperative acquisition. Ensure that the data set has sufficient diversity so that the model can generalize to different scenarios.

[0068] According to some embodiments, important features that have a significant impact on image reconstruction are identified and extracted. Traditional image processing techniques, such as edge detection, texture analysis, etc., can usually be used to extract features. A pre-trained convolutional neural network (CNN) is used to extract high-level abstract features.

[0069] According to some embodiments, first, determine which features are most important for image reconstruction. Then, the important features of the high-resolution image are used as the target variable Y, and the corresponding features of the low-resolution image are used as the explanatory variable X. The important feature data set is fitted using the Lasso regression model to determine the features of the non-zero coefficients to form a training image set for training. Before fitting, all features need to be standardized to ensure that they are suitable for the model.

[0070] According to some embodiments, a Stable Diffusion model and transfer learning technology are used to generate a high-resolution image from a low-resolution image. The Stable Diffusion model is a generative model based on a diffusion process, which generates high-quality images by gradually adding noise to the image and then inversely denoising. This model performs well in image generation tasks, and uses the knowledge in the pre-trained model to accelerate convergence and improve reconstruction quality, thereby achieving efficient terahertz image super-resolution reconstruction. The method designed by the scheme of the present invention can not only improve the quality of the image, but also reduce the need for a large amount of labeled data, speed up model training, and improve the accuracy and robustness of the final result.

[0071] Figure 2 A schematic diagram of a method for terahertz image super-resolution based on compressed sensing and transfer learning according to an example embodiment is shown.

[0072] See also Figure 2 , the figure shows a schematic diagram of a method for terahertz image super-resolution based on compressed sensing and transfer learning according to an example embodiment. First, a high-definition image data set is obtained and preprocessed. After that, data reconstruction is performed, important features are extracted from the high-definition image data set to obtain an important feature data set of the high-definition image data set, and the important feature data set is fitted using a Lasso regression model to obtain a fitting result for forming training data.

[0073] According to some embodiments, when the high-definition image is preprocessed, the resolution of the high-definition image is reduced by using a downsampling technique to simulate the resolution loss in terahertz imaging, thereby obtaining a sampled image; the downsampled image is interpolated by using a bicubic interpolation method to form a first low-resolution image. After finding the high-definition image, the high-definition image is preprocessed, downsampled on the high-definition image, and a low-resolution image is formed by a bicubic interpolation method. Downsampling is performed on the high-definition image to generate a corresponding first low-resolution image. Specifically, various downsampling techniques are used, such as average pooling, maximum pooling, etc., to reduce the resolution of the high-definition image, thereby simulating the common resolution loss in terahertz imaging. Afterwards, the downsampled image is interpolated by using a bicubic interpolation method to form a low-resolution image. Taking average pooling as an example, average pooling is a commonly used image downsampling technique, which is suitable for reducing the size of the image while retaining the main features of the original image as much as possible. Specifically, a pooling window is first defined, and a fixed-size pooling window 2x2 is selected. This window will traverse the entire image, and for each pixel in the pooling window, its average value is calculated. Replace all pixel values ​​in the original pooling window with the calculated average value. Slide the pooling window to the right by a certain step size (usually the same as the window size) and repeat the calculation until the width of the entire image is covered. Then, move the window downward and repeat this process until the height of the entire image is covered. The final image will be smaller than the original image, and the specific reduction ratio depends on the size of the pooling window. Calculate the scaling ratio of width and height based on the size difference between the original image and the downsampled image. For each pixel in the sampled image, find the corresponding coordinates in the original image, and use the bicubic interpolation formula to calculate the new pixel value based on the surrounding 16 pixels. Assign the calculated pixel value to the corresponding position of the new image. Finally, after completing the interpolation calculation of the entire image, output the new image (new low-resolution image) to form the first low-resolution image.

[0074] According to some embodiments, by applying the bicubic interpolation method, certain visual characteristics of an image can be maintained while reducing the resolution, so that the generated low-resolution image is visually similar to an actual low-resolution terahertz image.

[0075] According to some embodiments, when preprocessing the data set of the high-definition image, a transform domain is selected to sparsely represent the high-definition image; the sparsely represented high-definition image is randomly sampled using the Bernoulli matrix as a measurement matrix to obtain a compressed measurement vector; and a second low-resolution image is formed according to the measurement vector using matching pursuit. A transform domain is selected to sparsely represent the high-definition image, and a transform domain in which the imaging signal is sparse is determined, and the high-definition graphics are sparsely represented on the transform domain; the sparsely represented high-definition image is randomly sampled using the Bernoulli matrix as a measurement matrix to obtain a compressed measurement vector, and then the second low-resolution image is formed according to the measurement vector using matching pursuit. In compressed sensing technology, the complete process of two-dimensional image reconstruction involves obtaining the sparse representation of the image, designing an efficient measurement matrix, realizing compressed sampling of data, reconstructing the sparse signal, and post-processing and performance evaluation of the image. Specifically, in compressed sensing, it is first necessary to determine a transform domain that can make the signal sparse in the domain. Sparsity means that in the transform domain, most of the coefficients of the signal are zero or close to zero, while a few non-zero coefficients contain most of the signal information. The Haar wavelet transform is used to decompose the signal step by step, dividing it into high-frequency and low-frequency parts. In each level of decomposition, the signal is divided into two parts: approximate (low frequency) and detail (high frequency), and the approximate part continues to be decomposed. This hierarchical decomposition method is very suitable for image data, because the important information of the image is often concentrated in the lower frequency part, while the high frequency part contains detail information such as edges, but the number is far less than that of the low frequency part. Next, it is crucial to design an effective measurement matrix, which requires the matrix to satisfy the restricted isometry property (RIP) to ensure that the original signal can be reliably reconstructed from fewer measurements. Using the Bernoulli matrix as the measurement matrix is ​​an effective choice in compressed sensing. The Bernoulli matrix consists of random +1 and -1, and this type of matrix is ​​very suitable for implementing simple and efficient random sampling. The randomness and simplicity of the Bernoulli matrix reduce the number of necessary samples while maintaining the important features of the data. Taking the terahertz grayscale image with an original image size of 512x512 pixels as an example, the process of obtaining a low-resolution image based on compressed sensing is as follows: First, select the transform domain for sparse representation. Use the Haar wavelet transform to perform multi-level decomposition of the original image and divide the image into multiple frequency levels. This operation usually generates several different frequency bands, including a low-frequency sub-band (containing most of the image information) and multiple high-frequency sub-bands (containing edge and detail information). After that, create a 512x512 Bernoulli matrix in which each element independently takes a value of +1 or -1 with a probability of 0.5. After that, perform compressed sampling of the data, multiply the Bernoulli matrix with the sparsely represented image signal (the result after the Haar wavelet transform mentioned above) to obtain a compressed measurement vector.The length of this vector can be much smaller than the length of the original image data. Finally, the image is reconstructed using Matching Pursuit (MP).

[0076] According to some embodiments, finally, the low-resolution image is fitted using a Lasso regression model to determine the features of non-zero coefficients, the first low-resolution image and the second low-resolution image are fitted using a Lasso regression model, and the intersection of the features of non-zero coefficients is determined to form a training image. The importance of features of low-resolution images obtained by two different methods, compressed sensing and downsampling, and bicubic interpolation, are analyzed, and the first low-resolution image and the second low-resolution image are fitted using a Lasso regression model, and the intersection of the features of non-zero coefficients is determined to form a training image.

[0077] According to some embodiments, the important feature dataset of the high-definition image dataset may be standardized. After performing feature importance analysis on the low-resolution image dataset and obtaining the important feature dataset of the high-definition image dataset, the next step is to standardize these important feature datasets. Standardization ensures that all features are on the same scale to facilitate subsequent model training and optimization.

[0078] According to some embodiments, the image data is fitted using a Lasso regression model to determine the features of non-zero coefficients for data reconstruction. Specifically, the high-resolution image features are set as the target variable (Y), and the low-resolution image features are set as the explanatory variables (X). The data is fitted using a Lasso regression model, and cross-validation is used to determine the optimal regularization strength, thereby controlling the complexity of the model and avoiding overfitting. The features of the non-zero coefficients are analyzed through the Lasso regression results. These features are the most important in image reconstruction and have a significant impact on predicting high-resolution images.

[0079] Lasso (Least Absolute Shrinkage and Selection Operator) is a widely used sparse regression technique that introduces the L1 regularization term to solve the sparse solution. L1 loss represents the absolute value of the difference between the predicted value and the true value, also known as the least absolute deviation (LAD) and the absolute loss function (LAE). In general, it minimizes the sum of the absolute differences between the target value yi and the estimated value f(x;).

[0080]

[0081] Objective function: The optimization problem that Lasso tries to solve can be expressed as:

[0082]

[0083] Among them, y is the measurement value obtained from the original image, x is obtained through the measurement matrix Φ, "∥·∥2" represents the L2 norm, that is, the Euclidean distance, "∥·∥1" represents the L1 norm, that is, the regularization term that promotes sparsity, and λ is the regularization parameter that controls the strength of sparsity. L1 regularization is the core of Lasso, which tends to produce solutions with a few non-zero weights. This is because the L1 penalty tends to compress the coefficients of less important variables to 0. For images, this means that in the appropriate transform domain (i.e., the wavelet domain), only a few important coefficients are used to reconstruct the image, and most of the unimportant coefficients are set to zero.

[0084] The Lasso problem is usually solved by coordinate descent, gradient descent, or more advanced optimization algorithms such as ISTA (Iterative Shrinkage-Thresholding Algorithm). These methods iteratively update the value of x until the convergence condition is met. Not only does it help improve the prediction performance of the model, but it also helps us identify the most important features for image reconstruction. Ultimately, these important features can be used for further image processing tasks, such as super-resolution reconstruction.

[0085] According to some embodiments, the Stable Diffusion model is pre-trained on a color image dataset to learn texture, edge and high-frequency information; the pre-trained Stable Diffusion model is fine-tuned by transfer learning using the low-resolution image set used for training, thereby obtaining the pre-trained diffusion model fine-tuned by transfer learning. The Stable Diffusion model is pre-trained on a color image dataset to learn texture, edge and high-frequency information. And during the training process, the Stable Diffusion model is trained by gradually adding noise to the image and then restoring the original image. The choice of using the Stable Diffusion model is based on its excellent image generation ability. The model gradually builds high-quality images through a diffusion process and is particularly suitable for generating high-resolution outputs from low-resolution data. In the pre-training stage, first, the Stable Diffusion model is trained on a large-scale color image dataset, such as ImageNet. Doing so allows the model to learn rich image features, including texture, edge and high-frequency information. During this process, the model is trained to gradually add noise to the image, and then learns how to remove the noise to restore the original image. This "noise-denoising" training method allows the model to effectively master the key techniques of image reconstruction.

[0086] According to some embodiments, when a first image is obtained from a terahertz imaging system using compressed sensing technology, a transformation domain in which an imaging signal is sparse is determined, and the imaging signal is decomposed and represented on the transformation domain; the decomposed imaging signal is randomly sampled using a Bernoulli matrix as a measurement matrix to obtain the first image.

[0087] According to some embodiments, when data reconstruction is performed on the first image through compressed sensing technology, the image data is fitted using a Lasso regression model to determine features of non-zero coefficients for data reconstruction.

[0088] According to some embodiments, before super-resolution reconstruction of the second image is performed using a pre-trained diffusion model fine-tuned by transfer learning, the pre-trained Stable Diffusion model is fine-tuned by transfer learning according to the characteristics of the terahertz image. The pre-trained Stable Diffusion model is fine-tuned by transfer learning according to the characteristics of the terahertz image, and the Stable Diffusion model is fine-tuned using the low-resolution image set used for training, wherein the Stable Diffusion model is optimized by combining the regularized intensity loss and the perceptual loss function; the step size and noise level in the diffusion process of the Stable Diffusion model are adjusted so that the Stable Diffusion model adapts to the grayscale characteristics of the terahertz image during the denoising and reconstruction stages.

[0089] According to some embodiments, the pre-trained Stable Diffusion model needs to be fine-tuned for the characteristics of terahertz images. Due to its acquisition method and imaging characteristics, terahertz images are usually presented as grayscale images with low resolution and high noise. It is unrealistic to directly obtain high-quality terahertz image data for training, so it is necessary to adopt a transfer learning strategy to use a limited terahertz data set to specifically train the model.

[0090] According to some embodiments, the Stable Diffusion model is optimized based on the characteristics of the terahertz image by combining the regularized intensity loss and the perceptual loss function, and the step size and noise level in the diffusion process of the Stable Diffusion model are adjusted to ensure that the model can better adapt to the grayscale characteristics of the terahertz image during the denoising and reconstruction stages. During the fine-tuning process, the model is optimized by combining the L1 loss and the perceptual loss function. These loss functions help the model focus more on the restoration of image details and adapt to the higher noise and low contrast characteristics in the terahertz image. In addition, the step size and noise level in the diffusion process need to be adjusted to ensure that the model can better adapt to the grayscale characteristics of the terahertz image during the denoising and reconstruction stages.

[0091] According to some embodiments, after fine-tuning the transfer learning, the low-resolution terahertz image (features derived from Lasso feature importance analysis) will be input into the Stable Diffusion model, and the model will gradually generate a high-resolution image using a diffusion denoising process. This image reconstruction process is completed through the inverse diffusion process of the model, that is, a clear high-resolution image is gradually restored from the noisy image. This process not only enhances the edges and details of the image, but also improves the overall visual quality of the image. The generated image has clearer edges and richer details, which significantly improves the application value of terahertz images, such as their use in security inspections or medical diagnosis.

[0092] Figure 3 A low-resolution image is shown according to an experimental method of terahertz image super-resolution based on compressed sensing and transfer learning.

[0093] Figure 4 A high-resolution image is shown according to an experimental method of terahertz image super-resolution based on compressed sensing and transfer learning.

[0094] Figure 5 A LOSS fitting curve diagram of an experiment based on a terahertz image super-resolution method based on compressed sensing and transfer learning is shown.

[0095] See also Figure 3 , Figure 4 as well as Figure 5 , the low-resolution image obtained by applying terahertz imaging technology is used as the input image, and the method based on compressed sensing and transfer learning terahertz image super-resolution is used to process the low-resolution input image. First, important features are extracted from the low-resolution image, and the important feature data set is fitted using the Lasso regression model to obtain the fitting result, and the image super-resolution reconstruction is performed using the pre-trained Stablediffusion model of transfer learning. The loss curve of Stablediffusion is shown in Figure 5 shown.

[0096] Next, a quantitative analysis of the method designed by the scheme of the present invention is carried out by comparing multiple groups of data: 5 groups of low-resolution images obtained by using terahertz imaging technology and corresponding high-resolution images reconstructed by the method designed by the scheme of the present invention are taken for analysis and comparison. For specific data, see Table 1.

[0097] Table 1 PSNR and SSIM data comparison analysis table

[0098]

[0099] The data in Table 1 show that super-resolution processing significantly improves image quality, specifically in terms of PSNR and SSIM. For each image, the PSNR and SSIM after super-resolution have increased significantly, indicating that the overall quality and visual effect of the image have improved. The increase in PSNR indicates that the image error is reduced, while the increase in SSIM reflects the better preservation of the image structure, which is especially important for terahertz imaging, as it is often used in detail-rich application scenarios such as security detection or medical diagnosis.

[0100] According to some embodiments, the method designed by the present invention utilizes compressed sensing technology to recover high-quality images from data below the traditional Nyquist sampling rate, which greatly reduces the amount of data required in the data acquisition process, thereby reducing the bandwidth and storage requirements of the imaging device, so that efficient image acquisition can be performed even in resource-limited environments. By applying compressed sensing technology, the speed of data processing is significantly improved, which is particularly important for occasions that require fast imaging (such as real-time monitoring and rapid diagnosis).

[0101] According to some embodiments, the method designed by the scheme of the present invention uses the Lasso regression model to perform feature importance analysis. The Lasso regression model is used to select the most influential features from a large number of potential image features, which is crucial to improving the quality of super-resolution reconstruction. It not only provides a sparse solution, but also avoids overfitting through regularization, ensuring the generalization ability of the model. Traditional reconstruction technology often fails to achieve ideal results for terahertz image processing, because terahertz images are usually noisy. The method designed by the scheme of the present invention can effectively reduce noise and denoise during the diffusion process, thereby reliably and efficiently generating high-resolution images.

[0102] According to some embodiments, the method designed in the scheme of the present invention combines the transfer learning of the Stable Diffusion model. After pre-training on color images, it is adjusted to the terahertz image processing task through transfer learning, thereby improving the model's ability to process single-channel grayscale images, overcoming the difficulty of applying the model directly from color to grayscale, and realizing the effective transfer of model knowledge, so that the pre-trained model can better adapt to the characteristics of terahertz images.

[0103] Figure 6 A block diagram of a computing device according to an example embodiment of the invention is shown.

[0104] like Figure 6 As shown, computing device 30 includes processor 12 and memory 14. Computing device 30 may also include bus 22, network interface 16, and I / O interface 18. Processor 12, memory 14, network interface 16, and I / O interface 18 may communicate with each other via bus 22.

[0105] The processor 12 may include one or more general-purpose CPUs (Central Processing Units, processors), microprocessors, or application-specific integrated circuits, etc., for executing relevant program instructions. According to some embodiments, the computing device 30 may also include a high-performance graphics card (GPU) 20 for accelerating the processor 12.

[0106] The memory 14 may include a machine system readable medium in the form of a volatile memory, such as a random access memory (RAM), a read-only memory (ROM) and / or a cache memory. The memory 14 is used to store one or more programs including instructions and data. The processor 12 can read the instructions stored in the memory 14 to execute the above-mentioned method according to the embodiment of the present invention.

[0107] The computing device 30 may also communicate with one or more networks via the network interface 16. The network interface 16 may be a wireless network interface.

[0108] The bus 22 may include an address bus, a data bus, a control bus, etc. The bus 22 provides a path for exchanging information between components.

[0109] It should be noted that, in the specific implementation process, the computing device 30 may also include other components necessary for normal operation. In addition, those skilled in the art may understand that the above device may only include components necessary for implementing the embodiments of this specification, and need not include all components shown in the figure.

[0110] The present invention also provides a computer-readable storage medium having a computer program stored thereon, which implements the steps of the above method when executed by a processor. The computer-readable storage medium may include, but is not limited to, any type of disk, including a floppy disk, an optical disk, a DVD, a CD-ROM, a microdrive, and a magneto-optical disk, a ROM, a RAM, an EPROM, an EEPROM, a DRAM, a VRAM, a flash memory device, a magnetic card or an optical card, a nanosystem (including a molecular memory IC), a network storage device, a cloud storage device, or any type of medium or device suitable for storing instructions and / or data.

[0111] An embodiment of the present invention also provides a computer program product, which includes a non-transitory computer-readable storage medium storing a computer program, and the computer program is operable to cause a computer to execute part or all of the steps of any method recorded in the above method embodiments.

[0112] Those skilled in the art can clearly understand that the technical solution of the present invention can be implemented with the help of software and / or hardware. "Unit" and "module" in this specification refer to software and / or hardware that can independently complete or cooperate with other components to complete specific functions, where the hardware can be, for example, a field programmable gate array, an integrated circuit, etc.

[0113] It should be noted that, for the above-mentioned method embodiments, for the sake of simplicity, they are all described as a series of action combinations, but those skilled in the art should know that the present invention is not limited by the described action sequence, because according to the present invention, certain steps can be performed in other sequences or simultaneously. Secondly, those skilled in the art should also know that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present invention.

[0114] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0115] In the several embodiments provided by the present invention, it should be understood that the disclosed devices can be implemented in other ways. For example, the device embodiments described above are only schematic, such as the division of units, which is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some service interfaces, and the indirect coupling or communication connection of devices or units can be electrical or other forms.

[0116] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0117] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0118] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable memory. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art or the whole or part of the technical solution can be embodied in the form of a software product, which is stored in a memory and includes several instructions for a computer device (which can be a personal computer, server or network device, etc.) to perform all or part of the steps of the methods of various embodiments of the present invention.

[0119] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0120] The exemplary embodiments of the present invention are specifically shown and described above. It should be understood that the present invention is not limited to the detailed structures, configurations or implementations described herein; on the contrary, the present invention is intended to cover various modifications and equivalent configurations included in the spirit and scope of the appended clauses.

Claims

1. A method for super-resolution reconstruction of terahertz images, characterized in that: include: Obtaining a first image from a terahertz imaging system, wherein the first image has a resolution below a first threshold; Reconstructing the first image using compressed sensing technology to form a second image; The second image is super-reconstructed using a pre-trained diffusion model fine-tuned by transfer learning.

2. The method according to claim 1, characterized in that It also includes collecting and processing data for training the pre-trained diffusion model: Acquire a data set of high-definition images, wherein the high-definition images have a resolution higher than a second threshold; Preprocessing the high-definition image to form a low-resolution image, wherein the low-resolution image has a resolution lower than the first threshold; The low-resolution image is fitted using a Lasso regression model to determine the features of non-zero coefficients and form a training image set for training.

3. The method according to claim 2, characterized in that Preprocessing the high-definition image includes: Using a downsampling technique to reduce the resolution of the high-definition image to simulate the resolution loss in terahertz imaging, thereby obtaining a sampled image; The downsampled image is interpolated using a bicubic interpolation method to form a first low-resolution image.

4. The method according to claim 3, characterized in that: Preprocessing the data set of the high-definition image also includes: Selecting a transform domain to perform sparse representation on the high-definition image; The Bernoulli matrix is ​​used as the measurement matrix to randomly sample the sparsely represented high-definition image to obtain a compressed measurement vector; A second low-resolution image is formed based on the measurement vectors using matching pursuit.

5. The method according to claim 4, characterized in that The low-resolution image is fitted using a Lasso regression model to determine the characteristics of non-zero coefficients, including: The first low-resolution image and the second low-resolution image are respectively fitted using a Lasso regression model, and an intersection of features of non-zero coefficients is determined to form a training image.

6. The method according to claim 2, characterized in that Also includes: Pre-train the Stable Diffusion model on a color image dataset to learn texture, edge, and high-frequency information; The pre-trained Stable Diffusion model is fine-tuned by transfer learning using the low-resolution image set used for training, thereby obtaining the pre-trained diffusion model fine-tuned by transfer learning.

7. The method according to claim 1, characterized in that Obtaining a first image from a terahertz imaging system using a compressed sensing technique, including: Determine a transform domain in which the imaging signal is in a sparse state, and decompose and represent the imaging signal in the transform domain; The imaging signal after decomposition representation is randomly sampled using a Bernoulli matrix as a measurement matrix to obtain the first image.

8. The method according to claim 1, characterized in that: Reconstructing data of the first image by using a compressed sensing technology includes: The Lasso regression model is used to fit the image data and the features of non-zero coefficients are determined for data reconstruction.

9. A computer program product, characterized in that include: A computer program, wherein when the computer program is executed by a processor, the method according to any one of claims 1 to 8 is implemented.

10. A computing device, characterized in that include: processor; as well as A memory storing a computer program, which, when executed by the processor, enables the processor to perform the method according to any one of claims 1 to 8.

Images