A method for optimizing low-resolution inspection images in complex environments

Abstract

The present invention discloses a method for optimizing low-resolution inspection images in complex environments, and belongs to the field of image optimization technology. S100, image acquisition, uses a low-resolution camera with adaptive exposure and low noise characteristics installed on the inspection to perform dynamic inspection image acquisition in complex links, and the camera can automatically expose parameters according to the light of the link; S101, preprocessing, preprocessing the collected inspection images, including using a denoising algorithm based on deep learning to remove image noise, using an adaptive contrast increase system to improve the contrast of the image, and using a nonlinear change method to increase the detail information of the image. Through the image preprocessing step, the noise in the image is effectively removed, the contrast and grayscale transformation are enhanced, and the image is clearer. At the same time, the super-resolution reconstruction step uses a multi-frame low-resolution image sequence, and achieves a significant improvement in image resolution through signal processing or a deep learning algorithm, further improving the image quality.

Description

Technical Field

[0001] The invention relates to the technical field of image optimization, and in particular to a method for optimizing low-resolution inspection images in a complex environment. Background Art

[0002] Inspection work plays a vital role in many fields such as industrial production, safety monitoring, and environmental monitoring. Inspectors need to use inspection equipment to conduct regular inspections of specific areas to discover and deal with potential safety hazards or abnormal situations. However, in the actual inspection process, due to complex environments, poor lighting conditions, low camera resolution and other factors, the collected inspection images often have problems such as high noise, low contrast, and poor resolution, which brings great challenges to subsequent image analysis and target recognition.

[0003] Traditional image optimization methods mainly rely on image processing algorithms, such as filtering, denoising, contrast enhancement, etc., but these methods can only improve image quality to a certain extent, and have limited effect on image optimization in complex environments and low-resolution conditions. In addition, with the rapid development of deep learning technology, although some deep learning-based image super-resolution reconstruction and target recognition methods have been proposed, these methods usually need to be run on high-performance computing platforms and have high requirements on the quality and quantity of training data, which makes it difficult to meet the real-time and accuracy requirements of actual inspection work.

[0004] Therefore, it is of great practical significance and application value to develop an optimization method that can adapt to complex environments and effectively process low-resolution inspection images, improve the clarity and resolution of inspection images, and thus improve the accuracy and efficiency of target recognition. Summary of the invention

[0005] The purpose of the present invention is to provide a method for optimizing low-resolution inspection images in complex environments to solve the problems raised in the above-mentioned background technology.

[0006] To achieve the above object, the present invention provides the following technical solutions:

[0007] A method for optimizing low-resolution inspection images in a complex environment comprises the following steps:

[0008] S100, image acquisition, using a low-resolution camera with adaptive exposure and low noise characteristics installed on the inspection to perform dynamic inspection image acquisition in complex links. The camera can automatically set exposure parameters according to the light of the link;

[0009] S101, preprocessing, preprocessing the collected inspection images, including removing image noise by using a denoising algorithm based on deep learning, improving image contrast by using an adaptive contrast enhancement system, and increasing image detail information by using a nonlinear change method;

[0010] S102, super-resolution reconstruction, first based on the interpolation method, using known pixels to estimate the value of unknown pixels; secondly, based on the reconstruction method, by building a mathematical model of image degradation and solving it inversely to restore the high-resolution image; and using GAN to optimize image details, GAN consists of a generator and a discriminator. The generator tries to generate realistic high-resolution images, while the discriminator tries to distinguish between real images and generated images;

[0011] S103, feature extraction and optimization, feature extraction is performed on the reconstructed image, and feature learning is performed using a convolutional neural network based on an attention mechanism to extract key feature information from the image, and image features are optimized through feature fusion and feature enhancement techniques.

[0012] S104: Target recognition and classification: using a deep learning detection model to perform target recognition and classification on the optimized image.

[0013] As a further solution of the present invention: wherein, the deep learning denoising algorithm removes image noise using the following formula:

[0014]

[0015] Where I represents the input image; K represents the convolution kernel; O represents the output feature map; (i, j) represents the position in the output feature map; (m, n) represents the position in the convolution kernel. The features in the input image are extracted by sliding a small window called a convolution kernel (or filter). Each element in the convolution kernel is multiplied by the element at the corresponding position in the image, and then these convolution cameras are combined to obtain a value in the output feature map. This process is repeated at each position in the image until a complete output feature map is generated.

[0016] As a further solution of the present invention: wherein the adaptive contrast enhancement system includes the following method

[0017] Local contrast analysis: the system performs contrast analysis on each local area of ​​the image and calculates the brightness difference between each pixel or pixel block and the surrounding pixels.

[0018] Contrast adjustment: Based on the results of local contrast analysis, the system dynamically adjusts the contrast of each area. For areas with low contrast, the system will increase the contrast to make the details clearer; for areas with high contrast, the system will maintain or reduce the contrast to avoid distortion caused by excessive increase.

[0019] As a further solution of the present invention: wherein, the adaptive contrast enhancement system also includes adaptive threshold setting, which adaptively sets one or more thresholds through the overall brightness and contrast distribution of the image to distinguish between areas that need to be increased and areas that need to be maintained.

[0020] As a further solution of the present invention: wherein the nonlinear transformation method comprises the following steps

[0021] On the basis of the adaptive contrast enhancement system, the nonlinear grayscale transformation method is further adopted to change the mapping relationship between gray levels so that the grayscale distribution of the image is more uniform or conforms to the characteristic visual effect. For dark areas, the nonlinear grayscale transformation can stretch its grayscale value to a high range to increase the brightness. For bright areas, compression processing can be performed to reduce overexposure.

[0022] As a further solution of the present invention: wherein, the image acquisition step also includes a camera calibration method, including the following steps:

[0023] For radial distortion: the pixel coordinates after distortion (x distorted ,y distorted ) and the pixel coordinates before distortion (x undistorted ,y undistorted ) is represented by radial distortion coefficients k1, k2, and k3. For lenses with large distortion, k3 is required to represent it.

[0024] x distorted =x undistorted *(1+k1*r 2 +k2*r 4 +k3*r 6 )

[0025] y distorted =y undistorted *(1+k1*r 2 +k2*r 4 +k3*r 6 )

[0026] Where r is the pixel coordinate (x undistorted ,y undistorted ) is the distance from the center of the image;

[0027] For tangential distortion: the pixel coordinates after distortion (xcorrected ,y corrected ) and the pixel coordinates (x, y) before distortion and the relationship between the tangential distortion coefficients p1 and p2 is expressed as:

[0028] x corrected =x+[2*p1*y+p2*(r2)]

[0029] y corrected =y+[p1(r2)+2*p2*x

[0030] Here, r is also the distance between the pixel coordinate (x, y) and the center of the image.

[0031] Compared with the prior art, the present invention has the following beneficial effects:

[0032] Significantly improve image quality: Through the image preprocessing step, the noise in the image is effectively removed, the contrast and grayscale transformation are enhanced, making the image clearer and easier to analyze. At the same time, the super-resolution reconstruction step uses multiple frames of low-resolution image sequences and achieves a significant increase in image resolution through signal processing or deep learning algorithms, further improving image quality.

[0033] Improve target recognition accuracy: In the feature extraction and optimization steps, advanced image enhancement technology and deep learning models are used to extract and optimize the features of the reconstructed image, which significantly improves the model's ability to recognize targets in complex environments. This helps inspectors quickly and accurately discover and handle potential safety hazards or abnormal situations.

[0034] Enhanced model generalization capability: The present invention also includes the steps of training and validating the deep learning model, and by continuously optimizing the model parameters and structure, the recognition accuracy and generalization capability of the model are improved. This enables the method to adapt to the inspection task requirements in different scenarios and complex environments. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 It is a structural schematic diagram of the present invention; DETAILED DESCRIPTION

[0036] like Figure 1 As shown, a method for optimizing low-resolution inspection images in complex environments includes the following steps:

[0037] S100, image acquisition, using a low-resolution camera with adaptive exposure and low noise characteristics installed on the inspection to perform dynamic inspection image acquisition in complex links. The camera can automatically set exposure parameters according to the light of the link;

[0038] The image acquisition step further includes a camera calibration method, which includes the following steps:

[0039] For radial distortion: the pixel coordinates after distortion (x distorted ,y distorted ) and the pixel coordinates before distortion (x undistorted ,y undistorted ) is represented by radial distortion coefficients k1, k2, and k3. For lenses with large distortion, k3 is required to represent it.

[0040] x distorted =x undistorted *(1+k1*r 2 +k2*r 4 +k3*r 6 )

[0041] y distorted =y undistorted *(1+k1*r 2 +k2*r 4 +k3*r 6 )

[0042] Where r is the pixel coordinate (x undistorted ,y undistorted ) is the distance from the center of the image;

[0043] For tangential distortion: the pixel coordinates after distortion (x corrected ,y corrected ) and the pixel coordinates (x, y) before distortion and the relationship between the tangential distortion coefficients p1 and p2 is expressed as:

[0044] x corrected =x+[2*p1*y+p2*(r2)]

[0045] y corrected =y+[p1(r2)+2*p2*x

[0046] Here, r is also the distance between the pixel coordinate (x, y) and the center of the image.

[0047] In practical applications, it is necessary to obtain these distortion coefficients (k1, k2, k3, p1, p2) through camera calibration. The calibration process usually involves taking multiple photos of calibration patterns (such as a chessboard) with known size and layout, and using the feature points in these photos to calculate the distortion coefficients. Once these coefficients are obtained, they can be introduced into the above formula set to perform distortion correction on the captured image, thereby obtaining a more accurate and clear image.

[0048] S101, preprocessing, preprocessing the collected inspection images, including removing image noise by using a denoising algorithm based on deep learning, improving image contrast by using an adaptive contrast enhancement system, and increasing image detail information by using a nonlinear change method;

[0049] The deep learning denoising algorithm removes image noise using the following formula:

[0050]

[0051] Where I represents the input image; K represents the convolution kernel; O represents the output feature map; (i, j) represents the position in the output feature map; (m, n) represents the position in the convolution kernel. The features in the input image are extracted by sliding a small window called a convolution kernel (or filter). Each element in the convolution kernel is multiplied by the element at the corresponding position in the image, and then these convolution cameras are combined to obtain a value in the output feature map. This process is repeated at each position in the image until a complete output feature map is generated.

[0052] In this embodiment, assuming a 3*3 input image I and a 2*2 convolution kernel K,

[0053] Assume that the input image I is a 3*3 grayscale image containing noise

[0054] Suppose there is a simple 2*2 convolution kernel K, which is used to smooth the image (ie, denoise) This convolution kernel will calculate the average value of each 2*2 image patch it covers, thereby smoothing the image and reducing noise; Now, apply the convolution kernel K to the input image I to calculate an element of the output feature map O. Since the convolution kernel is 2x2, the output feature map O will be 2x2 (excluding boundary effects).

[0055] To calculate O(0,0), we place the convolution kernel K at the top left corner of the input image I and calculate the weighted sum:

[0056]

[0057] Similarly, we can calculate other elements of the output feature map O, but because the input image and the convolution kernel are very small, we can get a 2*2 output feature map (not considering padding and stride).

[0058] At the same time, the adaptive contrast enhancement system includes the following method

[0059] Local contrast analysis: the system performs contrast analysis on each local area of ​​the image and calculates the brightness difference between each pixel or pixel block and the surrounding pixels.

[0060] Contrast adjustment: Based on the results of local contrast analysis, the system dynamically adjusts the contrast of each area. For areas with low contrast, the system will increase the contrast to make the details clearer; for areas with high contrast, the system will maintain or reduce the contrast to avoid distortion caused by excessive increase.

[0061] In addition, the adaptive contrast enhancement system also includes adaptive threshold setting, which adaptively sets one or more thresholds through the overall brightness and contrast distribution of the image to distinguish the areas that need to be increased and maintained. After the contrast adjustment is completed, the image is post-processed, such as smoothing or edge enhancement, to further improve the quality of the image. Smoothing can reduce noise and artifacts in the image, making the image smoother and more natural. Edge enhancement can highlight the edges and details in the image, making the image clearer and easier to identify. Finally, the enhanced image is output. Compared with the original image, the enhanced image has better contrast and clarity, and the details are better displayed.

[0062] The nonlinear transformation method comprises the following steps:

[0063] On the basis of the adaptive contrast enhancement system, the nonlinear grayscale transformation method is further adopted to change the mapping relationship between grayscale levels to make the grayscale distribution of the image more uniform or in line with the characteristic visual effect. For dark areas, the nonlinear grayscale transformation can stretch its grayscale value to a high range.

[0064] This increases the brightness, and for bright areas, compression processing can be performed to reduce overexposure.

[0065] S102, super-resolution reconstruction, first based on the interpolation method, using known pixels to estimate the value of unknown pixels; secondly based on the reconstruction method, by building a mathematical model of image degradation and solving it inversely, to restore high-resolution images; specifically using convolutional neural networks (CNN) and generative adversarial networks (GAN), this method is used to train deep learning networks to learn the mapping relationship from low-resolution images to high-resolution images, where CNN extracts image features through multi-layer convolution operations and learns the mapping relationship between features to restore high-resolution images, and uses GAN to optimize image details. GAN consists of a generator and a discriminator. The generator tries to generate realistic high-resolution images, while the discriminator tries to distinguish between real images and generated images; the two make progress together in adversarial efforts to achieve a significant increase in image resolution. In GAN, the generator's loss function usually includes reconstruction loss (such as mean square error MSE) and adversarial loss (such as cross entropy loss). By minimizing these loss functions, an excellent generator can be trained.

[0066] Reconstruction loss formula (MSE):

[0067]

[0068] in, is the image generated by the generator, x i is the real image, and N is the number of images.

[0069] Adversarial loss formula (cross entropy):

[0070]

[0071] Among them, D is the discriminator, G is the generator, z is the random noise vector, is the real image distribution, P z(z) By minimizing this loss function, the discriminator learns to better distinguish between real data and fake data, while the generator tries to deceive the discriminator into thinking that the generated data is real. This competitive relationship drives the training process of GAN and enables the generator to generate more and more realistic data.

[0072] S103, feature extraction and optimization, for feature extraction after reconstruction, a convolutional neural network based on an attention mechanism is used for feature learning to extract key feature information in the image, and image features are optimized through feature fusion and feature enhancement techniques.

[0073] S104: Target recognition and classification: using a deep learning detection model to perform target recognition and classification on the optimized image.

[0074] What is described above is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can make equivalent replacements or changes according to the technical scheme and inventive concept of the present invention within the technical scope disclosed by the present invention, which should be covered by the protection scope of the present invention.

Claims

1. A method for optimizing low-resolution inspection images in complex environments, characterized in that: The steps include: S100, image acquisition, using a low-resolution camera with adaptive exposure and low noise characteristics installed on the inspection to perform dynamic inspection image acquisition in complex links. The camera can automatically set exposure parameters according to the light of the link; S101, preprocessing, preprocessing the collected inspection images, including removing image noise by using a denoising algorithm based on deep learning, improving image contrast by using an adaptive contrast enhancement system, and increasing image detail information by using a nonlinear change method; S102, super-resolution reconstruction, first based on the interpolation method, using known pixels to estimate the value of unknown pixels; secondly, based on the reconstruction method, by building a mathematical model of image degradation and solving it inversely to restore the high-resolution image; and using GAN to optimize image details, GAN consists of a generator and a discriminator. The generator tries to generate realistic high-resolution images, while the discriminator tries to distinguish between real images and generated images; S103, feature extraction and optimization, feature extraction is performed on the reconstructed image, and a convolutional neural network based on an attention mechanism is used for feature learning to extract key feature information in the image, and image features are optimized through feature fusion and feature enhancement techniques. S104: Target recognition and classification: using a deep learning detection model to perform target recognition and classification on the optimized image.

2. The method for optimizing low-resolution inspection images in complex environments according to claim 1, characterized in that: The deep learning denoising algorithm removes image noise using the following formula: Where I represents the input image; K represents the convolution kernel; O represents the output feature map; (i, j) represents the position in the output feature map; (m, n) represents the position in the convolution kernel. The features in the input image are extracted by sliding a small window called a convolution kernel (or filter). Each element in the convolution kernel is multiplied by the element at the corresponding position in the image, and then these convolution cameras are combined to obtain a value in the output feature map. This process is repeated at each position in the image until a complete output feature map is generated.

3. The method for optimizing low-resolution inspection images in complex environments according to claim 1, characterized in that: The adaptive contrast enhancement system includes the following method Local contrast analysis: the system performs contrast analysis on each local area of ​​the image and calculates the brightness difference between each pixel or pixel block and the brightness of surrounding pixels. Contrast adjustment: Based on the results of local contrast analysis, the system dynamically adjusts the contrast of each area. For areas with low contrast, the system will increase the contrast to make the details clearer; for areas with high contrast, the system will maintain or reduce the contrast to avoid distortion caused by excessive increase.

4. The method for optimizing low-resolution inspection images in complex environments according to claim 3 is characterized in that: The adaptive contrast enhancement system also includes adaptive threshold setting, which adaptively sets one or more thresholds according to the overall brightness and contrast distribution of the image to distinguish between areas that need to be enhanced and areas that need to be maintained.

5. The method for optimizing low-resolution inspection images in complex environments according to claim 1, characterized in that: The nonlinear transformation method comprises the following steps On the basis of the adaptive contrast enhancement system, the nonlinear grayscale transformation method is further adopted to change the mapping relationship between gray levels so that the grayscale distribution of the image is more uniform or conforms to the characteristic visual effect. For dark areas, the nonlinear grayscale transformation can stretch its grayscale value to a high range to increase the brightness. For bright areas, compression processing can be performed to reduce overexposure.

6. The method for optimizing low-resolution inspection images in complex environments according to claim 1, characterized in that: The image acquisition step also includes a camera calibration method, including the following steps: For radial distortion: the pixel coordinates after distortion (x distorted ,y distorted ) and the pixel coordinates before distortion (x undistorted ,y undistorted ) is represented by radial distortion coefficients k1, k2, and k3. For lenses with large distortion, k3 is required to represent it. x distorted =x undistorted *(1+k1*r 2 +k2*r 4 +k3*r 6 ) y distorted =y undistorted *(1+k1*r 2 +k2*r 4 +k3*r 6 ) Where r is the pixel coordinate (x undistorted ,y undistorted ) is the distance from the center of the image; For tangential distortion: the pixel coordinates after distortion (x corrected ,y correctedd ) and the pixel coordinates (x, y) before distortion and the relationship between the tangential distortion coefficients p1 and p2 is expressed as: x xorrected =x+[2*p1*y+p2*(r2)] y corrected =y+[p1(r2)+2*p2*x Here, r is also the distance between the pixel coordinate (x, y) and the center of the image.

Images