A method for evaluating infrared simulated image quality based on autoencoder
Through the grid training and evaluation of infrared images by the autoencoder, the pairing data problem of infrared simulation images in the existing technology is solved, and the rapid and robustness evaluation of large-scale simulated infrared images is achieved, thus improving the improvement of the rapid and robustness of large-scale simulated infrared images. The system's recognition capabilities.
Patent Information
- Application Number
- CN202210501002.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-05-09
- Publication Date
- 2024-11-01
- Estimated Expiration
- 2042-05-09
AI Technical Summary
The prior art requires paired real data when evaluating the authenticity of infrared simulation images, and it is difficult to effectively evaluate the quality of large batches of unpaired simulated infrared samples, resulting in insufficient recognition capabilities of the system in various disturbing environments.
The real infrared image is grid-trained by an autoencoder, and the grid-translated data is trained until convergence. Then the simulated infrared image is grid-translated and the loss value of each grid is evaluated. The authenticity is carried out and the entire image is weighted and averaged to achieve the quality evaluation of simulated infrared images.
The authenticity of simulated infrared images can be effectively evaluated without pairing real data, and can quickly complete the evaluation of large-scale simulated infrared images. It has robustness in different sizes and interference environments, improving the system's recognition capabilities.
Smart Images

Figure CN114897822B_ABST
Abstract
Description
Technical Field
[0001] The present application belongs to the field of digital image processing technology, and relates to a method for performing quality assessment on infrared simulation images using an autoencoder, and more specifically, a new method for performing authenticity assessment on infrared simulation images using loss values in the encoding and reconstruction process of an autoencoder. Background Art
[0002] In physics, any substance above absolute zero (i.e. -273.15℃) can generate infrared rays, which are essentially electromagnetic waves. Human eyes cannot see infrared radiation, but infrared imaging systems are different from human eyes. The central component detector is sensitive to infrared radiation. Based on the intensity of infrared radiation from objects, infrared images visible to the human eye can be quantified and generated through the display. Infrared imaging systems have stronger penetration than visible light systems, longer detection distances, can work all day and all weather, are not affected by changes in light, and have strong target recognition capabilities in dark environments, strong anti-interference capabilities, and high flexibility. In particular, they play an important role in forest search and rescue, public security, military operations, and many other aspects.
[0003] In recent years, deep learning technology has developed rapidly. The excellent performance of deep neural networks also relies on massive amounts of data to a large extent. However, the number of infrared images in many scenes is actually very small, which makes it difficult to support the training of a model. To ensure that the task is completed well, more data must be obtained. With the continuous advancement of infrared imaging technology, it has developed rapidly. At present, the intelligent detection system for infrared is being upgraded and transformed. It is necessary to use intelligent perception models to accurately and efficiently identify and locate targets. It is necessary to consider the ability to accurately capture targets in various natural and artificial interference environments. This requires a large number of infrared training samples under different imaging conditions and different interference scenarios, and the system is tested and evaluated in large-scale different scenarios to ensure the robustness of the system. Since infrared detectors and other equipment are generally expensive, large-scale field acquisition is costly. Moreover, in most scenarios, it is very difficult to obtain infrared images through real shooting. For example, the acquisition of satellite infrared images and military target infrared images requires a lot of manpower and material resources. To address this problem, data augmentation and infrared simulation methods are often used to expand infrared samples. Data augmentation needs to be based on existing infrared samples, and it cannot change the content of the image itself, so the improvement on training is limited. Currently, sample augmentation based on infrared simulation has become a new trend, and there are some relatively mature methods that can generate simulated infrared samples in large quantities. However, the use of simulated infrared samples to expand infrared training sample sets still faces the quality evaluation of infrared sample simulation. The authenticity and availability of simulated infrared samples must be guaranteed before they can be used in network training. Existing simulation image evaluation methods are mostly based on pixel level, which requires perfectly paired real infrared data for evaluation. There is a lack of unpaired evaluation methods for large quantities of simulated infrared samples.
[0004] In the current research context, in order to address the problem of lack of paired real infrared samples in the quality assessment of simulated infrared images, a quality assessment method for simulated infrared images based on autoencoders is designed. This method can utilize the content of existing real infrared images and perform effective quality assessment on simulated infrared images without pairing. Summary of the invention
[0005] In view of the above research status and existing problems, the present invention provides an infrared simulation image quality assessment method based on autoencoder. By using the prior knowledge of existing real infrared images, when there is no paired real image for the simulated infrared image, the image is evenly divided into many grid images using a grid, and the grid images are sent to the autoencoder for training until convergence. Then, the simulated infrared image is gridded and sent to the trained autoencoder in the same way. The loss value of each grid obtained represents the authenticity of the grid, and the authenticity of the whole image is obtained by weighting the whole image.
[0006] In order to achieve the above-mentioned purpose of the invention, the infrared simulation image quality assessment method based on the autoencoder of the present invention comprises the following steps:
[0007] S1. Grid the real infrared image and uniformly sample it in the horizontal and vertical directions to obtain a real infrared image grid map as training data for the autoencoder;
[0008] S2, inputting the real infrared image grid map into the autoencoder, and training the real infrared image grid map until the loss value of the autoencoder converges;
[0009] S3, the simulated infrared image to be evaluated is subjected to the same gridding operation as the real infrared image and then input into the trained autoencoder, and a reconstructed image of each simulated infrared image grid map is output, and the loss value of the simulated infrared image grid map and the corresponding reconstructed image is calculated, so as to calculate the average value of the loss of all simulated infrared image grid maps corresponding to each simulated infrared image;
[0010] S4. According to a preset score threshold, all simulated infrared image loss averages are screened to obtain a highly realistic simulated image.
[0011] Preferably, the S1 specifically includes:
[0012] S11. Define the real infrared image set Simulated infrared image set W1×H1 and W2×H2 are the sizes of the real infrared image and the simulated infrared image, respectively. N and M represent the number of real infrared images and simulated infrared images, respectively. C represents the number of channels of the image. n represents the nth real infrared image, F m represents the mth simulated infrared image;
[0013] S12, grid division definition, the sliding step lengths in the real infrared image and the simulated infrared image are S1 and S2 respectively, the size of the grid is w×h, adjacent grids are allowed to overlap, the real infrared image obtains W1 / S1×H1 / S1 grid graphs, and the simulated infrared image obtains W2 / S2×H2 / S2 network graphs;
[0014] The real infrared image grid map obtained by cutting the real infrared image according to the above parameters is used as the training data of the autoencoder;
[0015] S13, numbering the real infrared image grid map, The grid data is defined as Represents Z n The grid diagram of the i-th row and j-th column is formed, i≤W1 / S1, j≤H1 / S1, and the training set of the autoencoder is expressed as
[0016] Preferably, S2 specifically includes:
[0017] S21. Define the autoencoder: The encoder is defined as E(·), and the decoder is defined as D(·). The formula is as follows:
[0018]
[0019] Among them, x represents the input of the autoencoder, θ D and θ F are the parameters of the encoder and decoder respectively, y is the output of the middle layer, is the output of the decoder;
[0020] S22. Define the loss function, the formula is as follows:
[0021]
[0022] Among them, B is the number of training data, the training data set The input is sent to the autoencoder, and the autoencoder is optimized by an optimization algorithm, such as the momentum gradient descent algorithm or the Adam algorithm, so that the loss function value becomes smaller until convergence.
[0023] Preferably, S3 specifically includes:
[0024] S31, according to the parameters set in S1, each simulated infrared image to be evaluated is cut to obtain a simulated infrared image grid map, the simulated infrared image grid maps are numbered, and The grid data is defined as i≤W2 / S2, j≤H2 / S2, get the simulated infrared image grid map set The simulated infrared image grid map set is input into the trained autoencoder, and the autoencoder will output the reconstructed map. The mean square error is used to calculate the original input simulated infrared image grid map. And the corresponding reconstruction The reconstruction loss value So we get the score set
[0025] S32. From the score set Take out the grid loss score of each simulated infrared image, and average all grid loss scores to get image F m Score S m , the calculation formula is:
[0026]
[0027] Where i represents Fm The i-th row, j represents F m The jth column.
[0028] Preferably, the S4 specifically includes:
[0029] S41, S m represents the authenticity score of the mth simulated infrared image, and the score set {S m The elements of} are sorted from small to large. The lower the score, the stronger the authenticity of the image, and the higher the score, the weaker the authenticity of the image.
[0030] S42. Screening the average loss values of all simulated infrared images using a preset score threshold to obtain a highly realistic simulated image.
[0031] The infrared simulation image quality assessment method based on autoencoder designed in the present invention has the following advantages:
[0032] 1. No need to pair truth values.
[0033] 2. The authenticity of simulated infrared images of different sizes can be evaluated simultaneously.
[0034] 3. Rapidly complete large-scale simulated infrared image evaluation.
[0035] 4. Utilize the powerful fitting ability of neural networks to learn the deep features and domain features of real infrared images. BRIEF DESCRIPTION OF THE DRAWINGS
[0036] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the prior art descriptions are briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention, and for ordinary technicians in this field, other drawings can be obtained based on the provided drawings without creative work.
[0037] Figure 1 It is a flow chart of the infrared simulation image quality assessment method based on the autoencoder described in the present invention;
[0038] Figure 2 It is a schematic diagram of the grid graphs of the real infrared image and the simulated infrared image embedded in the middle layer space;
[0039] in, Figure 2 (a) The real grid map falls within the feature space range of the solid line, and the simulated infrared grid map falls within the real feature space range and does not fall within the real space range.
[0040] Figure 2(b) The farther the simulated infrared grid map is from the real infrared feature space range, the worse the authenticity. For example, if it falls within the feature range, the authenticity is the best. If it falls on the edge of the real feature, the authenticity becomes lower. The authenticity is poor near the outside of the space, and the authenticity is even worse if it is farther away. DETAILED DESCRIPTION
[0041] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0042] The following is combined with Figure 1 The embodiments of the present invention are described in detail.
[0043] Please see attached Figure 1 , is a flow chart of an implementation method of an infrared simulation image quality assessment method based on an autoencoder of the present invention. In this implementation method, the following steps are included:
[0044] S1. Grid the existing small amount of real infrared images, uniformly sample them in the horizontal and vertical directions, and the obtained real infrared image grid blocks constitute the training data of the autoencoder.
[0045] S2. Design an autoencoder according to the size of the grid map, including the design of the number of neural network layers, the optimization function used in training, the loss function and other parameters. The present invention does not limit the specific structure of the autoencoder. The real infrared image grid map is trained until the loss value converges.
[0046] S3. The simulated infrared image to be evaluated is gridded in the same way as the real infrared image and then input into the trained autoencoder. The network will output the reconstructed image of each simulated infrared image grid map. The loss value between the original input simulated infrared image grid map and the corresponding reconstructed image represents the authenticity of the grid. All grid map losses are restored to the corresponding simulated infrared image to obtain the loss of the entire simulated infrared image to be evaluated. The average of these loss values represents the authenticity of the simulated infrared image.
[0047] S4. The relative size value of the obtained simulated infrared authenticity is used as a quality evaluation standard and can be used for quality screening.
[0048] It should be noted that the present invention does not specifically limit the execution order of steps S1-S4, and there is no strict execution order, wherein the real infrared image and the simulated infrared image to be evaluated can be gridded at the same time.
[0049] Furthermore, S1 specifically includes:
[0050] S11. Define the real infrared image set Simulated infrared image set W1×H1 and W2×H2 are the sizes (width and height) of the real infrared image and the simulated infrared image, respectively. N and M represent the number of real infrared images and simulated infrared images, respectively. C represents the number of channels of the image. n represents the nth real infrared image, F m Represents the mth simulated infrared image.
[0051] S12. Grid division definition. The sliding step sizes in the real infrared image and the simulated infrared image are S1 and S2 respectively. The size of the grid is w×h. Adjacent grids are allowed to overlap. The real infrared image can obtain W1 / S1×H1 / S1 grid blocks, and the simulated infrared image can obtain W2 / S2×H2 / S2 network blocks. The real infrared image is cut according to the above parameters, and the obtained real infrared image grid map will become the training data for the subsequent autoencoder.
[0052] S13, numbering the grid diagrams of the real infrared image and the simulated infrared image, from The grid data is defined as Represents Z n The grid diagram of the i-th row and j-th column is formed, i≤W1 / S1, j≤H1 / S1, and the training set of the autoencoder is expressed as The same comes from The grid data is defined as i≤W2 / S2, j≤H2 / S2, forming the evaluation set of the autoencoder
[0053] S2 specifically includes:
[0054] S21. Define an autoencoder that can flatten each pixel of the grid map and input it into a fully connected neural network, or it can be used as an image and fed into a 2D convolutional network. The encoder is defined as E(·) and the decoder is defined as D(·). The formula is as follows:
[0055]
[0056] Where x refers to the input of the autoencoder, that is, the set and θ D and θ F are the parameters of the encoder and decoder respectively, y is the output of the middle layer, is the output of the decoder.
[0057] S22. Define the loss function, the formula is as follows:
[0058]
[0059] Where B is the number of data in a batch, and the loss function is also the mean square error loss function. Then the momentum gradient descent algorithm is used to optimize the autoencoder so that the loss function value becomes smaller until convergence. The training data set used in this part
[0060] S3 specifically includes:
[0061] S31, according to the parameters set in S1, each simulated infrared image to be evaluated is cut to obtain a set of simulated infrared image grid images The set is input into the trained autoencoder, which will output the reconstructed image. The reconstruction loss value of the original input simulated infrared image grid map and the corresponding reconstructed image is calculated using the mean square error to obtain the score set. Specifically refers to
[0062] S32, from Take out the grid loss score of each simulated infrared image, and average all grid loss scores to get the image score S m , the calculation formula is
[0063]
[0064] S4 specifically includes:
[0065] S41, S m represents the authenticity score of the mth simulated infrared image, and the score set {S m The elements of} are sorted from small to large. The lower the score, the stronger the authenticity of the image, and the higher the score, the weaker the authenticity of the image.
[0066] S42. By setting a score threshold, the average loss values of all simulated infrared images can be screened to obtain a batch of highly realistic simulated images.
[0067] The above is a detailed introduction to the infrared simulation image quality assessment method based on autoencoder provided by the present invention. This article uses specific examples to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea; at the same time, for general technical personnel in this field, according to the idea of the present invention, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present invention.
[0068] In this article, relational terms such as first and second, etc. are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "comprise", "include" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the statement "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.
Claims
1. A method for evaluating infrared simulated image quality based on an autoencoder, characterized in that: The steps include: Step 1: grid the real infrared image, uniformly sample it in the horizontal and vertical directions, and obtain a real infrared image grid map as training data for the autoencoder; Step 2: Input the real infrared image grid map into the autoencoder, and train the real infrared image grid map until the autoencoder loss value converges; Step 3: After the simulated infrared image to be evaluated is gridded in the same way as the real infrared image, it is input into the trained autoencoder, and the reconstructed image of each simulated infrared image grid map is output. The loss value of the simulated infrared image grid map and the corresponding reconstructed image is calculated, so as to calculate the average value of the loss of all simulated infrared image grid maps corresponding to each simulated infrared image; Step 4: According to a preset score threshold, the average loss values of all simulated infrared images are screened to obtain a highly realistic simulated image.
2. The infrared simulation image quality assessment method based on autoencoder according to claim 1 is characterized in that: Gridding infrared images specifically includes: S11. Define the real infrared image set Simulated infrared image set W1×H1 and W2×H2 are the sizes of the real infrared image and the simulated infrared image, respectively. N and M represent the number of real infrared images and simulated infrared images, respectively. C represents the number of channels of the image. represents the nth real infrared image, F m represents the mth simulated infrared image; S12, grid division definition, the sliding step lengths in the real infrared image and the simulated infrared image are S1 and S2 respectively, the size of the grid is w×h, adjacent grids are allowed to overlap, the real infrared image obtains W1 / S1×H1 / S1 grid maps, and the simulated infrared image obtains W2 / S2×H2 / S2 grid maps; The real infrared image grid map obtained by dividing and cutting the real infrared image is used as the training data of the autoencoder; S13, numbering the real infrared image grid map, The grid data is defined as Represents Z n The grid diagram of the i-th row and j-th column is formed, i≤W1 / S1, j≤H1 / S1, and the training set of the autoencoder is expressed as 3. The infrared simulation image quality assessment method based on autoencoder according to claim 2 is characterized in that: The step 2 specifically includes: S21. Define the autoencoder: The encoder is defined as E(·), and the decoder is defined as D(·). The formula is as follows: Among them, x represents the input of the autoencoder, θ D and θ F are the parameters of the encoder and decoder respectively, y is the output of the middle layer, is the output of the decoder; S22. Define the loss function, the formula is as follows: Among them, B is the number of training data, the training data set The input is sent to the autoencoder, and the autoencoder is optimized by the optimization algorithm so that the loss function value becomes smaller until convergence.
4. The infrared simulation image quality assessment method based on autoencoder according to claim 2 is characterized in that: The step 3 specifically includes: S31, dividing and cutting each simulated infrared image to be evaluated to obtain a simulated infrared image grid map, numbering the simulated infrared image grid maps, and The grid data is defined as Get the simulated infrared image grid map collection The simulated infrared image grid map set is input into the trained autoencoder, and the autoencoder will output the reconstructed map. The mean square error is used to calculate the original input simulated infrared image grid map. And the corresponding reconstruction The reconstruction loss value So we get the score set S32. From the score set Take out the grid loss score of each simulated infrared image, and average all grid loss scores to get image F m Score S m , the calculation formula is: Where i represents F m The i-th row, j represents F m The jth column.
5. The infrared simulation image quality assessment method based on autoencoder according to claim 4 is characterized in that: The step 4 specifically includes: S41, S m represents the authenticity score of the mth simulated infrared image, and the score set {S m The elements of} are sorted from small to large. The lower the score, the stronger the authenticity of the image, and the higher the score, the weaker the authenticity of the image. S42. Screening the average loss values of all simulated infrared images using a preset score threshold to obtain a highly realistic simulated image.
