Hyperspectral image noise reduction method for aquaculture production scene

Abstract

The invention provides a hyperspectral image noise reduction method for an aquaculture production scene, belongs to the technical field of water quality image processing, realizes hyperspectral image noise reduction based on non-local statistical similarity and segmented spectrum processing, and mainly solves the problems that in the prior art, non-local image block similarity calculation is poor in noise fluctuation adaptability, and noise fluctuation is poor. The influence of the brightness difference of the hyperspectral image on noise reduction processing is reduced; and according to different noise levels of different wavebands, iteratively reducing noise according to the wavebands, and reducing the problem of image texture detail loss of partial wavebands caused by global noise reduction. The method comprises the following steps: inputting a hyperspectral image; estimating the initial noise level of each wave band; iterative noise reduction: projecting the hyperspectrum to a low-dimensional subspace; calculating non-local statistical similarity of the image blocks; noise reduction based on low-rank constraint is carried out; dimension raising reconstruction is carried out; updating the image and judging whether a stopping condition is met or not; and finally, outputting a noise-reduced hyperspectral image.

Description

Technical Field

[0001] The present invention relates to the technical field of water quality image processing, and in particular to a hyperspectral image denoising method for an aquaculture production scene. Background Art

[0002] With the development of science and technology and people's deepening understanding of things, modern agricultural development research requires more and more information. Among them, hyperspectral images (HIS) are currently widely used in the agricultural field. The hyperspectral images obtained by the hyperspectral imaging system have three dimensions of information, of which x and y contain spatial information and λ contains band information. Through the spectral information of dozens or even hundreds of bands, the information of agricultural production target objects can be accurately obtained, including the study of the growth of aquatic plants, pond water quality, etc., to achieve water quality judgment, water algae and physiological and biochemical index detection.

[0003] However, due to the complex agricultural production environment and the influence of sensor hardware performance, there is a lot of noise in agricultural remote sensing and near-ground hyperspectral images, which brings limitations and difficulties to agricultural application research. At present, many methods have been proposed for hyperspectral image denoising, mainly in the following three categories: ① Denoising method based on image space, each band of the hyperspectral image is treated as a two-dimensional grayscale image for separate denoising; ② Denoising method based on spectral low rank, using the correlation between spectral bands to denoise the image; ③ Combining image spatial correlation and spectral space low rank for denoising.

[0004] The paper "He W, Yao Q, Li C, et al. Non-local Meets Global: An Integrated Paradigm for Hyperspectral Image Restoration [J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2020." projects the hyperspectral image into a low-bit space, searches for non-local similar blocks on the average spectral image, and uses WNNM for denoising. However, the disadvantage of this method is that it does not consider the difference in noise levels in different bands, resulting in the loss of image details in some bands after denoising the hyperspectral image. At the same time, this method requires prior knowledge, and the optimization effect is limited.

[0005] The method proposed by Xidian University in its patent application "Hyperspectral Remote Sensing Image Denoising Method Based on Four-Dimensional Block Matching Filter" sorts hyperspectral images according to the spectral signal-to-noise ratio and processes band images with different noise levels differently. However, this method does not consider the correlation effect between spectra, that is, low rank, when grouping bands, resulting in the separation of related bands into different groups, affecting the denoising effect. At the same time, it introduces the performance error of high-pass filtering.

[0006] At the same time, various hyperspectral denoising methods, including the above two methods, mostly use the Euclidean distance similarity formula when searching for similar blocks in images. This formula is insensitive to the fluctuation of pixel size within the image block and can easily lead to misjudgment of similarity. Summary of the invention

[0007] The purpose of the present invention is to overcome the shortcomings of the prior art and propose a hyperspectral image denoising method based on non-local statistical similarity and segmented spectrum processing, so that the denoised hyperspectral image can better maintain image texture information, reduce noise interference, and facilitate agricultural science research. The present invention uses each pixel point in the non-local area of ​​the hyperspectral image as a statistical sample, compares the similarity of different non-local image blocks to find similar blocks, and performs denoising based on this. At the same time, according to the different noise levels in different bands, denoising iterative processing is performed on the sub-band to achieve denoising of the hyperspectral image data.

[0008] The technical means adopted by the present invention are as follows:

[0009] A method for denoising a hyperspectral image of an aquaculture production scene is provided, which realizes denoising of a hyperspectral image based on non-local statistical similarity and segmented spectrum processing, and comprises the following steps:

[0010] Step 1: Input hyperspectral image;

[0011] Step 2: Estimate the initial noise level of the hyperspectral image;

[0012] Step 3: Iterative noise reduction;

[0013] Step 4: Output the denoised hyperspectral image data.

[0014] Furthermore, the step 3 in the iterative noise reduction process includes the following steps:

[0015] Step 31: Project low-dimensional subspace; input hyperspectral image I 0 , the hyperspectral image I 0 H×W×N, respectively length×width×number of bands, construct a two-dimensional tensor T 0 (H×W)×N, where (H×W) represents a row vector, N represents a column vector, and through singular value decomposition, K< <N;

[0016] Will I 0 Projecting into the K-dimensional subspace to obtain the reduced dimensionality image I 1 , and get the orthogonal matrix E0, the following relationship exists:

[0017] I 0 =I 1 ×E0;

[0018] oeLh T E0=I;

[0019] Step 32: Calculate the non-local statistical similarity of image blocks: Given a non-local similarity block size of 5×5, 1 Divide the size into The image patch collection {p j}, fill in the boundary points; search the image block p through the K nearest neighbor algorithm i M image blocks with similar image distances are obtained by calculating the local statistical similarity and obtaining the calculated non-local similar image block set {p l}∈{p j};

[0020] Step 33: Noise reduction based on low-rank constraint; calculate {p j} and p i The two-dimensional tensor Te is obtained by combining the two-dimensional tensors with the size of (25×K)×L, where L is the number of similar image blocks, that is, {p l} capacity, representing a column vector; 25×K represents all band data of 25 pixels in an image block, representing a row vector; the tensor Te is noise corrected and denoised; all image blocks are processed to complete denoising;

[0021] Step 34: Dimensional reconstructing: Perform dimensional reconstructing on the denoised image to obtain I 2 , calculate I 2 The noise level of the current band is compared with the previous noise change level to confirm whether the band has completed noise reduction;

[0022] Step 35: Update iteration; update the hyperspectral image and enter iteration, and stop when the number of iterations is met.

[0023] Furthermore, the step 2 estimates the initial noise level of the hyperspectral image, and estimates the noise level of each band by the local variance method. The i-th dimension band image has a mean value of u i , with variance σ i 2 The noise n i .

[0024] Furthermore, in step 32, when filling in the boundary points, each band in the non-image area of ​​the boundary points is filled with 0, and when searching by the K nearest neighbor algorithm, the Euclidean distance of the center point of the image block is used as the distance benchmark.

[0025] Furthermore, it is characterized in that the calculation of the non-local statistical similarity in step 32 is performed as follows:

[0026] Step 321: Take image block p i And the collection of M neighborhood image patches {p j}, j = 1, 2, 3...M; average the images of K bands to obtain the average spectral image block for For all pixels, calculate the average spectral intensity value δ and Subtract the mean value δ from all pixels to get the average spectral image with a spectral intensity mean of 0

[0027] Step 322: Take image block p j , calculate the average spectral image And conduct The same operation is performed to obtain an average spectrum image with a spectral intensity mean of 0.

[0028] Step 323: For the average spectral image p i 'With the average spectral image Each pixel is taken as a sample and variance ratio analysis, i.e., F test, is performed to evaluate the difference in spectral characteristics of the two image blocks by comparing the variances of corresponding pixels in all bands in the two image blocks.

[0029] Furthermore, the noise correction and noise reduction process in step 33 performs the following operations:

[0030] Step 331: Perform SVD decomposition on the tensor Te as follows to obtain a singular value matrix ∑;

[0031] Te=U∑V T ;

[0032] Among them, ∑ represents a diagonal matrix, and the elements on the diagonal are the singular values ​​of Te;

[0033] Step 332: Perform normalization correction based on the global noise variance of the band and remove noise using a soft threshold method:

[0034]

[0035] in, represents the jth singular value of the i-th band, represents the global noise variance of the i-th band, and λ represents the threshold parameter.

[0036] Furthermore, the image is reconstructed by dimensionality increase in step 34 as follows: the denoised tensor is obtained after denoising:

[0037] Te′=U∑′V T ;

[0038] Among them, Te′ represents the tensor after denoising, ∑′ represents the singular value matrix after removing noise according to the soft threshold method;

[0039] By processing all image blocks and non-locally similar image blocks, the complete denoising of the hyperspectral image can be achieved to obtain the denoised low-dimensional hyperspectral image I 2 ; According to the orthogonal matrix E0 obtained in step 31, the high-dimensional hyperspectral image I is obtained by back-projection 0’ :

[0040] I 0’ =I 2 ×E0.

[0041] Furthermore, the noise level is calculated in step 34 using the formula:

[0042]

[0043] Among them, γ represents the scale scaling factor, ε1 and ε2 represent the minimum thresholds; σ′ represents the noise level after this iteration; σ″i represents the noise level after the previous iteration, and σ″i represents the noise level after the first iteration. i =σ i ; Δσ1 represents the noise level of the image after denoising, and Δσ2 represents the change level of noise during the previous and subsequent iterations; when Δσ1 or Δσ2 approaches 0, it means that the spectral band image has completed the denoising process and will not enter the next iteration.

[0044] Furthermore, the step 4 updates the hyperspectral image to I 0’ (H×W×(NQ)); Q represents the band that will not be subjected to the next iterative noise reduction.

[0045] Compared with the prior art, the present invention has the following advantages:

[0046] 1. The non-local similarity of image blocks is calculated by using a small sample statistical theory method (F test), which has better adaptability to noise fluctuations than the Euclidean distance. At the same time, this method can eliminate the impact of the brightness difference of similar image blocks on noise reduction.

[0047] 2. Combined with the low-rank property of the spectrum, the noise level and the noise reduction change level are compared in different bands to eliminate the problem of missing texture details in some bands of images caused by global noise reduction.

[0048] 3. In view of the different noise intensities and distributions in different bands of non-local similar blocks, noise reduction processing with different intensities in different bands is achieved by setting noise intensity adaptive noise reduction thresholds.

[0049] 4. The method of the present invention fully considers the different intensities of hyperspectral noise in different bands, and avoids the situation where the band noise cannot be completely eliminated when the overall noise level of the band is large. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0051] Figure 1 It is the overall flow chart of the present invention;

[0052] Figure 2 This is a schematic diagram illustrating the division of image blocks and the supplementation of boundary points in the present invention; wherein the red and blue frames are image blocks, the orange frame is the central pixel point of the image block, and the green word 0 is the supplementary pixel point.

[0053] Figure 3 The local images before and after noise reduction in a certain band. (a) is before noise reduction; (b) is after noise reduction. DETAILED DESCRIPTION

[0054] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0055] It should be noted that the terms "first", "second", etc. in the description, claims and above-mentioned drawings of the present invention are used to distinguish similar objects, and do not necessarily describe a specific order or sequence. It should be understood that such data used can be interchanged under appropriate circumstances, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "comprising" and "having" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device comprising a series of steps or units does not necessarily have to be limited to those steps or units clearly listed, but may include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0056] As Figure 1-3 shown, the present invention provides a method for denoising hyperspectral images in an aquaculture production scenario, which realizes hyperspectral image denoising based on non-local statistical similarity and segmented spectrum processing, and includes the following steps:

[0057] Step 1: Input the hyperspectral image.

[0058] Step 2: Estimate the initial noise level of the hyperspectral image; wherein, the mathematical model of the hyperspectral image is:

[0059]

[0060] Wherein, represents the true noisy image, represents the original image without noise, represents the noise (usually Gaussian noise). The noise level of each band image of the hyperspectral image is obtained by the local variance method. Among them, the i-th band image has a noise n i with a mean of u i 2 and a variance of σ i .

[0061] Step 3: Iterative denoising; specifically, it includes the following steps:

[0062] Step 31: Project the low-dimensional subspace; input the hyperspectral image I 0 (5460×8192×256, which are length×width×number of bands respectively), convert the image of each band of I 0 into a column vector, and then construct a two-dimensional tensor T 0 ((5460×8192)×256, where (5460×8192) is the row vector and 256 is the column vector), perform SVD decomposition on T 0 , and select the first K dimensions (K << N) with a contribution rate exceeding 95%. For I 0Projecting into the K-dimensional subspace to obtain the reduced dimensionality image I 1 , and get the orthogonal matrix E0, where

[0063] I 0 =I 1 ×E0,

[0064] oeLh T E0=I,

[0065] Step 32: Calculate the non-local statistical similarity of image blocks; given a non-local similarity block size of 5×5, 1 The image blocks are divided into a collection of (H / 5×W / 5×K) or (1092×1638×K) blocks. For the boundary points, the data points with intensity of 0 in each band are added when calculating the similarity to facilitate the calculation, as follows Figure 3 Based on the low rank of the image space, the Euclidean distance of the center point of the image block is calculated, and the K nearest neighbor algorithm is used to search for the M image blocks closest to each image block, and the non-local statistical similarity is calculated, as follows:

[0066] Step 321: Take image block p i And the collection of M neighborhood image patches {p j}, j = 1, 2, 3...M; average the images of K bands to obtain the average spectral image block for For all pixels, calculate the average spectral intensity value δ and Subtract the mean value δ from all pixels to get the average spectral image with a spectral intensity mean of 0

[0067] Step 322: Get p j , calculate the average spectral image And conduct The same operation is performed to obtain an average spectrum image with a spectral intensity mean of 0.

[0068] Step 323: For p i 'and Each pixel is taken as a sample and the F test is used to check whether there is a significant difference between the two image blocks. If not, the two image blocks are considered similar.

[0069] Step 33: Noise reduction based on low-rank constraint:

[0070] The calculated non-local similarity image block {p l}∈{p j} and p i Combine to get a two-dimensional tensor T 1, the size is ((25×K)×L), where L is the number of similar image blocks, that is, {p l}, representing a column vector; 25×K represents all band data of 25 pixels in an image block, representing a row vector.

[0071] For the calculated {p j} and p i The combination is performed to obtain a two-dimensional tensor Te with a size of ((25×K)×L), where L is the number of similar image blocks, i.e., the capacity of {pl}, representing a column vector; 25×K represents all band data of 25 pixels in an image block, representing a row vector. The tensor Te is denoised and the row SVD decomposition of the tensor Te is as follows:

[0072] Te=U∑V T

[0073] Since the noise intensity of each band image is different, and the noise level of different blocks in the same band image is also different, in order to avoid incomplete noise reduction of high-intensity noise bands or excessive noise reduction of low-intensity noise when using soft thresholding to avoid loss of details, the noise intensity is corrected to meet the objective function as follows:

[0074]

[0075] Among them, i represents the i-th band, λ is the soft threshold coefficient, is the regularization term, Tr(∑) is the trace of the singular value matrix Σ, σ i is the global noise variance of the i-th band. The soft threshold denoising is obtained as follows:

[0076]

[0077] in, is the jth singular value of the i-th band in the singular value matrix, i = 1, 2, 3 ... K, j = 1, 2, 3 ... 25. ε0 is the minimum threshold. Update all singular values ​​to obtain the matrix Σ′.

[0078] Step 34: Dimensional reconstruction; the tensor after denoising is Te′=U∑′V T , image patch p i and its non-locally similar image patch {p j By processing all image blocks and non-local similar image blocks, the denoised hyperspectral image I is obtained. 2 The orthogonal matrix E0 calculated in step 2.1 is used for I 2 Back-projection is performed to obtain a high-dimensional hyperspectral image I 0’ , the formula is as follows:

[0079] I0’ =I 2 ×E0;

[0080] Get the denoised hyperspectral image I 0’ , for I 0’ For each band, recalculate each noise level to get a variance of σ i ’2 The noise n i ', and calculate the iterative noise change, the formula is as follows:

[0081]

[0082] Among them, γ is the scale scaling factor, ε1 and ε2 are the minimum thresholds. σ″ i is the noise level of the previous iteration process, and σ″ in the first iteration i =σ i Δσ1 represents the noise level of the image after denoising, and Δσ2 represents the change level of noise during the previous and subsequent iterations. When Δσ1 or Δσ2 approaches 0, it means that the spectral band image has completed the denoising process and will not enter the next iteration.

[0083] Step 35: Update iteration; update the hyperspectral image to I 0” (8192×5460×(256-Q)). Q is the band that will not be subjected to the next iteration of denoising. Stop when the number of iterations is met.

[0084] Step 4: Output the denoised hyperspectral image data, such as Figure 3 Shown are local images before and after noise reduction in a certain band.

[0085] The serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages and disadvantages of the embodiments. In the above embodiments of the present invention, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, please refer to the relevant description of other embodiments. In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways.

[0086] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for denoising hyperspectral images of aquaculture production scenes, which is based on non-local statistical similarity and segmented spectrum processing to achieve denoising of hyperspectral images, characterized in that: The following steps are involved: Step 1: Input hyperspectral image; Step 2: estimating the initial noise level of the hyperspectral image; Step 3: Perform iterative noise reduction; Step 4: Output the denoised hyperspectral image data.

2. The method for denoising a hyperspectral image of an aquaculture production scene according to claim 1, characterized in that: The step 3 in the iterative noise reduction process includes the following steps: Step 31: Project low-dimensional subspace; input hyperspectral image I 0 , the hyperspectral image I 0 H×W×N, respectively length×width×number of bands, construct a two-dimensional tensor T 0 (H×W)×N, where (H×W) represents a row vector, N represents a column vector, and through singular value decomposition, K< <N; Will I 0 Projecting into the K-dimensional subspace to obtain the reduced dimensionality image I 1 , and get the orthogonal matrix E0, the following relationship exists: I 0 =I 1 ×E0; stE0 T E0=I; Step 32: Calculate the non-local statistical similarity of image blocks; given a non-local similarity block size of 5×5, 1 Divide the size into The image patch collection {p j }, fill in the boundary points; search the image block p through the K nearest neighbor algorithm i M image blocks with similar image distances are obtained by calculating the local statistical similarity and obtaining the calculated non-local similar image block set {p l }∈{p j }; Step 33: Noise reduction based on low-rank constraint; calculate {p j } and p i The two-dimensional tensor Te is obtained by combining the two-dimensional tensors with the size of (25×K)×L, where L is the number of similar image blocks, that is, {p l } capacity, representing a column vector; 25×K represents all band data of 25 pixels in an image block, representing a row vector; the tensor Te is noise corrected and denoised; all image blocks are processed to complete denoising; Step 34: Dimensional reconstructing: Perform dimensional reconstructing on the denoised image to obtain I 2 , calculate I 2 The noise level of the current band is compared with the previous noise change level to confirm whether the band has completed noise reduction; Step 35: Update iteration; update the hyperspectral image and enter iteration, and stop when the number of iterations is met.

3. The method for denoising a hyperspectral image of an aquaculture production scene according to claim 1, characterized in that: The step 2 estimates the initial noise level of the hyperspectral image, and estimates the noise level of each band by the local variance method. The i-th dimension band image has a mean value of u i , with variance σ i 2 The noise n i .

4. The method for denoising a hyperspectral image of an aquaculture production scene according to claim 2, characterized in that: In step 32, when filling in the boundary points, each band in the non-image area of ​​the boundary points is filled with 0, and when searching by the K nearest neighbor algorithm, the Euclidean distance of the center point of the image block is used as the distance benchmark.

5. A method for denoising a hyperspectral image of an aquaculture production scene according to any one of claims 2 or 4, characterized in that: The calculation of non-local statistical similarity in step 32 is performed as follows: Step 321: Take image block p i And the collection of M neighborhood image patches {p j }, j = 1, 2, 3...M; average the images of K bands to obtain the average spectral image block for For all pixels, calculate the average spectral intensity value δ and Subtract the mean value δ from all pixels to get the average spectral image with a spectral intensity mean of 0 Step 322: Take image block p j , calculate the average spectral image And conduct The same operation is performed to obtain an average spectrum image with a spectral intensity mean of 0. Step 323: For the average spectral image p i 'With the average spectral image Each pixel is taken as a sample and variance ratio analysis, i.e., F test, is performed to evaluate the difference in spectral characteristics of the two image blocks by comparing the variances of corresponding pixels in all bands in the two image blocks.

6. The method for denoising a hyperspectral image of an aquaculture production scene according to claim 2, characterized in that: The noise correction and noise reduction process in step 33 is performed as follows: Step 331: Perform SVD decomposition on the tensor Te as follows to obtain a singular value matrix ∑; The=U∑V T ; Among them, ∑ represents a diagonal matrix, and the elements on the diagonal are the singular values ​​of Te; Step 332: Perform normalization correction based on the global noise variance of the band and remove noise using a soft threshold method: in, represents the jth singular value of the i-th band, represents the global noise variance of the i-th band, and λ represents the threshold parameter.

7. The method for denoising a hyperspectral image of an aquaculture production scene according to claim 2, characterized in that: The image is reconstructed by increasing the dimension in step 34 as follows: the denoised tensor is obtained after denoising: Te′=U∑′V T ; Among them, Te′ represents the tensor after denoising, ∑′ represents the singular value matrix after removing noise according to the soft threshold method; By processing all image blocks and non-locally similar image blocks, the complete denoising of the hyperspectral image can be achieved to obtain the denoised low-dimensional hyperspectral image I 2 ; According to the orthogonal matrix E0 obtained in step 31, the high-dimensional hyperspectral image I is obtained by back-projection 0’ : I 0’ =I 2 ×E0。 8. The method for denoising a hyperspectral image of an aquaculture production scene according to claim 2 or 7, characterized in that: The noise level is calculated in step 34 using the formula: Among them, γ represents the scale scaling factor, ε1 and ε2 represent the minimum thresholds; σ′ represents the noise level after this iterative processing; σ″ i Indicates the noise level after the previous iteration. In the first iteration, σ″ i =σ i ; Δσ1 represents the noise level of the image after denoising, and Δσ2 represents the change level of noise during the previous and subsequent iterations; when Δσ1 or Δσ2 approaches 0, it means that the spectral band image has completed the denoising process and will not enter the next iteration.

9. The method for denoising a hyperspectral image of an aquaculture production scene according to claim 1, characterized in that: The step 4 updates the hyperspectral image to I 0’ (H×W×(NQ)); Q represents the band that will not be subjected to the next iterative noise reduction.

Images