An underwater coral reef ecological monitoring image processing method and system
Through the combination of the minimum noise separation method and feature extraction network, the problem of low image fusion accuracy in the existing technology is solved, and high-precision image fusion of underwater coral reef ecological monitoring images is achieved, and the accuracy of judging the ecological situation of underwater coral reefs is improved. .
Patent Information
- Application Number
- CN202411139159.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-20
- Publication Date
- 2024-11-08
- Estimated Expiration
- 2044-08-20
AI Technical Summary
Existing image fusion technology is difficult to achieve high-precision fusion in underwater coral reef ecological monitoring, resulting in spectral distortion or spatial structure damage, affecting the accurate judgment of the ecological conditions of underwater coral reefs.
The hyperspectral image is dimensionality reduction and end element extraction by using the minimum noise separation method, combined with the feature extraction network for up-down sampling and feature fusion, and the hyperspectral demix results are fused with multispectral images by constructing a fusion model to generate high-precision underwater Coral reef ecological monitoring images.
It improves the accuracy of underwater coral reef ecological monitoring, achieves high-precision image fusion, and enhances the ability to clearly judge the ecological conditions of underwater coral reefs.
Smart Images

Figure CN118658034B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of image processing, and in particular to an underwater coral reef ecological monitoring image processing method and system. Background Art
[0002] Coral reefs are the most biodiverse marine ecosystems with a wide range of taxonomic diversity in a complex spatial habitat structure. Affected by climate change, coral diseases and various human activities, the global coral reef system is in rapid degradation, so it is very important to monitor and evaluate the health of coral reef ecosystems. Underwater image processing and target recognition are one of the key technologies for marine environmental monitoring. However, due to the complexity of the underwater environment and the refraction and scattering of water, underwater images are blurred and it is difficult to accurately capture the details of underwater coral reefs. With the continuous development of remote sensing technology, multispectral images and hyperspectral images are increasingly used in underwater monitoring. However, due to the differences in spectral resolution and spatial resolution between the two, image fusion technology has emerged in order to make full use of the advantages of these two images.
[0003] At present, commonly used image fusion technologies include the IHS transformation method, which converts the multispectral image from RGB space to IHS space, then replaces the I component with a component of the hyperspectral image, and finally converts it back to RGB space. This method is simple and intuitive, but it is easy to cause spectral distortion. Another method is the PCA transformation method, which performs principal component analysis on multispectral images and hyperspectral images, replaces the principal component of the hyperspectral image with the principal component of the multispectral image, and then performs an inverse transformation. This method can maintain spectral information to a certain extent, but it may damage the spatial structure of the image and the fusion accuracy is low, so as to clearly judge the ecological conditions of underwater coral reefs. Summary of the invention
[0004] In order to solve the above technical problems, the embodiments of the present invention provide an underwater coral reef ecological monitoring image processing method and system, which realizes high-precision fusion of underwater coral reef ecological monitoring images, thereby improving the accuracy of underwater coral reef ecological monitoring.
[0005] A first aspect of an embodiment of the present invention provides an underwater coral reef ecological monitoring image processing method, the method comprising:
[0006] Collect multispectral images and hyperspectral images of underwater coral reefs, and denoise the hyperspectral images and multispectral images to obtain denoised hyperspectral images and denoised multispectral images;
[0007] The denoised hyperspectral image is subjected to dimensionality reduction processing by using the minimum noise separation method to obtain a reduced-dimensional image, and endmember extraction is performed on the reduced-dimensional image to obtain an endmember extraction result. The abundance matrix is calculated based on the endmember extraction result to obtain a hyperspectral unmixing result.
[0008] The hyperspectral unmixing results and the denoised multispectral images are input into the constructed fusion model, so that the fusion model upsamples the hyperspectral unmixing results through the feature extraction network and downsamples the denoised multispectral images to obtain spatial sampling results and spectral sampling results. The spatial sampling results and spectral sampling results are input into the feature fusion network for feature fusion to obtain fusion results, so that personnel can judge the ecological conditions of underwater coral reefs based on the fusion results.
[0009] In a possible implementation of the first aspect, a dimensionality reduction process is performed on the denoised hyperspectral image using a minimum noise separation method to obtain a dimensionality reduced image, including:
[0010] Performing high-pass filtering on the signal data in the denoised hyperspectral image to obtain a noise matrix, and calculating the covariance matrices of the signal data and the noise matrix to obtain a first covariance matrix and a second covariance matrix;
[0011] Calculate the transformation matrix of the second covariance matrix, and obtain the third covariance matrix according to the transformation matrix and the signal matrix;
[0012] The third covariance matrix is used to obtain the eigenvector matrix, and calculations are performed based on the eigenvector matrix and the signal data to obtain a dimensionally reduced image.
[0013] In a possible implementation of the first aspect, endmember extraction is performed on the image after dimensionality reduction to obtain an endmember extraction result, and abundance inversion is performed according to the endmember extraction result to obtain a hyperspectral unmixing result, including:
[0014] According to the preset number of endmembers, the preset number of endmembers is selected from the image after dimensionality reduction to obtain an endmember matrix;
[0015] According to the endmember matrix, the volume is calculated using the volume calculation formula, and the pixels in the reduced-dimensional image are sequentially substituted into an endmember in the endmember matrix to obtain a new endmember matrix. The new volume is calculated according to the new endmember matrix, and the corresponding pixels when the new volume is greater than the preset value are selected as the new endmembers in the endmember matrix. This step is repeated until the new endmembers in all endmember matrices are determined to obtain the endmember extraction result, where the volume formula is:
[0016]
[0017]
[0018] In the formula, is the preset number of end members set, is the column vector of the nth end member;
[0019] The abundance matrix of the endmember extraction results is calculated using the least squares method to obtain the hyperspectral unmixing results.
[0020] In a possible implementation of the first aspect, the least squares method is used to perform abundance matrix calculation on the endmember extraction result to obtain a hyperspectral unmixing result, including:
[0021] The end member extraction results are calculated using the least squares calculation formula to obtain a non-restrictive least squares solution, where the calculation formula is:
[0022]
[0023]
[0024] Where M is the end member extraction result, is a pixel;
[0025] A control vector is obtained according to the non-restrictive least squares solution, and an initial abundance matrix is calculated according to the control vector. If the components in the abundance matrix are not all greater than 0, the components in the initial abundance matrix that are not greater than 0 are divided by the corresponding components in the control vector to obtain a division value. If the division value is greater than a preset value, the component corresponding to the division value in the endmember extraction result is deleted to obtain a new endmember extraction result. The abundance matrix is recalculated according to the new endmember extraction result to obtain a new abundance matrix. If the components in the new abundance matrix are all greater than 0, the new abundance matrix is output as a hyperspectral unmixing result.
[0026] In a possible implementation of the first aspect, upsampling the hyperspectral unmixing result through a feature extraction network and downsampling the denoised multispectral image to obtain a spatial sampling result and a spectral sampling result, including:
[0027] Performing upsampling and convolution operations on the hyperspectral unmixing result through a feature extraction network to obtain a first sampling result;
[0028] The denoised multispectral image is downsampled to obtain a second sampling result, and the second sampling result and the first sampling result are downsampled a preset number of times to obtain a spatial sampling result and a spectrum sampling result.
[0029] In a possible implementation manner of the first aspect, inputting the spatial sampling result and the spectrum sampling result into a feature fusion network for feature fusion to obtain a fusion result includes:
[0030] Performing a 3×3 convolution operation on the spatial sampling result and the spectrum sampling result respectively to obtain a first feature map and a second feature map;
[0031] The first feature map and the second feature map are weightedly fused according to a preset weight to obtain a fusion result.
[0032] A second aspect of an embodiment of the present invention provides an underwater coral reef ecological monitoring image processing system, the system comprising a collection module, an extraction module and a fusion module.
[0033] The acquisition module is used to acquire multispectral images and hyperspectral images of underwater coral reefs, and to denoise the hyperspectral images and multispectral images to obtain denoised hyperspectral images and denoised multispectral images;
[0034] The extraction module is used to perform dimensionality reduction processing on the denoised hyperspectral image using the minimum noise separation method to obtain a reduced-dimensional image, perform endmember extraction on the reduced-dimensional image to obtain an endmember extraction result, calculate the abundance matrix based on the endmember extraction result, and obtain a hyperspectral unmixing result;
[0035] The fusion module is used to input the hyperspectral unmixing results and the denoised multispectral images into the constructed fusion model, so that the fusion model upsamples the hyperspectral unmixing results through the feature extraction network and downsamples the denoised multispectral images to obtain spatial sampling results and spectral sampling results. The spatial sampling results and spectral sampling results are input into the feature fusion network for feature fusion to obtain fusion results, so that personnel can judge the ecological conditions of underwater coral reefs based on the fusion results.
[0036] In a possible implementation manner of the second aspect, the extraction module includes a filtering unit, a transformation unit, and a dimension reduction unit.
[0037] The filtering unit is used to perform high-pass filtering on the signal data in the denoised hyperspectral image to obtain a noise matrix, and respectively calculate the covariance matrix of the signal data and the noise matrix to obtain a first covariance matrix and a second covariance matrix;
[0038] The transform unit is used to calculate a transform matrix of the second covariance matrix, and obtain a third covariance matrix according to the transform matrix and the signal matrix;
[0039] The dimension reduction unit is used to obtain the eigenvector matrix using the third covariance matrix, and to perform calculations based on the eigenvector matrix and the signal data to obtain a dimensionally reduced image.
[0040] In a possible implementation manner of the second aspect, the extraction module further includes an endmember matrix calculation unit, an endmember extraction unit, and a calculation unit.
[0041] The endmember matrix calculation unit is used to select a preset number of endmembers from the image after dimensionality reduction according to the preset number of endmembers, and obtain an endmember matrix;
[0042] The endmember extraction unit is used to calculate the volume according to the endmember matrix using the volume calculation formula, and substitute the pixels in the reduced-dimensional image into an endmember in the endmember matrix in turn to obtain a new endmember matrix, calculate the new volume according to the new endmember matrix, and select the pixels corresponding to the new volume greater than the preset value as the new endmember in the endmember matrix. Repeat this step until the new endmembers in all endmember matrices are determined to obtain the endmember extraction result, where the volume formula is:
[0043]
[0044]
[0045] In the formula, is the preset number of end members set, is the column vector of the nth end member;
[0046] The calculation unit is used to calculate the abundance matrix of the end member extraction result by using the improved least square method to obtain the hyperspectral unmixing result.
[0047] In a possible implementation manner of the second aspect, the computing unit includes a first computing subunit and a second computing subunit.
[0048] The first calculation subunit is used to calculate the end member extraction result using the least squares calculation formula to obtain a non-restrictive least squares solution, wherein the calculation formula is:
[0049]
[0050]
[0051] Where M is the end member extraction result, is a pixel;
[0052] The second computing subunit is used to obtain a control vector according to the non-restrictive least squares solution, and to calculate an initial abundance matrix according to the control vector. If the components in the abundance matrix are not all greater than 0, the components in the initial abundance matrix that are not greater than 0 are divided by the corresponding components in the control vector to obtain a division value. If the division value is greater than a preset value, the components corresponding to the division value in the end member extraction result are deleted to obtain a new end member extraction result. The abundance matrix is recalculated according to the new end member extraction result to obtain a new abundance matrix. If the components in the new abundance matrix are all greater than 0, the new abundance matrix is output as a hyperspectral unmixing result.
[0053] The underwater coral reef ecological monitoring image processing method provided by the embodiment of the present invention collects multispectral images and hyperspectral images of underwater coral reefs, performs denoising on the hyperspectral images and multispectral images, obtains denoised hyperspectral images and denoised multispectral images, uses the minimum noise separation method to reduce the dimension of the denoised hyperspectral images, obtains the reduced dimension images, performs endmember extraction on the reduced dimension images, obtains the endmember extraction results, calculates the abundance matrix according to the endmember extraction results, obtains the hyperspectral unmixing results, inputs the hyperspectral unmixing results and the denoised multispectral images into the constructed fusion model, so that the fusion model upsamples the hyperspectral unmixing results through the feature extraction network, downsamples the denoised multispectral images, obtains the spatial sampling results and the spectral sampling results, inputs the spatial sampling results and the spectral sampling results into the feature fusion network for feature fusion, obtains the fusion results, so that personnel can judge the ecological conditions of underwater coral reefs according to the fusion results. Through the above method, high-precision fusion of underwater coral reef ecological monitoring images is achieved, thereby improving the accuracy of underwater coral reef ecological monitoring. BRIEF DESCRIPTION OF THE DRAWINGS
[0054] Figure 1 : A schematic diagram of a detection process of an embodiment of an underwater coral reef ecological monitoring image processing method provided by the present invention;
[0055] Figure 2 : A schematic diagram of the fusion process of an embodiment of the underwater coral reef ecological monitoring image processing method provided by the present invention;
[0056] Figure 3 : A system block diagram of an embodiment of the underwater coral reef ecological monitoring image processing method provided by the present invention. DETAILED DESCRIPTION
[0057] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0058] Please refer to Figure 1 , which is a flow chart of an embodiment of the underwater coral reef ecological monitoring image processing method provided by an embodiment of the present invention, including steps S101 to S103, each of which is specifically as follows:
[0059] S101, collecting a multispectral image and a hyperspectral image of an underwater coral reef, and performing denoising processing on the hyperspectral image and the multispectral image to obtain a denoised hyperspectral image and a denoised multispectral image.
[0060] In this embodiment, in the weather when the water surface is basically calm, the coral reefs are measured underwater using multispectral imaging sensors, drone hyperspectral remote sensing, and ship-borne water hyperspectral remote sensing to obtain multispectral images and hyperspectral images. Due to the complex environment of water and land, interference signals will be generated during the signal collection process. Therefore, it is necessary to use the wavelet transform method to denoise the hyperspectral image and the multispectral image to obtain the denoised hyperspectral image and the denoised multispectral image.
[0061] S102, using a minimum noise separation method to perform dimensionality reduction processing on the denoised hyperspectral image to obtain a dimensionality reduced image, performing endmember extraction on the dimensionality reduced image to obtain an endmember extraction result, calculating an abundance matrix based on the endmember extraction result, and obtaining a hyperspectral unmixing result.
[0062] In this embodiment, since processing hyperspectral images of hundreds of bands may increase the amount of calculation and affect the classification accuracy. Therefore, it is necessary to reduce the dimension of the hyperspectral image. MNF calculates the principal component that maximizes the signal-to-noise ratio, rather than the variance. The MNF transformation is effective in extracting principal components from noisy images. The principal component band is a band with obvious spectra and low correlation between bands. The denoised hyperspectral image is transformed using the minimum noise separation method to obtain a hyperspectral image after dimensionality reduction. Endmember extraction is then performed on the image after dimensionality reduction to obtain an endmember extraction result. The abundance matrix is calculated based on the endmember extraction result to obtain a hyperspectral unmixing result.
[0063] In some embodiments, step S102 of “using the minimum noise separation method to perform dimensionality reduction processing on the denoised hyperspectral image to obtain a dimensionality reduced image” includes but is not limited to the following steps:
[0064] Performing high-pass filtering on the signal data in the denoised hyperspectral image to obtain a noise matrix, and calculating the covariance matrices of the signal data and the noise matrix to obtain a first covariance matrix and a second covariance matrix;
[0065] Calculate the transformation matrix of the second covariance matrix, and obtain the third covariance matrix according to the transformation matrix and the signal matrix;
[0066] The third covariance matrix is used to obtain the eigenvector matrix, and calculations are performed based on the eigenvector matrix and the signal data to obtain a dimensionally reduced image.
[0067] In this embodiment, the high-pass filtering module is used to perform high-pass filtering on the signal data in the denoised hyperspectral image to obtain a noise matrix, and then the covariance matrix of the signal data in the hyperspectral image is calculated respectively. and the covariance matrix of the noise matrix , and then calculate the covariance matrix of the noise matrix The eigenvalues and eigenvectors of , since the eigenvectors form the matrix E, and the eigenvalues form the vector N, the transformation matrix is obtained by using the matrix E and the matrix N. .
[0068] The covariance matrix of the transformation matrix and the signal matrix is calculated, and then the covariance matrix is calculated to obtain the eigenvector matrix. The eigenvector matrix and the signal data are used for calculation to obtain the image after dimensionality reduction.
[0069] In some embodiments, in step S102, "performing endmember extraction on the reduced-dimensional image to obtain an endmember extraction result, and performing abundance inversion according to the endmember extraction result to obtain a hyperspectral unmixing result" includes but is not limited to the following steps:
[0070] According to the preset number of endmembers, the preset number of endmembers is selected from the image after dimensionality reduction to obtain an endmember matrix;
[0071] According to the endmember matrix, the volume is calculated using the volume calculation formula, and the pixels in the reduced-dimensional image are sequentially substituted into an endmember in the endmember matrix to obtain a new endmember matrix. The new volume is calculated according to the new endmember matrix, and the corresponding pixels when the new volume is greater than the preset value are selected as the new endmembers in the endmember matrix. This step is repeated until the new endmembers in all endmember matrices are determined to obtain the endmember extraction result, where the volume formula is:
[0072]
[0073]
[0074] In the formula, is the preset number of end members set, is the column vector of the nth end member;
[0075] The abundance matrix of the endmember extraction results is calculated using the least squares method to obtain the hyperspectral unmixing results.
[0076] In this embodiment, the number of end members X is set, X data are selected from the image after dimensionality reduction and the number of end members to obtain an x-dimensional vector, and then x data are randomly selected and added to the vector to obtain an end member matrix. The volume V is calculated using the volume formula and the end member matrix. The volume formula is:
[0077]
[0078]
[0079] In the formula, is the preset number of end members set, is the column vector of the nth end member;
[0080] Then, the data in the reduced-dimensional image are substituted into the position of an endmember in the endmember matrix in turn to obtain a new endmember matrix. The new volume is calculated based on the new endmember matrix. If the new volume is greater than 1, the data is selected as the new endmember. This step is repeated until the new endmembers in all endmember matrices are determined to obtain the endmember extraction result. The abundance matrix of the endmember extraction result is then calculated using the least squares method to obtain the hyperspectral unmixing result.
[0081] In some embodiments, the step S102 of "using the least squares method to perform abundance matrix calculation on the endmember extraction result to obtain a hyperspectral unmixing result" includes but is not limited to the following steps:
[0082] The end member extraction results are calculated using the least squares calculation formula to obtain a non-restrictive least squares solution, where the calculation formula is:
[0083]
[0084]
[0085] Where M is the end member extraction result, is a pixel;
[0086] A control vector is obtained according to the non-restrictive least squares solution, and an initial abundance matrix is calculated according to the control vector. If the components in the abundance matrix are not all greater than 0, the components in the initial abundance matrix that are not greater than 0 are divided by the corresponding components in the control vector to obtain a division value. If the division value is greater than a preset value, the component corresponding to the division value in the endmember extraction result is deleted to obtain a new endmember extraction result. The abundance matrix is recalculated according to the new endmember extraction result to obtain a new abundance matrix. If the components in the new abundance matrix are all greater than 0, the new abundance matrix is output as a hyperspectral unmixing result.
[0087] In this embodiment, the end member extraction result is calculated using the least squares calculation formula to obtain a non-restrictive least squares solution, wherein the calculation formula is:
[0088]
[0089]
[0090] Where M is the end member extraction result, is a pixel;
[0091] Then, we use the unrestricted least squares method to solve get Value and s, where
[0092]
[0093]
[0094] In the formula, is the unrestricted least squares solution, is an X-dimensional vector with component 1, and s represents the control vector.
[0095] The initial abundance matrix C is calculated based on the control vector. If the components in the abundance matrix are not all greater than 0, the components in the initial abundance matrix that are not greater than 0 are divided by the corresponding components in the control vector, that is, If the division value is greater than the preset value, the component corresponding to the division value in the endmember extraction result is deleted to obtain a new endmember extraction result. The abundance matrix is recalculated according to the new endmember extraction result to obtain a new abundance matrix. If the components in the new abundance matrix are all greater than 0, the new abundance matrix is output as the hyperspectral unmixing result.
[0096] S103, inputting the hyperspectral unmixing result and the denoised multispectral image into the constructed fusion model, so that the fusion model upsamples the hyperspectral unmixing result through the feature extraction network, and downsamples the denoised multispectral image to obtain spatial sampling results and spectral sampling results, and inputting the spatial sampling results and the spectral sampling results into the feature fusion network for feature fusion to obtain fusion results, so that personnel can judge the ecological conditions of underwater coral reefs according to the fusion results.
[0097] In this embodiment, if Figure 2 ,The fusion model constructed based on the convolutional neural network,the hyperspectral unmixing result and the denoised multispectral image are input into the constructed fusion model.,The fusion model includes a feature extraction network, a feature fusion network, and an output layer.
[0098] In the feature extraction network, the hyperspectral unmixing results are first upsampled, and the denoised multispectral image is downsampled to obtain spatial sampling results and spectral sampling results. The spatial sampling results and spectral sampling results are then input into the feature fusion network for feature fusion to obtain the fusion results. The staff judge the ecological conditions of the underwater coral reefs based on the fusion results.
[0099] In some embodiments, step S103 of “inputting the spatial sampling result and the spectrum sampling result into a feature fusion network for feature fusion to obtain a fusion result” includes but is not limited to the following steps:
[0100] Performing a 3×3 convolution operation on the spatial sampling result and the spectrum sampling result respectively to obtain a first feature map and a second feature map;
[0101] The first feature map and the second feature map are weightedly fused according to a preset weight to obtain a fusion result.
[0102] In this embodiment, a 3X3 convolution operation is performed on the spatial sampling results and the spectral sampling results respectively to obtain different feature maps, and then the feature maps are connected in series by sampling weighted fusion. The fused features are then mapped and output using a fully connected layer to obtain a fusion result.
[0103] It should be understood that although Figure 1 The steps in the flowchart are shown in sequence as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. Moreover, Figure 1 At least part of the steps may include multiple steps or multiple stages. These steps or stages are not necessarily performed at the same time, but can be performed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed in turn or alternately with other steps or at least part of the steps or stages in other steps.
[0104] In some embodiments, Figure 3 As shown, it shows a block diagram of an underwater coral reef ecological monitoring image processing system 300 provided in an embodiment of the present application, including: a collection module 301, an extraction module 302 and a fusion module 303, wherein:
[0105] The acquisition module 301 is used to acquire multispectral images and hyperspectral images of underwater coral reefs, and to perform denoising on the hyperspectral images and multispectral images to obtain denoised hyperspectral images and denoised multispectral images;
[0106] The extraction module 302 is used to perform dimensionality reduction processing on the denoised hyperspectral image using the minimum noise separation method to obtain a dimensionality reduced image, perform endmember extraction on the dimensionality reduced image to obtain an endmember extraction result, calculate the abundance matrix according to the endmember extraction result, and obtain a hyperspectral unmixing result;
[0107] The fusion module 303 is used to input the hyperspectral unmixing result and the denoised multispectral image into the constructed fusion model, so that the fusion model upsamples the hyperspectral unmixing result through the feature extraction network and downsamples the denoised multispectral image to obtain spatial sampling results and spectral sampling results, and input the spatial sampling results and the spectral sampling results into the feature fusion network for feature fusion to obtain the fusion result, so that personnel can judge the ecological conditions of underwater coral reefs according to the fusion result.
[0108] In some embodiments, the extraction module includes a filtering unit, a transformation unit and a dimensionality reduction unit.
[0109] The filtering unit is used to perform high-pass filtering on the signal data in the denoised hyperspectral image to obtain a noise matrix, and respectively calculate the covariance matrix of the signal data and the noise matrix to obtain a first covariance matrix and a second covariance matrix;
[0110] The transform unit is used to calculate a transform matrix of the second covariance matrix, and obtain a third covariance matrix according to the transform matrix and the signal matrix;
[0111] The dimension reduction unit is used to obtain an eigenvector matrix using the third covariance matrix, and perform calculations based on the eigenvector matrix and the signal data to obtain an image after dimensionality reduction.
[0112] In some embodiments, the extraction module further includes an endmember matrix calculation unit, an endmember extraction unit and a calculation unit.
[0113] The endmember matrix calculation unit is used to select a preset number of endmembers from the image after dimensionality reduction according to the preset number of endmembers, and obtain an endmember matrix;
[0114] The endmember extraction unit is used to calculate the volume according to the endmember matrix using the volume calculation formula, and substitute the pixels in the reduced-dimensional image into an endmember in the endmember matrix in turn to obtain a new endmember matrix, calculate the new volume according to the new endmember matrix, and select the pixels corresponding to the new volume greater than the preset value as the new endmember in the endmember matrix. Repeat this step until the new endmembers in all endmember matrices are determined to obtain the endmember extraction result, where the volume formula is:
[0115]
[0116]
[0117] In the formula, is the preset number of end members set, is the column vector of the nth end member;
[0118] The calculation unit is used to calculate the abundance matrix of the end member extraction result by using the least square method to obtain the hyperspectral unmixing result.
[0119] In some embodiments, the computing unit includes a first computing subunit and a second computing subunit.
[0120] The first calculation subunit is used to calculate the end member extraction result using the least squares calculation formula to obtain a non-restrictive least squares solution, wherein the calculation formula is:
[0121]
[0122]
[0123] Where M is the end member extraction result, is a pixel;
[0124] The second computing subunit is used to obtain a control vector according to a non-restrictive least squares solution, and to obtain an initial abundance matrix according to the control vector. If the components in the abundance matrix are not all greater than 0, the components in the initial abundance matrix that are not greater than 0 are divided by the corresponding components in the control vector to obtain a division value. If the division value is greater than a preset value, the components corresponding to the division value in the end member extraction result are deleted to obtain a new end member extraction result. The abundance matrix is recalculated according to the new end member extraction result to obtain a new abundance matrix. If the components in the new abundance matrix are all greater than 0, the new abundance matrix is output as a hyperspectral unmixing result.
[0125] The specific implementation of the underwater coral reef ecology monitoring image processing system is basically the same as the specific implementation of the underwater coral reef ecology monitoring image processing method described above, and will not be repeated here.
[0126] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).
[0127] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.
[0128] The specific embodiments described above further illustrate the purpose, technical solutions and beneficial effects of the present invention. It should be understood that the above description is only a specific embodiment of the present invention and is not intended to limit the scope of protection of the present invention. It is particularly pointed out that for those skilled in the art, any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention should be included in the scope of protection of the present invention.
Claims
1. An image processing method for underwater coral reef ecological monitoring, It is characterized in that include: Collecting a multispectral image and a hyperspectral image of an underwater coral reef, and performing denoising on the hyperspectral image and the multispectral image to obtain a denoised hyperspectral image and a denoised multispectral image; Performing dimensionality reduction processing on the denoised hyperspectral image using a minimum noise separation method to obtain a dimensionality reduced image, performing endmember extraction on the dimensionality reduced image to obtain an endmember extraction result, calculating an abundance matrix based on the endmember extraction result to obtain a hyperspectral unmixing result; The hyperspectral unmixing result and the denoised multispectral image are input into the constructed fusion model, so that the fusion model upsamples the hyperspectral unmixing result through a feature extraction network, and downsamples the denoised multispectral image to obtain a spatial sampling result and a spectrum sampling result, and the spatial sampling result and the spectrum sampling result are input into a feature fusion network for feature fusion to obtain a fusion result, so that personnel can judge the ecological conditions of underwater coral reefs according to the fusion result; The method of performing dimensionality reduction processing on the denoised hyperspectral image using the minimum noise separation method to obtain a dimensionality reduced image includes: Performing high-pass filtering on the signal data in the denoised hyperspectral image to obtain a noise matrix, and respectively calculating covariance matrices of the signal data and the noise matrix to obtain a first covariance matrix and a second covariance matrix; Calculating a transformation matrix of the second covariance matrix, and obtaining a third covariance matrix according to the transformation matrix and the signal data; Using the third covariance matrix to obtain an eigenvector matrix, and performing calculations based on the eigenvector matrix and the signal data to obtain a dimensionally reduced image; The step of performing endmember extraction on the image after dimension reduction to obtain an endmember extraction result, and performing abundance inversion according to the endmember extraction result to obtain a hyperspectral unmixing result includes: According to a preset number of endmembers, data of the preset number of endmembers are selected from the image after dimensionality reduction to obtain an endmember matrix; According to the endmember matrix, the volume is calculated using the volume calculation formula, the pixels in the reduced-dimensional image are sequentially substituted into an endmember in the endmember matrix to obtain a new endmember matrix, a new volume is calculated according to the new endmember matrix, and the pixels corresponding to the new volume when it is greater than a preset value are selected as new endmembers in the endmember matrix. This step is repeated until the new endmembers in all endmember matrices are determined to obtain an endmember extraction result, wherein the volume formula is: In the formula, is the preset number of end members set, is the column vector of the nth end member; Using the least squares method, the abundance matrix of the end member extraction result is calculated to obtain a hyperspectral unmixing result; The method of using the least squares method to perform abundance matrix calculation on the end member extraction result to obtain a hyperspectral unmixing result includes: The end member extraction results are calculated using the least squares calculation formula to obtain a non-restrictive least squares solution, wherein the calculation formula is: Where M is the end member extraction result, is a pixel; A control vector is obtained according to the non-restrictive least squares solution, and an initial abundance matrix is calculated according to the control vector. If the components in the abundance matrix are not all greater than 0, the components in the initial abundance matrix that are not greater than 0 are divided by the corresponding components in the control vector to obtain a division value. If the division value is greater than a preset value, the components corresponding to the division value in the end member extraction result are deleted to obtain a new end member extraction result. The abundance matrix is recalculated according to the new end member extraction result to obtain a new abundance matrix. If the components in the new abundance matrix are all greater than 0, the new abundance matrix is output as a hyperspectral unmixing result.
2. The underwater coral reef ecological monitoring image processing method according to claim 1, It is characterized in that The method of upsampling the hyperspectral unmixing result through the feature extraction network and downsampling the denoised multispectral image to obtain a spatial sampling result and a spectrum sampling result includes: Performing upsampling and convolution operations on the hyperspectral unmixing result through a feature extraction network to obtain a first sampling result; The denoised multispectral image is downsampled to obtain a second sampling result, and the second sampling result and the first sampling result are downsampled a preset number of times respectively to obtain a spatial sampling result and a spectral sampling result.
3. The underwater coral reef ecological monitoring image processing method according to claim 1, It is characterized in that The inputting the spatial sampling result and the spectrum sampling result into a feature fusion network to perform feature fusion to obtain a fusion result includes: Performing a 3×3 convolution operation on the spatial sampling result and the spectrum sampling result respectively to obtain a first feature map and a second feature map; The second feature map and the second feature map are weightedly fused according to a preset weight to obtain a fusion result.
4. An underwater coral reef ecological monitoring image processing system, It is characterized in that Including acquisition module, extraction module and fusion module, The acquisition module is used to acquire multispectral images and hyperspectral images of underwater coral reefs, and to perform denoising on the hyperspectral images and the multispectral images to obtain denoised hyperspectral images and denoised multispectral images; The extraction module is used to perform dimensionality reduction processing on the denoised hyperspectral image using a minimum noise separation method to obtain a dimensionality reduced image, perform endmember extraction on the dimensionality reduced image to obtain an endmember extraction result, calculate an abundance matrix based on the endmember extraction result, and obtain a hyperspectral unmixing result; The fusion module is used to input the hyperspectral unmixing result and the denoised multispectral image into the constructed fusion model, so that the fusion model upsamples the hyperspectral unmixing result through a feature extraction network, downsamples the denoised multispectral image, obtains a spatial sampling result and a spectrum sampling result, inputs the spatial sampling result and the spectrum sampling result into a feature fusion network for feature fusion, and obtains a fusion result, so that personnel can judge the ecological conditions of underwater coral reefs according to the fusion result; The extraction module includes a filtering unit, a transformation unit and a dimension reduction unit. The filtering unit is used to perform high-pass filtering on the signal data in the denoised hyperspectral image to obtain a noise matrix, and respectively calculate the covariance matrix of the signal data and the noise matrix to obtain a first covariance matrix and a second covariance matrix; The transform unit is used to calculate a transform matrix of the second covariance matrix, and obtain a third covariance matrix according to the transform matrix and the signal data; The dimension reduction unit is used to obtain an eigenvector matrix using the third covariance matrix, and to perform calculations based on the eigenvector matrix and the signal data to obtain an image after dimension reduction; The extraction module also includes an endmember matrix calculation unit, an endmember extraction unit and a calculation unit. The endmember matrix calculation unit is used to select the preset number of endmembers from the image after dimensionality reduction according to the preset number of endmembers, to obtain the endmember matrix; The endmember extraction unit is used to calculate the volume according to the endmember matrix using the volume calculation formula, substitute the pixels in the reduced-dimensional image into an endmember in the endmember matrix in turn to obtain a new endmember matrix, calculate the new volume according to the new endmember matrix, select the pixels corresponding to the new volume greater than the preset value as the new endmember in the endmember matrix, repeat this step until the new endmembers in all endmember matrices are determined, and obtain the endmember extraction result, wherein the volume formula is: In the formula, is the preset number of end members set, is the column vector of the nth end member; The calculation unit is used to perform abundance matrix calculation on the end member extraction result by using the least square method to obtain a hyperspectral unmixing result; The computing unit comprises a first computing subunit and a second computing subunit, The first calculation subunit is used to calculate the end member extraction result using the least squares calculation formula to obtain a non-restrictive least squares solution, wherein the calculation formula is: Where M is the end member extraction result, is a pixel; The second computing subunit is used to obtain a control vector according to the non-restrictive least squares solution, and to calculate an initial abundance matrix according to the control vector. If the components in the abundance matrix are not all greater than 0, the components in the initial abundance matrix that are not greater than 0 are divided by the corresponding components in the control vector to obtain a division value. If the division value is greater than a preset value, the components corresponding to the division value in the end member extraction result are deleted to obtain a new end member extraction result. The abundance matrix is recalculated according to the new end member extraction result to obtain a new abundance matrix. If the components in the new abundance matrix are all greater than 0, the new abundance matrix is output as a hyperspectral unmixing result.
