Image correction method and system for underwater swing-scanning hyperspectral camera
Through auxiliary positioning and sonar information, combined with the feature matching and finite element analysis of the wide field of view RGB correction camera, the problem of pixel distortion of high-spectral cameras in complex underwater terrain is solved, and high-quality underwater image correction is achieved.
Patent Information
- Application Number
- CN202111393494.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-11-23
- Publication Date
- 2024-11-22
- Estimated Expiration
- 2041-11-23
AI Technical Summary
When existing linear array pendulum sweep hyperspectral cameras use traditional geometric correction methods in complex underwater terrain, it is difficult to accurately correct pixel distortion, resulting in image distortion.
The auxiliary positioning device and sonar obtain the geographical location, attitude angle and target distance information of the underwater sweep hyperspectral camera, combined with the wide field of view RGB correction camera, feature matching and finite element analysis are performed, self-corrected RGB images are generated and displacement matrix is solved. , correct the original hyperspectral image.
Improve the imaging quality of the underwater sweep hyperspectral camera, ensure the accuracy of the image target size and the shape of the real object, and reduce cell distortion.
Smart Images

Figure CN114092350B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of image processing technology, and in particular to an image correction method and system for an underwater swing-scanning hyperspectral camera. Background Art
[0002] Underwater images are one of the means for humans to obtain effective underwater information. Hyperspectral imaging systems can be miniaturized and can obtain spectral information of target objects, making hyperspectral imaging the main means of obtaining underwater target information.
[0003] Existing linear array swing-scan hyperspectral cameras mainly use geometric correction methods to correct pixel distortion generated during the swing-scan process. However, this method is only applicable to application scenarios where the target field of view can be approximated as a plane. The underwater terrain is often more complex and the target surface shape is undulating. The use of traditional geometric correction methods will inevitably cause large errors, resulting in the acquired image being unable to reflect the original shape of the target. Summary of the invention
[0004] The purpose of the present invention is to provide an image correction method and system for an underwater swing-scanning hyperspectral camera, which can improve the pixel distortion problem in images acquired by the underwater swing-scanning hyperspectral camera, improve the imaging quality of the camera, and ensure that the target size of the underwater image finally acquired will not be significantly distorted compared with the actual object.
[0005] The objective of the present invention is achieved through the following technical solutions:
[0006] An image correction method for an underwater swing-scanning hyperspectral camera, the method comprising:
[0007] Step 1: Obtain the current geographical location, shooting attitude angle and target distance value information of the underwater swing scanning hyperspectral camera through an auxiliary positioning device and sonar;
[0008] Step 2: using the underwater swing-scanning hyperspectral camera and the wide-field-of-view RGB correction camera to collect images of the target area, and obtaining an original hyperspectral image to be corrected and a wide-field-of-view RGB image for providing correction reference information;
[0009] Step 3: input the current geographical location and shooting attitude angle information of the underwater swing-scanning hyperspectral camera obtained in step 1 into the wide-field RGB correction camera to generate a self-corrected RGB image of the target area;
[0010] Step 4: Perform feature matching on the self-corrected RGB image and the original hyperspectral image to be corrected;
[0011] Step 5: mesh the feature points matched in step 4, solve the displacement matrix by finite element analysis method, and use it to correct the original hyperspectral image to be corrected.
[0012] Based on the above method, an embodiment of the present invention further provides an image correction system for an underwater swing-scanning hyperspectral camera, the system comprising:
[0013] The camera information acquisition module is used to obtain the current geographical location, shooting attitude angle and target distance value information of the underwater swing scanning hyperspectral camera through the auxiliary positioning device and sonar;
[0014] An image acquisition module, including an underwater swing-scanning hyperspectral camera and a wide-field RGB correction camera, is used to acquire images of the target area to obtain an original hyperspectral image to be corrected and a wide-field RGB image for providing correction reference information;
[0015] A self-correction RGB image generation module, including an RGB image processing and storage device, is used to input the current geographical location and shooting attitude angle information of the underwater swing scanning hyperspectral camera obtained by the camera information acquisition module into the wide field of view RGB correction camera to generate a self-correction RGB image of the target area;
[0016] The hyperspectral image correction module includes a hyperspectral image processing and storage device, which is used to perform feature matching on the self-corrected RGB image generated by the self-corrected RGB image generation module and the original hyperspectral image to be corrected obtained by the image acquisition module, and then grid the matched feature points, solve the displacement matrix through the finite element analysis method, and use it to correct the original hyperspectral image to be corrected.
[0017] It can be seen from the technical solution provided by the present invention that the above method and system can improve the pixel distortion problem in the image acquired by the underwater swing-scanning hyperspectral camera, improve the imaging quality of the camera, and ensure that the target size of the underwater image finally acquired will not be significantly distorted compared with the actual object. BRIEF DESCRIPTION OF THE DRAWINGS
[0018] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative work.
[0019] Figure 1 A schematic flow chart of an image correction method for an underwater swing-scanning hyperspectral camera provided in an embodiment of the present invention;
[0020] Figure 2 It is a structural schematic diagram of the system described in an embodiment of the present invention;
[0021] Figure 3 A schematic diagram of an implementation of the system described in an embodiment of the present invention. DETAILED DESCRIPTION
[0022] The following is a clear and complete description of the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments, which does not constitute a limitation of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0023] like Figure 1 The figure is a flow chart of an image correction method for an underwater swing-scanning hyperspectral camera provided by an embodiment of the present invention, the method comprising:
[0024] Step 1: Obtain the current geographical location, shooting attitude angle and target distance value information of the underwater swing scanning hyperspectral camera through an auxiliary positioning device and sonar;
[0025] In this step, the current geographic coordinate information of the underwater swing-scanning hyperspectral camera is obtained based on the auxiliary positioning device to record the geographic location of the current captured image;
[0026] The acquired shooting attitude angle is used to provide attitude information for the self-correction of wide-field RGB images and the initial geometric correction of swing-scan hyperspectral images;
[0027] The sonar is used to obtain the returned target distance value, which is used to provide the absolute value of the field of view size corresponding to the target for the swing-scan hyperspectral image.
[0028] Step 2: using the underwater swing-scanning hyperspectral camera and the wide-field-of-view RGB correction camera to collect images of the target area, and obtaining an original hyperspectral image to be corrected and a wide-field-of-view RGB image for providing correction reference information;
[0029] Step 3: input the current geographical location and shooting attitude angle information of the underwater swing-scanning hyperspectral camera obtained in step 1 into the wide-field RGB correction camera to generate a self-corrected RGB image of the target area;
[0030] Step 4: Perform feature matching on the self-corrected RGB image and the original hyperspectral image to be corrected;
[0031] In this step, the original hyperspectral image to be corrected is firstly subjected to initial geometric correction by using the current shooting attitude angle information and the swing angle information of the underwater swing-scanning hyperspectral camera to obtain an initial corrected hyperspectral image matrix I;
[0032] Through Difference of Gaussian (DoG) spatial extreme value detection, the feature point set and corresponding SIFT feature descriptor of the self-corrected RGB image matrix R and the initial corrected hyperspectral image matrix I are calculated respectively;
[0033] Match the feature points of the matrix R and I to generate a matching feature point pair set S, and sort the elements in the feature point pair set S from high to low according to the matching degree;
[0034] Determine the number of elements N in the feature point pair set S. If N is greater than 800, truncate the feature point pair set S, remove the 801th and subsequent elements, and retain only the first 800 feature point pairs; if N is less than or equal to 800, keep the elements in the feature point pair set S unchanged.
[0035] In addition, the above-mentioned DoG space extreme value detection can be replaced by Harris corner detection; the SIFT feature descriptor can be replaced by BRIEF or HOG feature descriptor.
[0036] Step 5: mesh the feature points matched in step 4, solve the displacement matrix by finite element analysis method, and use it to correct the original hyperspectral image to be corrected.
[0037] In this step, firstly, the matrix R of the self-corrected RGB image and the matrix I of the initial corrected hyperspectral image are triangulated according to the elements in the feature point pair set S;
[0038] For each grid, the unit stiffness matrix K and unit load matrix F are solved according to the triangular unit nodes through the finite element analysis method, which are expressed as:
[0039]
[0040]
[0041]
[0042]
[0043] Where D is the elastic matrix; B is the unit strain matrix; I is the image grayscale value; N is the shape function; E is the elastic modulus; Δ is the mesh area; x and y are the coordinate values of the mesh nodes; the superscript T represents the matrix transpose, and the subscripts 1, 2, and 3 correspond to the three nodes of the triangular unit respectively; d represents the differential; ▽ is the gradient operator;
[0044] Solve the finite element equation Kμ=F to obtain the displacement matrix μ of all pixels in each grid;
[0045] Repeat the above solution process to obtain the displacement of all pixel points in the grid, and then transform the initial corrected hyperspectral image matrix I according to the displacement to obtain the corrected hyperspectral image I2.
[0046] In addition, the above-mentioned triangular mesh division can also be replaced by quadrilateral mesh division and other methods.
[0047] Based on the above method, the embodiment of the present invention also provides an image correction system for an underwater swing-scanning hyperspectral camera, such as Figure 2 FIG. 1 is a schematic diagram of the structure of a system according to an embodiment of the present invention, wherein the system comprises:
[0048] The camera information acquisition module is used to obtain the current geographical location, shooting attitude angle and target distance value information of the underwater swing scanning hyperspectral camera through the auxiliary positioning device and sonar;
[0049] An image acquisition module, including an underwater swing-scanning hyperspectral camera and a wide-field RGB correction camera, is used to acquire images of the target area to obtain an original hyperspectral image to be corrected and a wide-field RGB image for providing correction reference information;
[0050] A self-correction RGB image generation module, including an RGB image processing and storage device, is used to input the current geographical location and shooting attitude angle information of the underwater swing scanning hyperspectral camera obtained by the camera information acquisition module into the wide field of view RGB correction camera to generate a self-correction RGB image of the target area;
[0051] The hyperspectral image correction module includes a hyperspectral image processing and storage device, which is used to perform feature matching on the self-corrected RGB image generated by the self-corrected RGB image generation module and the original hyperspectral image to be corrected obtained by the image acquisition module, and then grid the matched feature points, solve the displacement matrix through the finite element analysis method, and use it to correct the original hyperspectral image to be corrected.
[0052] The specific implementation process of the above modules is described in the method embodiment. Figure 3 The figure shows a schematic diagram of an implementation of the system described in an embodiment of the present invention, which includes a sonar (1), an underwater swing-scanning hyperspectral camera (2), a wide-field RGB correction camera (3), an RGB image processing and storage device (4) and a hyperspectral image processing and storage device (5). The corresponding modules are integrated into a design so that they can be easily carried and operated.
[0053] It is worth noting that the contents not described in detail in the embodiments of the present invention belong to the prior art known to professional and technical personnel in the field.
[0054] The above is only a preferred specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily thought of by any technician familiar with the technical field within the technical scope disclosed in the present invention should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims. The information disclosed in the background technology section of this article is only intended to deepen the understanding of the overall background technology of the present invention, and should not be regarded as an admission or in any form that the information constitutes prior art known to those skilled in the art.
Claims
1. An image correction method for an underwater swing-scanning hyperspectral camera, characterized in that: The method comprises: Step 1: Obtain the current geographical location, shooting attitude angle and target distance value information of the underwater swing scanning hyperspectral camera through an auxiliary positioning device and sonar; Step 2: using the underwater swing-scanning hyperspectral camera and the wide-field-of-view RGB correction camera to collect images of the target area, and obtaining an original hyperspectral image to be corrected and a wide-field-of-view RGB image for providing correction reference information; Step 3: input the current geographical location and shooting attitude angle information of the underwater swing-scanning hyperspectral camera obtained in step 1 into the wide-field RGB correction camera to generate a self-corrected RGB image of the target area; Step 4: Perform feature matching on the self-corrected RGB image and the original hyperspectral image to be corrected; The process of step 4 is specifically as follows: Performing initial geometric correction on the original hyperspectral image to be corrected by using the current shooting attitude angle information and the swing angle information of the underwater swing-scanning hyperspectral camera to obtain an initial corrected hyperspectral image matrix I; Through Gaussian difference DoG space extreme value detection, the feature point set and corresponding SIFT feature descriptor of the matrix R of the self-corrected RGB image and the matrix I of the initial corrected hyperspectral image are calculated respectively; Match the feature points of the matrix R and I to generate a matching feature point pair set S, and sort the elements in the feature point pair set S from high to low according to the matching degree; Determine the number of elements N in the feature point pair set S. If N is greater than 800, truncate the feature point pair set S, remove the 801th and subsequent elements, and retain only the first 800 feature point pairs; if N is less than or equal to 800, keep the elements in the feature point pair set S unchanged; Step 5, meshing the feature points matched in step 4, solving the displacement matrix by finite element analysis method, and correcting the original hyperspectral image to be corrected; The process of step 5 is specifically as follows: According to the elements in the feature point pair set S, the matrix R of the self-corrected RGB image and the matrix I of the initial corrected hyperspectral image are respectively divided into triangular meshes; For each grid, the unit stiffness matrix K and unit load matrix F are solved according to the triangular unit nodes through the finite element analysis method, which are expressed as: Where D is the elastic matrix; B is the unit strain matrix; I is the image grayscale value; N is the shape function; E is the elastic modulus; Δ is the mesh area; x and y are the coordinate values of the mesh nodes; the superscript T represents the matrix transpose, and the subscripts 1, 2, and 3 correspond to the three nodes of the triangular unit respectively; d represents the differential; is the gradient operator; Solve the finite element equation Kμ=F to obtain the displacement matrix μ of all pixels in each grid; Repeat the above solution process to obtain the displacement of all pixel points in the grid, and then transform the initial corrected hyperspectral image matrix I according to the displacement to obtain the corrected hyperspectral image.
2. The image correction method of the underwater swing-scanning hyperspectral camera according to claim 1, characterized in that: In step 1, the current geographic coordinate information of the underwater swing-scanning hyperspectral camera is obtained based on the auxiliary positioning device to record the geographic location of the current captured image; The acquired shooting attitude angle is used to provide attitude information for the self-correction of wide-field RGB images and the initial geometric correction of swing-scan hyperspectral images; The sonar is used to obtain the returned target distance value, which is used to provide the absolute value of the field of view size corresponding to the target for the swing-scan hyperspectral image.
3. The image correction method of the underwater swing-scanning hyperspectral camera according to claim 1, characterized in that: DoG spatial extreme value detection can be replaced by Harris corner detection; The SIFT feature descriptor can be replaced by the BRIEF or HOG feature descriptor.
4. The image correction method of the underwater swing-scanning hyperspectral camera according to claim 1, characterized in that: Triangular meshing can be replaced by quadrilateral meshing.
5. An image correction system for an underwater swing-scanning hyperspectral camera, characterized in that: The system comprises: The camera information acquisition module is used to obtain the current geographical location, shooting attitude angle and target distance value information of the underwater swing scanning hyperspectral camera through the auxiliary positioning device and sonar; An image acquisition module, including an underwater swing-scanning hyperspectral camera and a wide-field RGB correction camera, is used to acquire images of the target area to obtain an original hyperspectral image to be corrected and a wide-field RGB image for providing correction reference information; A self-correction RGB image generation module, including an RGB image processing and storage device, is used to input the current geographical location and shooting attitude angle information of the underwater swing scanning hyperspectral camera obtained by the camera information acquisition module into the wide field of view RGB correction camera to generate a self-correction RGB image of the target area; The hyperspectral image correction module includes a hyperspectral image processing and storage device, which is used to perform feature matching on the self-corrected RGB image generated by the self-corrected RGB image generation module and the original hyperspectral image to be corrected obtained by the image acquisition module, and then mesh the matched feature points, solve the displacement matrix by the finite element analysis method, and use it to correct the original hyperspectral image to be corrected; The process of feature matching between the self-corrected RGB image and the original hyperspectral image to be corrected is as follows: Performing initial geometric correction on the original hyperspectral image to be corrected by using the current shooting attitude angle information and the swing angle information of the underwater swing-scanning hyperspectral camera to obtain an initial corrected hyperspectral image matrix I; Through Gaussian difference DoG space extreme value detection, the feature point set and corresponding SIFT feature descriptor of the matrix R of the self-corrected RGB image and the matrix I of the initial corrected hyperspectral image are calculated respectively; Match the feature points of the matrix R and I to generate a matching feature point pair set S, and sort the elements in the feature point pair set S from high to low according to the matching degree; Determine the number of elements N in the feature point pair set S. If N is greater than 800, truncate the feature point pair set S, remove the 801th and subsequent elements, and retain only the first 800 feature point pairs; if N is less than or equal to 800, keep the elements in the feature point pair set S unchanged; The specific process of solving the displacement matrix by finite element analysis method and correcting the original hyperspectral image to be corrected is as follows: According to the elements in the feature point pair set S, the matrix R of the self-corrected RGB image and the matrix I of the initial corrected hyperspectral image are respectively divided into triangular meshes; For each grid, the unit stiffness matrix K and unit load matrix F are solved according to the triangular unit nodes through the finite element analysis method, which are expressed as: Where D is the elastic matrix; B is the unit strain matrix; I is the image grayscale value; N is the shape function; E is the elastic modulus; Δ is the mesh area; x and y are the coordinate values of the mesh nodes; the superscript T represents the matrix transpose, and the subscripts 1, 2, and 3 correspond to the three nodes of the triangular unit respectively; d represents the differential; is the gradient operator; Solve the finite element equation Kμ=F to obtain the displacement matrix μ of all pixels in each grid; Repeat the above solution process to obtain the displacement of all pixel points in the grid, and then transform the initial corrected hyperspectral image matrix I according to the displacement to obtain the corrected hyperspectral image.
Citation Information
Patent Citations
Airborne hyperspectral image geometric correction device based on three-way array camera and airborne hyperspectral image geometric correction method thereof
CN107221010A
Method and system of unmanned aerial vehicle imaging hyperspectral geometric correction based on RGB image
CN107798668A
Geometric correction method of hyperspectral remote sensing data based on image processing technology
CN109118443A
Unmanned aerial vehicle large-view-field hyperspectral image generation method and system
CN112991186A
Drone-mounted imaging hyperspectral geometric correction method and system
US20170200259A1
