Method and apparatus for auto-focus correcting hyperspectral camera

Abstract

Provided are a method and an apparatus for auto-focus correcting a hyperspectral camera. A method for auto-focus correcting a hyperspectral image, according to an embodiment of the present invention, calculates the degree to which each of a plurality of channel images constituting the hyperspectral image are blurred at a reference focal length, and corrects the degree of blurring of the channel images of the hyperspectral image acquired at the reference focal length. Accordingly, an image blurring problem that occurs when acquiring information about light of a wide range of wavelengths through the hyperspectral camera because the optimal focal for each wavelength is different can be solved by post-processing the acquired hyperspectral image.

Description

Hyperspectral camera autofocus correction method and device

[0001] The present invention relates to an image acquisition technique using a hyperspectral camera, and more particularly, to a method for solving an image blur problem caused by focus mismatch when acquiring image information over a wide wavelength range.

[0002] A hyperspectral camera is an image acquisition device that digitizes and stores light information at different wavelengths for each channel. Because the degree of light diffraction varies depending on wavelength, focusing on one channel causes other channels to become out of focus, resulting in a blurry image.

[0003] To solve these problems, conventional technologies repeatedly capture images of a subject while changing the focus, and then select and collate clear images for each channel from the acquired hyperspectral images.

[0004] However, this solution has the problem of requiring a lot of time, resources, and effort to acquire hyperspectral images due to its high difficulty.

[0005] The present invention has been devised to solve the above problems, and the purpose of the present invention is to provide a hyperspectral image autofocus correction method and device for obtaining clear hyperspectral images by alleviating the image blur problem that occurs because the optimal focal length is different for each wavelength when obtaining light information of a wide wavelength through a hyperspectral camera.

[0006] In order to achieve the above object, a hyperspectral image auto-focus correction method according to one embodiment of the present invention includes the steps of: calculating the degree of blurring at a reference focal distance for each of a plurality of channel images constituting a hyperspectral image; and correcting the degree of blurring for channel images of a hyperspectral image acquired at a reference focal distance.

[0007] The calculation step may include a step of acquiring hyperspectral images while matching the focal lengths of each channel image; a step of estimating the degree of blurring by comparing each channel image of a hyperspectral image of a reference focal length with a corresponding channel image of a hyperspectral image having a matching focal length.

[0008] The reference focal length may be a focal length that matches one of the channel images of the hyperspectral image.

[0009] The degree of blurring can be the Point Spread Function (PSF).

[0010] The correction step may be to correct each channel image by performing a deconvolution operation between each channel image and the corresponding PSF.

[0011] The correction step may be to perform a deconvolution operation using a Wiener filter.

[0012] The correction step may be to obtain corrected channel images by inputting hyperspectral images of a reference focal length into a machine learning model that predicts channel images with matching focal lengths from channel images with mismatched focal lengths.

[0013] A machine learning model can be trained using a training dataset that uses channel images with mismatched focal lengths as input data and channel images with matching focal lengths as correct data.

[0014] The input data may be channel images with non-matching focal lengths generated from channel images with matching focal lengths using the blurred degrees estimated in the estimation step.

[0015] According to another aspect of the present invention, a hyperspectral image auto-focus correction method is provided, characterized in that it includes an estimation module for estimating the degree of blurring at a reference focal distance for each of a plurality of channel images constituting a hyperspectral image; and a correction module for correcting the degree of blurring for channel images of a hyperspectral image acquired at a reference focal distance.

[0016] According to another aspect of the present invention, a machine learning model training method for hyperspectral image autofocus correction is provided, comprising: a step of obtaining channel images having matching focal lengths and channel images having mismatched focal lengths from a plurality of channel images constituting a hyperspectral image; a step of constructing a learning dataset using the obtained channel images having mismatched focal lengths as input data and channel images having matching focal lengths as correct data; and a step of training a machine learning model for predicting channel images having matching focal lengths from channel images having mismatched focal lengths using the constructed learning dataset.

[0017] According to another aspect of the present invention, a machine learning model training system for hyperspectral image autofocus correction is provided, comprising: a hyperspectral camera that acquires channel images with matching focal lengths and channel images with mismatched focal lengths from a plurality of channel images constituting a hyperspectral image; a training unit that constructs a training dataset using the acquired channel images with mismatched focal lengths as input data and the channel images with matching focal lengths as correct data, and trains a machine learning model that predicts channel images with matching focal lengths from the channel images with mismatched focal lengths using the constructed training dataset;

[0018] As described above, according to embodiments of the present invention, when obtaining light information of a wide wavelength through a hyperspectral camera, the image blur problem that occurs because the optimal focal length is different for each wavelength can be solved by post-processing the obtained hyperspectral image.

[0019] In addition, according to embodiments of the present invention, the process of repeatedly photographing a subject while adjusting the focus of a hyperspectral camera and collating the images can be omitted, thereby lowering the difficulty of obtaining a high-quality hyperspectral image, thereby reducing the time, resources, and effort required to obtain a hyperspectral image.

[0020] Figure 1 is a step for calculating the degree of blurring among these.

[0021] Figure 2 is a blur correction step,

[0022] Figure 3 is an example of a hyperspectral image generation system.

[0023] Figure 4 shows a blur correction step using a machine learning model.

[0024] Figure 5 is another example of a hyperspectral image generation system.

[0025] Hereinafter, the present invention will be described in more detail with reference to the drawings.

[0026] A hyperspectral camera is a recording device that digitizes and stores light information across a wide wavelength range. Each channel of a hyperspectral image records information at a different wavelength. The degree of light diffraction varies depending on the wavelength.

[0027] Therefore, if the focal length of the hyperspectral camera lens is not adjusted to suit each wavelength, image blurring occurs when acquiring hyperspectral images, causing images to appear blurry in certain channels. To address this issue, a variable focal length lens or manual adjustment of the focal length can be used to repeatedly capture the same subject, and the resulting hyperspectral images can be combined to obtain a hyperspectral image.

[0028] However, in situations where the surrounding environment is not controlled, when performing repeated shooting and obtaining hyperspectral images, there is a limitation in that it is difficult to perform repeated shooting in the same environment because the images are not taken at the same time, and the subject's posture changes or the environment, such as lighting, changes.

[0029] Accordingly, in an embodiment of the present invention, a hyperspectral image autofocus correction method for resolving the image blur phenomenon is proposed. This technology is to obtain a clear hyperspectral image by post-processing a hyperspectral image acquired from a fixed camera focus without adjusting the focus during the image acquisition stage to resolve the image blur problem caused by focus mismatch in a hyperspectral image acquired through a hyperspectral camera.

[0030] A hyperspectral image autofocus correction method according to an embodiment of the present invention comprises a step of calculating a degree of blurring and a step of correcting the blur based on the calculation result.

[0031] Figure 1 is a diagram illustrating a step for calculating the degree of blurring among these. Specifically, it is a process for calculating the degree of blurring at a reference focal length for each of the multiple channel images (images with different wavelengths) that constitute a hyperspectral image.

[0032] To this end, as illustrated in Fig. 1, first, using a hyperspectral image acquisition device in a controlled environment, hyperspectral images are acquired while matching the focal lengths of each channel image (S110).

[0033] Specifically, a hyperspectral image 1 is acquired with a focal length 1 that matches channel image 1 of wavelength 1, a hyperspectral image 2 is acquired with a focal length 2 that matches channel image 2 of wavelength 2, a hyperspectral image 3 is acquired with a focal length 3 that matches channel image 3 of wavelength 3, ..., a hyperspectral image N is acquired with a focal length N that matches channel image N of wavelength N. Accordingly, N hyperspectral images are acquired, and each hyperspectral image is composed of N channel images.

[0034] For each channel image of a hyperspectral image of the following reference focal length (focal length 1 in Fig. 1), the degree of blurring is estimated by comparing it with the corresponding channel image of a hyperspectral image of the same focal length (S120). The degree of blurring can be estimated using the Point Spread Function (PSF).

[0035] Specifically, since channel image 1 matches the focal length 1 and no blur occurs, the PSF is not estimated, but for channel image 2, the PSF is estimated by comparing channel image 2 of the hyperspectral image acquired with the focal length 1 with channel image 2 of the hyperspectral image acquired with the focal length 2, and for channel image 3, the PSF is estimated by comparing channel image 3 of the hyperspectral image acquired with the focal length 1 with channel image 3 of the hyperspectral image acquired with the focal length 3, ..., for channel image N, the PSF is estimated by comparing channel image N of the hyperspectral image acquired with the focal length 1 with channel image N of the hyperspectral image acquired with the focal length N.

[0036] Meanwhile, in Fig. 1, the reference focal length is assumed to be focal length 1 when estimating the PSF, but this is merely exemplary. Therefore, it is also possible to apply a focal length other than focal length 1 as the reference focal length.

[0037] Afterwards, the hyperspectral image is post-processed using the PSFs for each estimated channel image to correct blur. Figure 2 is a diagram for explaining the blur correction step.

[0038] As illustrated, for each channel image of the hyperspectral image acquired at the reference focal length (focal length 1), blur is corrected through post-processing using a deconvolution operation with the corresponding PSF (S130). The deconvolution operation in step S130 can be performed using a Wiener filter, but other means may also be used.

[0039] FIG. 3 is a diagram illustrating the configuration of a hyperspectral image generation system according to another embodiment of the present invention. The hyperspectral image generation system according to the embodiment of the present invention is configured to include, as illustrated, a hyperspectral camera (210), a PSF estimation module (220), a deconvolution module (230), and an output unit (240).

[0040] The hyperspectral camera (210) acquires hyperspectral images while matching the focal length of each channel image.

[0041] The PSF estimation module (220) estimates the PSF by comparing each channel image of a hyperspectral image of a reference focal length acquired from a hyperspectral camera (210) with a corresponding channel image of a hyperspectral image of a matching focal length.

[0042] The deconvolution module (230) performs a deconvolution operation on the hyperspectral image using the PSFs for each channel image estimated by the PSF estimation module (220) to correct blur.

[0043] The output unit (240) outputs a hyperspectral image corrected by the deconvolution module (230).

[0044] Meanwhile, the blur correction step of Figure 2 can also be implemented using a machine learning model, as shown in Figure 4. This involves building a training dataset, training the machine learning model, and then performing an inference step to correct the blur using the trained machine learning model. This is described in detail below.

[0045] First, channel images with matching focal lengths are acquired using the method illustrated in Fig. 1, and channel images with mismatched focal lengths are generated from these, and a learning dataset is constructed using the channel images with mismatched focal lengths as input data and the channel images with matching focal lengths as correct data (S141).

[0046] Channel images with non-matching focal lengths are generated by performing a convolution operation (the inverse operation of the aforementioned deconvolution operation) with the corresponding PSFs for channel images with matching focal lengths.

[0047] Alternatively, it is possible to utilize the channel images of the hyperspectral image acquired with the reference focal length in the method illustrated in Fig. 1 as channel images with non-matching focal lengths.

[0048] In the following step S141, a machine learning model is trained to predict channel images with matching focal lengths from channel images with mismatched focal lengths using the training dataset constructed (S142).

[0049] Afterwards, the channel images constituting the hyperspectral image of the reference focal length are input into the machine learning model that has completed learning in step S142 to generate channel images with blur correction (S143).

[0050] Machine learning models can be implemented through Random Forest, Support Vector Machine, etc., and there are no special restrictions on the specific type, structure, learning method, or loss function.

[0051] FIG. 5 is a diagram illustrating the configuration of a hyperspectral image generation system according to another embodiment of the present invention. As illustrated, the hyperspectral image generation system according to the embodiment of the present invention comprises a hyperspectral camera (210), a PSF estimation module (220), a learning unit (250), an inference unit (260), and an output unit (240).

[0052] The hyperspectral image generation system of FIG. 5 is the hyperspectral image generation system of FIG. 4, in which the deconvolution module (230) is replaced with a learning unit (250) and an inference unit (260).

[0053] The learning unit (250) is configured to train a machine learning model using a learning dataset constructed according to step S141 of FIG. 4 described above, and the inference unit (260) is configured to correct a hyperspectral image according to step S143 of FIG. 4 using the machine learning model trained by the learning unit (250).

[0054] So far, a preferred embodiment of a hyperspectral camera autofocus correction method and device has been described in detail.

[0055] In the above embodiment, the image blur problem that occurs when obtaining light information of a wide wavelength using a hyperspectral camera because the optimal focal length is different for each wavelength can be resolved by post-processing the obtained hyperspectral image.

[0056] This allows the process of repeatedly photographing a subject while adjusting the focus of a hyperspectral camera and collating them to be omitted, thereby lowering the difficulty of obtaining high-quality hyperspectral images and reducing the time, resources, and effort required to obtain hyperspectral images.

[0057] Meanwhile, it goes without saying that the technical idea of ​​the present invention can also be applied to a computer-readable recording medium containing a computer program that performs the functions of the device and method according to the present embodiment. In addition, the technical idea according to various embodiments of the present invention can be implemented in the form of computer-readable code recorded on a computer-readable recording medium. The computer-readable recording medium can be any data storage device that can be read by a computer and store data. For example, the computer-readable recording medium can be a ROM, a RAM, a CD-ROM, a magnetic tape, a floppy disk, an optical disk, a hard disk drive, etc. In addition, the computer-readable code or program stored on the computer-readable recording medium can be transmitted through a network connected between computers.

[0058] In addition, although the preferred embodiments of the present invention have been illustrated and described above, the present invention is not limited to the specific embodiments described above, and various modifications can be made by a person having ordinary skill in the art to which the present invention pertains without departing from the gist of the present invention as claimed in the claims, and such modifications should not be understood individually from the technical idea or prospect of the present invention.

Claims

1. A step of calculating the degree of blurring at a reference focal length for each of a plurality of channel images constituting a hyperspectral image; A hyperspectral image auto-focus correction method, characterized by including a step of correcting a blurred degree for channel images of a hyperspectral image acquired at a reference focal length.

2. In claim 1, The calculation steps are: A step of acquiring hyperspectral images while matching the focal length of each channel image; A hyperspectral image auto-focus correction method, characterized by including a step of estimating the degree of blurring by comparing each of the channel images of a hyperspectral image of a reference focal length with a corresponding channel image of a hyperspectral image having a matching focal length.

3. In claim 2, The standard focal length is, A hyperspectral image auto-focus correction method characterized in that the focal length matches one of the channel images of the hyperspectral image.

4. In claim 2, The degree of blurring is, A hyperspectral image autofocus correction method characterized by a PSF (Point Spread Function).

5. In claim 4, The correction step is, A hyperspectral image auto-focus correction method characterized by correcting each channel image through a deconvolution operation of each channel image and the corresponding PSF.

6. In claim 5, The correction step is, A hyperspectral image autofocus correction method characterized by performing a deconvolution operation using a Wiener filter.

7. In claim 2, The correction step is, A hyperspectral image auto-focus correction method characterized by obtaining corrected channel images by inputting hyperspectral images of a reference focal length into a machine learning model that predicts channel images with matching focal lengths from channel images with mismatched focal lengths.

8. In claim 7, The machine learning model is, A hyperspectral image auto-focus correction method characterized in that it is learned using a learning dataset that uses channel images with mismatched focal lengths as input data and channel images with matching focal lengths as correct data.

9. In claim 8, The input data is, A hyperspectral image auto-focus correction method characterized in that channel images with mismatched focal lengths are generated from channel images with matching focal lengths by using the blurred degrees estimated in the estimation step.

10. An estimation module that estimates the degree of blurring at a reference focal length for each of a plurality of channel images constituting a hyperspectral image; A hyperspectral image auto-focus correction method, characterized by including a correction module for correcting the degree of blurring of channel images of a hyperspectral image acquired at a reference focal length.

11. A step of obtaining channel images having matching focal lengths and channel images having non-matching focal lengths from a plurality of channel images constituting a hyperspectral image; A step of constructing a learning dataset using channel images with mismatched focal lengths as input data and channel images with matching focal lengths as correct data; A method for training a machine learning model for hyperspectral image autofocus correction, characterized by including the step of training a machine learning model that predicts channel images with matching focal lengths from channel images with mismatched focal lengths using a constructed learning dataset.

12. A hyperspectral camera that acquires channel images with matching focal lengths and channel images with non-matching focal lengths from a plurality of channel images constituting a hyperspectral image; A machine learning model learning system for hyperspectral image autofocus correction, characterized by including a learning unit which constructs a learning dataset using channel images with mismatched focal lengths as input data and channel images with matching focal lengths as correct data, and trains a machine learning model that predicts channel images with matching focal lengths from channel images with mismatched focal lengths using the constructed learning dataset.

Images