Low-power, high-efficiency spectroscopic image signal processing method and apparatus
The spectroscopic image signal processing device addresses the weight and power issues of existing cameras by separating and fusing infrared signals to detect multiple gases efficiently on small unmanned vehicles.
Patent Information
- Application Number
- KR1020240110319
- Authority / Receiving Office
- KR · KR
- Patent Type
- Patents
- Current Assignee / Owner
- Filing Date
- 2024-08-19
- Publication Date
- 2026-11-06
- Estimated Expiration
- 2044-08-19
AI Technical Summary
Existing spectroscopic cameras are too heavy for use on small unmanned vehicles and lack the capability to detect multiple gases like carbon dioxide and sulfur dioxide efficiently while minimizing power consumption.
A spectroscopic image signal processing device that separates mid-wavelength and long-wavelength infrared signals, corrects and fuses the images to detect multiple gases, and dynamically adjusts correction and fusion timings based on altitude and power consumption.
Enables efficient detection of multiple gases on unmanned vehicles by reducing power consumption and weight, enhancing detection accuracy through dynamic image correction and fusion.
Smart Images

Figure 112024089712478-PAT00005_ABST
Abstract
Description
Technology Field
[0001] The technical field to which the present invention belongs relates to a spectroscopic image signal processing apparatus and method, and more specifically, to a spectroscopic image signal processing apparatus and method that spectrally analyzes a mid-wavelength infrared signal, spectrally analyzes a long-wavelength infrared signal, corrects each spectrally analyzed image, and uses a fused image to detect multiple gases. Background Technology
[0002] The content described in this section merely provides background information regarding the present embodiment and does not constitute prior art.
[0003] A spectroscopic camera is a device capable of acquiring spectral images of three-dimensional structures by rapidly generating images of tens to hundreds of segmented wavelength bands ranging from ultraviolet and visible light to far-infrared light. Since spectroscopic cameras can detect differences in reflected / radiated energy depending on the chemical properties of a substance, they can be utilized in geological surveys and mineral exploration using aerial unmanned vehicles, or in detecting atmospheric or leaking gases occurring remotely using ground or marine unmanned vehicles.
[0004] Existing commercial spectroscopic cameras weigh between 5 and 10 kg, which poses a weight issue for application to small unmanned vehicles. Therefore, to utilize them in unmanned vehicles in various environments, the spectroscopic camera needs to be miniaturized and lightweight.
[0005] In particular, there is a need for equipment capable of detecting multiple gases, such as carbon dioxide and sulfur dioxide, through multi-band analysis while mounted on an unmanned vehicle for an extended period using power reduction technology.
[0006] Regarding power processing technology, Korean Registered Patent No. 10-1990316 describes a multi-wavelength band monitoring device. Korean Registered Patent No. 10-1990316 merely controls the power supply and does not consider a configuration for detecting multiple gases, such as carbon dioxide and sulfur dioxide, by analyzing spectroscopic images of mid-wavelength infrared and long-wavelength infrared through power reduction technology. Prior art literature
[0007] Korean Registered Patent No. 10-1990316 (June 18, 2019) The problem to be solved
[0008] The main objective of the embodiments of the present invention is to provide a spectroscopic image signal processing device mountable on an unmanned vehicle capable of detecting multiple gases, such as carbon dioxide and sulfur dioxide, by separating a mid-wavelength infrared signal and a long-wavelength infrared signal through a beam splitter, spectrally analyzing the mid-wavelength infrared signal and the long-wavelength infrared signal, controlling the correction type and correction timing for correcting each spectrally analyzed image, and fusing the corrected images to form a fused image.
[0009] Other unspecified objects of the present invention may be further considered to the extent that they can be easily inferred from the following detailed description and effects. means of solving the problem
[0010] According to one aspect of the present embodiment, a spectroscopic image signal processing device may include an infrared optical system that receives an infrared signal and separates it into a mid-wavelength infrared signal and a long-wavelength infrared signal for outputting; a mid-wavelength infrared detector that converts the mid-wavelength infrared signal into a first spectroscopic image; a long-wavelength infrared detector that converts the long-wavelength infrared signal into a second spectroscopic image; and a spectroscopic image processing unit that detects a plurality of gases using the first spectroscopic image and the second spectroscopic image.
[0011] The above-described spectral image processing unit may include a correction type selection unit that selects at least one correction type among a plurality of correction types and corrects the first spectral image and the second spectral image according to the selected correction type, and a first fusion image processing unit that generates a first fusion image by fusing the corrected first spectral image and the corrected second spectral image and detects a plurality of gases from the first fusion image.
[0012] The above-described spectroscopic image processing unit may include an altitude sensor for measuring the altitude of an unmanned mobile body, and a power measuring unit for measuring a first power consumed in correcting the first spectroscopic image and the second spectroscopic image, measuring a second power consumed in fusing the corrected first spectroscopic image and the corrected second spectroscopic image, and measuring a third power consumed in detecting a plurality of gases from the fused image.
[0013] The spectroscopic image processing unit may include a timing change unit that controls a re-correction timing for changing the correction type based on the altitude, the first power, the second power, and the third power, and controls a re-fusion timing for fusing the re-corrected first spectroscopic image and the re-corrected second spectroscopic image.
[0014] The above-described spectral image processing unit may include a correction type changing unit that, in accordance with the above-described re-correction timing, changes the correction type for correcting the first spectral image and the second spectral image based on the altitude, the first power, the second power, and the third power to another correction type, and re-corrects the first spectral image and the second spectral image according to the changed correction type.
[0015] The spectroscopic image processing unit may include a second fusion image processing unit that generates a second fusion image by fusing a re-corrected first spectroscopic image and a re-corrected second spectroscopic image in accordance with the re-fusion timing, and detects a plurality of gases from the second fusion image.
[0016] According to another aspect of the present embodiment, a spectral image signal processing method comprises the steps of: receiving an infrared signal and separating it into a mid-wavelength infrared signal and a long-wavelength infrared signal and outputting them; converting the mid-wavelength infrared signal into a first spectral image; converting the long-wavelength infrared signal into a second spectral image; selecting at least one correction type among a plurality of correction types and correcting the first spectral image and the second spectral image according to the selected correction type; generating a first fused image by fusing the corrected first spectral image and the corrected second spectral image and detecting a plurality of gases from the first fused image; measuring the altitude of an unmanned vehicle through an altitude sensor; measuring a first power consumed in correcting the first spectral image and the second spectral image, measuring a second power consumed in fusing the corrected first spectral image and the corrected second spectral image, and measuring a third power consumed in detecting a plurality of gases from the fused image; and, based on the altitude, the first power, the second power, and the third power, the correction type The method may include the steps of: controlling a recalibration timing to change and controlling a refusion timing that fuses a recalibrated first spectral image and a recalibrated second spectral image; changing the correction type that corrects the first spectral image and the second spectral image based on the altitude, the first power, the second power, and the third power to a different correction type in accordance with the recalibration timing, and recalibrating the first spectral image and the second spectral image according to the changed correction type; and generating a second fused image that fuses the recalibrated first spectral image and the recalibrated second spectral image in accordance with the refusion timing, and detecting a plurality of gases from the second fused image.
[0017] According to another aspect of the present embodiment, the recording medium is a computer-readable recording medium on which a program for performing a spectroscopic image signal processing method is recorded.
[0018] According to another aspect of the present embodiment, the computer program is a computer program stored on a computer-readable recording medium to perform a spectroscopic image signal processing method. Effects of the invention
[0019] As described above, according to the embodiments of the present invention, there is an effect of providing a spectroscopic image signal processing device mountable on an unmanned vehicle capable of detecting multiple gases such as carbon dioxide and sulfur dioxide by separating a mid-wavelength infrared signal and a long-wavelength infrared signal through a beam splitter, spectrally analyzing the mid-wavelength infrared signal and the long-wavelength infrared signal, controlling the correction type and correction timing for correcting each spectrally analyzed image, and fusing the corrected images into a fused image.
[0020] Even if an effect is not explicitly mentioned herein, the effects described in the following specification and other effects that are expected by the technical features of the present invention and can be easily inferred are treated as described in the specification of the present invention. Brief explanation of the drawing
[0021] FIG. 1 is a flowchart illustrating a spectral image signal processing method according to one embodiment of the present invention. FIG. 2 is a block diagram illustrating a spectroscopic image signal processing device according to one embodiment of the present invention. FIG. 3 is a diagram illustrating an infrared band processed by a spectroscopic image signal processing device according to one embodiment of the present invention. FIG. 4 is a diagram illustrating the basic operation of a spectroscopic image signal processing device according to one embodiment of the present invention. Figure 5 is a diagram illustrating the vignette phenomenon of an image captured by a conventional spectroscopic camera device. FIGS. 6 to 8 are drawings illustrating the mechanical structure of a spectroscopic image signal processing device according to one embodiment of the present invention. FIG. 9 is a diagram illustrating a spectral image processed by a spectral image signal processing device according to one embodiment of the present invention. FIG. 10 is a diagram illustrating an infrared path separated by a spectroscopic image signal processing device according to one embodiment of the present invention. FIGS. 11 to 13 are drawings illustrating a diffraction spectroscopic grating and simulation verification applied to a spectroscopic image signal processing device according to one embodiment of the present invention. FIG. 14 is a diagram illustrating a lens structure and a linear variable filter applied to a spectroscopic image signal processing device according to one embodiment of the present invention. FIG. 15 is a diagram illustrating a signal processing module applied to a spectroscopic image signal processing device according to one embodiment of the present invention. FIG. 16 is a diagram illustrating a spectroscopic image signal processing device detecting gas according to one embodiment of the present invention. FIG. 17 is a diagram illustrating a fused image obtained by fusing spectral images using a spectral image signal processing device according to one embodiment of the present invention. FIG. 18 is a diagram illustrating an image corrected by a spectral image signal processing device according to one embodiment of the present invention. FIG. 19 is a diagram illustrating the heat treatment of an optical system applied to a spectroscopic image signal processing device according to one embodiment of the present invention. FIG. 20 is a diagram illustrating temperature / thermal deformation according to altitude of an optical system applied to a spectroscopic image signal processing device according to one embodiment of the present invention. FIG. 21 is a drawing illustrating equipment for inspecting a spectroscopic image signal processing device according to one embodiment of the present invention. FIG. 22 is a drawing illustrating a lightweight structure applied to a spectroscopic image signal processing device according to one embodiment of the present invention. Specific details for implementing the invention
[0022] Hereinafter, in describing the present invention, detailed descriptions of related known functions are omitted if they are deemed obvious to a person skilled in the art and could unnecessarily obscure the essence of the invention, and some embodiments of the present invention are described in detail through exemplary drawings.
[0023] FIG. 1 is a flowchart illustrating a spectral image signal processing method according to one embodiment of the present invention.
[0024] The spectral image signal processing method can be performed by a spectral image signal processing device, and each operation performed by the spectral image signal processing device can be performed individually or the operations can be performed in combination.
[0025] In S10, a step is performed to receive an infrared signal, separate it into a medium-wavelength infrared signal and a long-wavelength infrared signal, and output them.
[0026] In S20, the step of converting the above-mentioned medium-wavelength infrared signal into a first spectroscopic image is performed.
[0027] In S30, the step of converting the above long-wavelength infrared signal into a second spectroscopic image is performed.
[0028] In S40, at least one correction type is selected from a plurality of correction types, and the first spectral image and the second spectral image are corrected according to the selected correction type.
[0029] In S50, a first fused image is generated by fusing a corrected first spectral image and a corrected second spectral image, and a step of detecting a plurality of gases from the first fused image is performed.
[0030] In S60, the step of measuring the altitude of an unmanned mobile object through an altitude sensor is performed, the first power consumed to correct the first spectral image and the second spectral image is measured, the second power consumed to fuse the corrected first spectral image and the corrected second spectral image is measured, and the third power consumed to detect a plurality of gases from the fused image is measured.
[0031] In S70, a re-correction timing is controlled to change the correction type based on the altitude, the first power, the second power, and the third power, and a re-fusion timing is controlled to fuse the re-corrected first spectral image and the re-corrected second spectral image.
[0032] In S80, in accordance with the recalibration timing, a correction type that corrects the first spectral image and the second spectral image based on the altitude, the first power, the second power, and the third power is changed to a different correction type, and a step is performed to recalibrate the first spectral image and the second spectral image according to the changed correction type.
[0033] In S90, a second fused image is generated by fusing the re-corrected first spectral image and the re-corrected second spectral image in accordance with the re-fusion timing, and a step of detecting a plurality of gases from the second fused image is performed.
[0034] FIG. 2 is a block diagram illustrating a spectroscopic image signal processing device according to one embodiment of the present invention.
[0035] The spectroscopic image signal processing device (10) may include an infrared optical system (100), a medium-wavelength infrared detector (201), a long-wavelength infrared detector (202), and a spectroscopic image processing unit (300). The infrared optical system (100), the medium-wavelength infrared detector (201), the long-wavelength infrared detector (202), and the spectroscopic image processing unit (300) may be interconnected to transmit and receive signals or data.
[0036] The infrared optical system (100) may include a first lens (100), a first mirror (121), a second mirror (122), a third mirror (123), and a beam splitter (130). The first lens (100), the first mirror (121), the second mirror (122), the third mirror (123), and the beam splitter (130) may form an optical path to transmit and receive infrared signals.
[0037] The spectral image processing unit (300) may include a first spectral image processing unit (311), a second spectral image processing unit (312), a correction type selection unit (320), a first fusion image processing unit (330), a correction type change unit (340), a second fusion image processing unit (350), an altitude sensor (360), a power measurement unit (370), and a timing change unit (380). The first spectral image processing unit (311), the second spectral image processing unit (312), the correction type selection unit (320), the first fusion image processing unit (330), the correction type change unit (340), the second fusion image processing unit (350), the altitude sensor (360), the power measurement unit (370), and the timing change unit (380) may be interconnected to transmit and receive data.
[0038] The infrared optical system (100) receives an infrared signal and separates it into a medium-wavelength infrared signal and a long-wavelength infrared signal to output them.
[0039] A lens (100) included in an infrared optical system (100) receives an infrared signal. A first mirror (121) reflects the infrared signal. A second mirror (122) reflects the infrared signal reflected from the first mirror. A third mirror (123) reflects the infrared signal reflected from the second mirror. A beam splitter (130) splits the infrared signal reflected from the third mirror into a medium-wavelength infrared signal and a long-wavelength infrared signal.
[0040] The medium-wavelength infrared detector (201) converts the medium-wavelength infrared signal into a first spectroscopic image. The medium-wavelength infrared detector (201) spectrally analyzes the medium-wavelength infrared signal using a diffraction spectroscopic grating and converts the spectrally analyzed medium-wavelength infrared signal into a first spectroscopic image.
[0041] The long-wavelength infrared detector (202) converts the long-wavelength infrared signal into a second spectral image. The long-wavelength infrared detector (202) spectrally analyzes the long-wavelength infrared signal using a linear variable filter and converts the spectrally analyzed long-wavelength infrared signal into a second spectral image.
[0042] The spectral image processing unit (300) detects a plurality of gases using the first spectral image and the second spectral image.
[0043] The first spectral image processing unit (311) included in the spectral image processing unit (300) analyzes the first spectral image and detects the first gas from the first spectral image. The second spectral image processing unit (312) analyzes the second spectral image and detects the second gas from the second spectral image.
[0044] The correction type selection unit (320) selects at least one correction type among a plurality of correction types and corrects the first spectral image and the second spectral image according to the selected correction type.
[0045] The first fused image processing unit (330) generates a first fused image by fusing the first spectral image and the second spectral image and detects a plurality of gases from the first fused image. The first fused image processing unit (330) generates a first fused image by fusing the first spectral image and the corrected second spectral image and detects a plurality of gases from the first fused image.
[0046] The correction type changing unit (340) can change the correction type that corrects the first spectral image and the second spectral image based on altitude, first power, second power, and third power to another correction type in accordance with the re-correction timing calculated by the timing changing unit (380), and can re-correct the first spectral image and the second spectral image according to the changed correction type.
[0047] The second fusion image processing unit (350) can generate a second fusion image by fusing the re-corrected first spectral image and the re-corrected second spectral image in accordance with the re-fusion timing calculated by the timing change unit (380), and can detect multiple gases from the second fusion image.
[0048] The altitude sensor (360) measures the altitude of the unmanned vehicle. The altitude sensor (360) can measure the altitude using physical principles, such as a barometer.
[0049] The power measuring unit (370) measures the first power consumed in correcting the first spectral image and the second spectral image, measures the second power consumed in fusing the corrected first spectral image and the corrected second spectral image, and measures the third power consumed in detecting multiple gases from the fused image. Power can be calculated by measuring the voltage and current according to the time before and after correction, power can be calculated by measuring the voltage and current according to the time before and after fusion, and power can be calculated by measuring the voltage and current according to the time before and after detection.
[0050] The timing change unit (380) can control a re-correction timing that changes the correction type based on altitude, first power, second power, and third power, and can control a re-fusion timing that fuses the re-corrected first spectral image and the re-corrected second spectral image.
[0051] FIG. 3 is a diagram illustrating an infrared band processed by a spectroscopic image signal processing device according to one embodiment of the present invention.
[0052] The spectroscopic image signal processing device (10) classifies infrared wavelength bands according to atmospheric transmittance. Infrared rays can be classified into near-infrared, short-wavelength infrared, medium-wavelength infrared, long-wavelength infrared, far-infrared, etc.
[0053] Near-infrared rays have a wavelength of 0.75–1.4 μm, a frequency of 214–400 THz, and a temperature of 1800–3600 degrees.
[0054] Short-wavelength infrared rays have a wavelength of 1.4 to 3 μm, a frequency of 100 to 214 THz, and a temperature of 700 to 1800 degrees.
[0055] Mid-wavelength infrared rays have a wavelength of 3 to 8 μm, a frequency of 37 to 100 THz, and a temperature of 90 to 700 degrees.
[0056] Long-wavelength infrared rays have a wavelength of 8–15 μm, a frequency of 20–37 THz, and a temperature of -80–90 degrees.
[0057] Far-infrared rays have a wavelength of 15–1000 μm, a frequency of 0.3–20 THz, and a temperature of -270–-80 degrees.
[0058] The spectroscopic image signal processing device (10) separates the paths of the mid-wavelength infrared signal and the long-wavelength infrared signal by separating the optical paths with a beam splitter.
[0059] FIG. 4 is a diagram illustrating the basic operation of a spectroscopic image signal processing device according to one embodiment of the present invention.
[0060] The spectroscopic image signal processing device (10) may include an imaging and scanning optical system, a spectroscopic optical system, a sensor array, and a data processing and search algorithm. Through the imaging and scanning optical system, the transmittance of mid-wavelength and long-wavelength infrared rays is increased, and the optical system is processed using a material with excellent imaging power. The spectroscopic effect can be enhanced through the manufacturing process of the spectroscopic optical system. The sensor array detects mid-wavelength and long-wavelength infrared rays, and software spectroscopic data processing is performed through the data processing and search algorithm.
[0061] Figure 5 is a diagram illustrating the vignette phenomenon of an image captured by a conventional spectroscopic camera device.
[0062] Far-field spectral images acquired while mounted on an unmanned vehicle may exhibit a vignette phenomenon with dark edges. To solve this problem, in the case of MWIR, dozens of bands are applied using a diffraction spectroscopic grating method to secure the mid-wavelength band, and in the case of LWIR, more than ten band linearly variable filters (LVF) are applied to secure a resolution of 300 nm compared to 3-band (RGB) in the long-wavelength band.
[0063] FIGS. 6 to 8 are drawings illustrating the mechanical structure of a spectroscopic image signal processing device according to one embodiment of the present invention.
[0064] Referring to FIGS. 6 to 8, it is possible to manufacture it in a compact size through a zigzag-shaped optical path.
[0065] FIG. 9 is a diagram illustrating a spectral image processed by a spectral image signal processing device according to one embodiment of the present invention.
[0066] Spectral imaging modules that process spectral images may include PLS (Partial Least Squares), SVM (Support Vector Machine), ANN (Artificial Neural Network), CNN (Convolutional Neural Network), etc.
[0067] FIG. 10 is a diagram illustrating an infrared path separated by a spectroscopic image signal processing device according to one embodiment of the present invention. The infrared path can be separated through a beam splitter.
[0068] FIGS. 11 to 13 are drawings illustrating a diffraction spectroscopic grating and simulation verification applied to a spectroscopic image signal processing device according to one embodiment of the present invention.
[0069] After the incident beam is reflected, it can form a path to pass through a diffraction spectroscopic grating and then back through a reflection optical system.
[0070] To solve the machining problems of V-grooves occurring at the nanoscale, mechanical machining allows for tool shapes such as V or U, and derives the ductile-brittle transition and the resulting optimal process and lubrication conditions depending on parameters and tool conditions.
[0071] Depending on the processing results of the spectroscopic element according to optimal processing conditions, additional processes (secondary correction processing techniques) can be applied by utilizing methods such as irradiating GCIB (Gas Cluster Ion Beam) and ALE (Atomic Layer Etching) to remove burrs generated when necessary and to increase surface roughness.
[0072] It is possible to evaluate the performance of the Modulation Transfer Function (MTF) of the off-axis aspherical optical module, and to verify the situation in which it is mounted on an unmanned vehicle by injecting scattered light onto the optical system and baffle designed through scattered light analysis.
[0073] FIG. 14 is a diagram illustrating a lens structure and a linear variable filter applied to a spectroscopic image signal processing device according to an embodiment of the present invention, FIG. 15 is a diagram illustrating a signal processing module applied to a spectroscopic image signal processing device according to an embodiment of the present invention, FIG. 16 is a diagram illustrating the detection of gas by a spectroscopic image signal processing device according to an embodiment of the present invention, FIG. 17 is a diagram illustrating a fused image obtained by fusing spectroscopic images by a spectroscopic image signal processing device according to an embodiment of the present invention, and FIG. 18 is a diagram illustrating an image obtained by correcting spectroscopic images by a spectroscopic image signal processing device according to an embodiment of the present invention.
[0074] The correction types for correcting spectral images are explained as follows.
[0075] Georeferencing is a correction method that uses pixel-by-pixel GPS data and the sensor's IMU values at the time of capture to correct the geometric position of each pixel, thereby rectifying image distortion caused by the environment during shooting for each line of the image.
[0076] Mosaicking is a correction that overcomes the narrow FOV of line images by aligning multiple line images into a single raster using an algorithm based on overlap and IGM data set at the time of capture for images divided into segments.
[0077] Atmospheric correction is a correction designed to rectify signal errors caused by atmospheric conditions in the spectral signal reflected from the target, by considering environmental variables (solar altitude and azimuth, atmospheric aerosol concentration, and shooting altitude) according to the time of shooting.
[0078] Radiometric Correction is a correction performed to perform analysis that represents the physicochemical properties of an object by converting spectral information received as digital signal intensity into reflectance values. Model-based Radiometric Correction is a correction performed to simulate the deformation that a signal undergoes within the atmosphere by constructing an atmospheric model using the composition of the atmosphere. Empirical Linear Regression is an atmospheric correction that minimizes atmospheric influence by establishing representative ground-level indicators with well-known spectra, linearly regressing their observed and actual values, and applying the regression equation to other pixels within the image to correct for ground-level spectral values.
[0079] Tarps can be utilized to correct infrared spectroscopic images. Ground targets can be set up to perform correction even at high altitudes. The 3-Point Reflectance Tarpaulin System consists of three tarps of a uniform area that maintain specific reflectance values uniformly on the ground surface, ensuring that the tarps are also included in the image during aerial imaging.
[0080] A fused image can be defined as in Equation 1.
[0081] [Mathematical Formula 1]
[0082]
[0083] FI represents a fused image, I1 represents a first spectral image, and I2 represents a second spectral image. K1 and K2 represent a first weight and a second weight, which are the ratios for blending corresponding pixels between multiple spectral images. The blending ratio can be set by referring to a pre-stored transformation table, taking into account altitude, first power, second power, and third power. In particular, the blending ratio can be set by considering the size of the diffraction spectroscopic grating and the number of bands of the linear variable filter. This can improve detection accuracy.
[0084] The recalibration timing can be defined as in Equation 2.
[0085] [Mathematical Formula 2]
[0086]
[0087] T1 represents the recalibration timing, A represents the measured altitude, P1 represents the measured first power consumed to correct the first and second spectral images, K3 is a third weight that adjusts the numerical ratio between altitude and first power, and C1 is a parameter that converts the numerical unit to which altitude and first power are applied into a time unit.
[0088] The refusion timing can be defined as in Equation 3.
[0089] [Mathematical Formula 3]
[0090]
[0091] T2 represents the refusion timing, A represents the measured altitude, P2 represents the measured second power consumed to fuse the corrected first spectral image and the corrected second spectral image, and P3 represents the third power consumed to detect multiple gases from the first fused image. K4 is a fourth weight that adjusts the numerical ratio between the power and altitude obtained by synthesizing the second power and the third power at a certain ratio, and K5 and K6 are fifth and sixth weights that represent the ratio between the second power and the third power. C2 is a parameter that converts the numerical unit between the power and altitude obtained by synthesizing the second power and the third power at a certain ratio into a time unit.
[0092] The re-detection timing can be defined as in Equation 4.
[0093] [Mathematical Formula 4]
[0094]
[0095] T3 represents the re-detection timing, and P7 and K8 are the 7th and 8th weights representing the ratio between the re-correction timing and the re-fusion timing.
[0096] The spectral image processing unit calculates a re-correction timing based on altitude and a first power, and in accordance with the re-correction timing, changes the correction type that corrects the first spectral image and the second spectral image based on altitude, a first power, a second power, and a third power to another correction type, and can re-correct the first spectral image and the second spectral image according to the changed correction type.
[0097] The spectral image processing unit can calculate a refusion timing based on altitude, second power, and third power, and generate a second fused image by fusing the re-corrected first spectral image and the re-corrected second spectral image in accordance with the refusion timing.
[0098] The spectroscopic image processing unit calculates a re-detection timing based on the re-correction timing and the re-fusion timing, and can detect multiple gases from the second fusion image in accordance with the re-detection timing.
[0099] As shown in Equations 1 to 4, the mid-wavelength / long-wavelength infrared signal acquired by being mounted on an unmanned vehicle is converted into a spectral image, and the timing for correction, fusion, and detection are dynamically set in three stages considering altitude and power consumption, so it was easy to see that the power consumption required to analyze the spectral image can be improved (approximately 5~10%).
[0100] FIG. 19 is a diagram illustrating heat treatment of an optical system applied to a spectroscopic image signal processing device according to one embodiment of the present invention, FIG. 20 is a diagram illustrating temperature / thermal deformation according to altitude of an optical system applied to a spectroscopic image signal processing device according to one embodiment of the present invention, and FIG. 21 is a diagram illustrating equipment for inspecting a spectroscopic image signal processing device according to one embodiment of the present invention.
[0101] It is possible to develop equipment for analyzing the material characteristics of high-temperature heat-treated optical components according to environmental changes and measuring and inspecting free-form reflectors.
[0102] FIG. 22 is a drawing illustrating a lightweight structure applied to a spectroscopic image signal processing device according to one embodiment of the present invention.
[0103] This shows an example of a frame structure with some space cut away to reduce the weight of an optical system mounted on an unmanned vehicle.
[0104] The spectroscopic image signal processing device (10) includes a processing unit, a storage unit, and a communication bus.
[0105] The processing unit can control the operation of the spectroscopic image signal processing device (10). For example, the processing unit can execute one or more programs stored in the storage unit. One or more programs may include one or more computer-executable instructions, and the computer-executable instructions may be configured to cause the spectroscopic image signal processing device (10) to perform operations according to exemplary embodiments when executed by the processing unit.
[0106] The storage unit is configured to store computer-executable instructions or program code, program data and / or other suitable forms of information. Computer-executable instructions or program code, program data and / or other suitable forms of information may also be provided through an input / output interface or a communication interface. A program stored in the storage unit includes a set of instructions executable by a processing unit. In one embodiment, the storage unit may be a memory (volatile memory such as random access memory, non-volatile memory, or a suitable combination thereof), one or more magnetic disk storage devices, optical disk storage devices, flash memory devices, or other forms of storage media that are accessed by the spectroscopic image signal processing device (100) and capable of storing desired information, or a suitable combination thereof.
[0107] The communication bus interconnects various other components of the spectroscopic image signal processing device (10).
[0108] The spectroscopic image signal processing device (10) may also include one or more input / output interfaces and one or more communication interfaces that provide interfaces for one or more input / output devices. The input / output interfaces and communication interfaces are connected to a communication bus. An input / output device (not shown) may be connected to other components of the spectroscopic image signal processing device (100) through the input / output interfaces.
[0109] The spectroscopic image signal processing device (10) may be implemented in a logic circuit by hardware, firmware, software, or a combination thereof, and may be implemented using a general-purpose or specific-purpose computer. The device may be implemented using a hardwired device, a field programmable gate array (FPGA), an application-specific integrated circuit (ASIC), etc. Additionally, the device may be implemented as a system on chip (SoC) including one or more processors and controllers.
[0110] The spectroscopic image signal processing device (10) may be installed in a computing device or server equipped with hardware elements in the form of software, hardware, or a combination thereof. The computing device or server may refer to various devices including, in whole or in part, a communication device such as a communication modem for communicating with various devices or wired / wireless communication networks, a memory for storing data for executing a program, and a microprocessor for executing a program to perform calculations and commands.
[0111] Although FIG. 1 describes each process as being executed sequentially, this is merely an illustrative description, and a person skilled in the art may modify and adapt the process in various ways without departing from the essential characteristics of the embodiment of the present invention, such as changing the order described in FIG. 1, executing one or more processes in parallel, or adding other processes.
[0112] Operations according to the embodiments may be implemented in the form of program instructions that can be executed through various computer means and recorded on a computer-readable medium. A computer-readable medium refers to any medium that participates in providing instructions to a processor for execution. A computer-readable medium may include program instructions, data files, data structures, or a combination thereof. Examples include magnetic media, optical recording media, memory, etc. Computer programs may be distributed over networked computer systems, and computer-readable code may be stored and executed in a distributed manner. Functional programs, code, and code segments for implementing the embodiments may be easily deduced by programmers skilled in the art to which the embodiments belong.
[0113] These embodiments are intended to explain the technical concept of the present embodiment, and the scope of the technical concept of the present embodiment is not limited by these embodiments. The scope of protection of the present embodiment shall be interpreted by the claims below, and all technical concepts within an equivalent scope shall be interpreted as being included within the scope of rights of the present embodiment. Explanation of the symbols
[0114] 10: Spectroscopic imaging signal processing device 100: Infrared optical system 110: First lens 121: First mirror 122: Second mirror 123: Third mirror 130: Beam Splitter 201: Medium-wavelength infrared detector 202: Long-wavelength infrared detector 300: Spectral image processing unit 311: 1st Spectral Image Processing Unit 312: 2nd Spectral Image Processing Unit 320: Correction Type Selection 330: 1st Fusion Image Processing Unit 340: Correction Type Change Section 350: 2nd Fusion Image Processing Unit 360: Altitude sensor 370: Power measurement unit 380: Timing change section
Claims
Claim 1 A spectroscopic image signal processing device comprises: an infrared optical system that receives an infrared signal and separates it into a mid-wavelength infrared signal and a long-wavelength infrared signal for outputting; a mid-wavelength infrared detector that converts the mid-wavelength infrared signal into a first spectroscopic image; a long-wavelength infrared detector that converts the long-wavelength infrared signal into a second spectroscopic image; and a spectroscopic image processing unit that detects a plurality of gases using the first spectroscopic image and the second spectroscopic image, wherein the spectroscopic image processing unit comprises: a correction type selection unit that selects at least one correction type among a plurality of correction types and corrects the first spectroscopic image and the second spectroscopic image according to the selected correction type; and a first fusion image processing unit that generates a first fusion image by fusing the corrected first spectroscopic image and the corrected second spectroscopic image and detects a plurality of gases from the first fusion image, and wherein the spectroscopic image processing unit comprises an altitude sensor that measures the altitude of an unmanned mobile object; A spectral image signal processing device comprising a power measuring unit that measures a first power consumed in correcting the first spectral image and the second spectral image, measures a second power consumed in fusing the corrected first spectral image and the corrected second spectral image, and measures a third power consumed in detecting a plurality of gases from the first fused image. Claim 2 delete Claim 3 A spectral image signal processing device according to claim 1, wherein the spectral image processing unit controls a re-correction timing that changes the correction type based on the altitude, the first power, the second power, and the third power, and controls a re-fusion timing that fuses the re-corrected first spectral image and the re-corrected second spectral image. Claim 4 A spectral image signal processing device according to claim 3, wherein the spectral image processing unit includes a correction type changing unit that, in accordance with the re-correction timing, changes the correction type for correcting the first spectral image and the second spectral image based on the altitude, the first power, the second power, and the third power to another correction type, and re-corrects the first spectral image and the second spectral image according to the changed correction type. Claim 5 A spectral image signal processing device according to claim 4, wherein the spectral image processing unit comprises a second fusion image processing unit that generates a second fusion image by fusing a re-corrected first spectral image and a re-corrected second spectral image in accordance with the re-fusion timing and detects a plurality of gases from the second fusion image. Claim 6 A method for processing spectral image signals comprises: receiving an infrared signal and separating it into a mid-wavelength infrared signal and a long-wavelength infrared signal for output; converting the mid-wavelength infrared signal into a first spectral image; converting the long-wavelength infrared signal into a second spectral image; selecting at least one correction type among a plurality of correction types and correcting the first spectral image and the second spectral image according to the selected correction type; generating a first fused image by fusing the corrected first spectral image and the corrected second spectral image and detecting a plurality of gases from the first fused image; measuring the altitude of an unmanned vehicle through an altitude sensor; measuring a first power consumed in correcting the first spectral image and the second spectral image, measuring a second power consumed in fusing the corrected first spectral image and the corrected second spectral image, and measuring a third power consumed in detecting a plurality of gases from the first fused image; and based on the altitude, the first power, the second power, and the third power, the A method for processing spectral image signals, comprising: a step of controlling a re-correction timing that changes a correction type and controlling a re-fusion timing that fuses a re-corrected first spectral image and a re-corrected second spectral image; a step of changing the correction type that corrects the first spectral image and the second spectral image based on the altitude, the first power, the second power, and the third power to a different correction type in accordance with the re-correction timing, and re-correcting the first spectral image and the second spectral image according to the changed correction type; and a step of generating a second fused image by fusing the re-corrected first spectral image and the re-corrected second spectral image in accordance with the re-fusion timing and detecting a plurality of gases from the second fused image. Claim 7 A computer-readable recording medium having a program for performing the spectroscopic image signal processing method described in paragraph 6. Claim 8 A computer program stored on a computer-readable recording medium to perform the spectroscopic image signal processing method described in paragraph 6.
Citation Information
Patent Citations
Infrared and visible light video fusion method and fusion system adaptive to distance
CN105354851A
Apparatus and method for measuring gas
KR1020140074059A
Fusion dual IR camera and image fusion algorithm using LWIR and SWIR
KR1020190076188A
Apparatuses, systems, and methods for determining gas emssion rate detection sensitivity and gas flow speed using remote gas concentration measurements
US20230221219A1
Three-band infrared detector supporting different-speed image motion compensation function and aerial camera
CN116519146A
