Sensing system, information processing device, sensing method and program

The sensing system addresses specular reflection challenges by using multiple sensors to detect electromagnetic waves at different angles and wavelengths, enabling accurate image generation despite high specular reflection.

JP7580269B2Active Publication Date: 2024-11-11PASCO CORP
View PDF 4 Cites 0 Cited by

Patent Information

Application Number
JP2020219382
Authority / Receiving Office
JP · JP
Patent Type
Patents
Current Assignee / Owner
Filing Date
2020-12-28
Publication Date
2024-11-11
Estimated Expiration
2040-12-28

AI Technical Summary

Technical Problem

Sensing systems face challenges in accurately observing surfaces such as water bodies or areas with specular reflection, where sunlight is highly reflected, leading to inaccurate observations due to the intensity of specularly reflected light overpowering diffused light.

Method used

A sensing system installed on a flying object uses multiple sensors to detect electromagnetic waves at different angles and wavelengths, determining if the signals are due to specular reflection, and adjusts pixel values based on signals from different angles to generate accurate output images.

Benefits of technology

The system effectively compensates for specular reflection by using multiple sensors to generate accurate output images even in conditions with high specular reflection, ensuring reliable observations.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 0007580269000001
    Figure 0007580269000001
  • Figure 0007580269000002
    Figure 0007580269000002
  • Figure 0007580269000003
    Figure 0007580269000003
Patent Text Reader

Abstract

To provide a sensing system, information processing device, sensing method and program which can properly perform observation even when mirror reflection occurs.SOLUTION: A sensing system comprises: a first sensor which generates a first signal according to the electromagnetic wave in a first wavelength region incident from a first direction; a second sensor which generates a second signal according to the electromagnetic wave in a second wavelength region incident from the first direction; a third sensor which generates a third signal according to the electromagnetic wave in the first wavelength region incident from the second direction; determination means which determines whether or not the electromagnetic wave corresponding to the first signal is caused by mirror reflection on the basis of the second signal; and generation means which sets a pixel value based on the first signal when the electromagnetic wave corresponding to the first signal is not caused by the mirror reflection, sets a pixel value on the basis of the third signal generated according to the electromagnetic wave from the same ground position as the electromagnetic wave corresponding to the first signal when the electromagnetic wave corresponding to the first signal is caused by the mirror reflection, and generates an output image.SELECTED DRAWING: Figure 2
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] The present invention relates to a sensing system, an information processing device, a sensing method, and a program. [Background technology]

[0002] Conventionally, sensing systems that use electromagnetic waves such as visible light and near-infrared light to observe the vicinity of the ground surface from aircraft and the like are known. A sensing system using visible light observes the vicinity of the ground surface by acquiring the intensity of light such as sunlight reflected on the ground surface with an optical sensor mounted on the flying object. For example, Patent Document 1 discloses a sensing device that detects electromagnetic waves from a relatively moving observation target to observe the observation target. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] International Publication No. 2015 / 068395 Summary of the Invention [Problem to be solved by the invention]

[0004] Typically, sunlight is diffused in various directions on the earth's surface, and a sensing system detects a portion of the diffused light using an optical sensor. On the other hand, when observing the surface of water such as a river, lake, or ocean, sunlight may be specularly reflected by the water surface. Since the light specularly reflected by the water surface has an intensity far greater than the intensity of light diffused by the riverbed or ocean bottom, when specular reflection occurs, the sensing system may not be able to perform correct observations. In addition, when observing the ground, a similar problem may occur if the road is frozen or if artificial objects such as solar panels are installed.

[0005] The present invention has been made to solve the above-mentioned problems, and aims to provide a sensing system, an information processing device, a sensing method, and a program that are capable of appropriate observation even when specular reflection occurs. [Means for solving the problem]

[0006] A sensing system according to the present invention is a sensing system installed on a flying object, comprising: a first sensor that is provided to be capable of detecting electromagnetic waves in a first wavelength range that are incident from a first direction forming a first pitch angle downward with respect to a traveling direction of the flying object and that sequentially generates a first signal in response to the detected electromagnetic waves; a second sensor that is provided to be capable of detecting electromagnetic waves in a second wavelength range different from the first wavelength range that are incident from the first direction and that sequentially generates a second signal in response to the detected electromagnetic waves; and a third sensor that is provided to be capable of detecting electromagnetic waves in the first wavelength range that are incident from a second direction forming a second pitch angle downward with respect to the traveling direction of the flying object and that sequentially generates a third signal in response to the detected electromagnetic waves. The image generating device has a determination means for determining whether or not an electromagnetic wave corresponding to a first signal generated sequentially is due to specular reflection, based on a second signal corresponding to the first signal; and a generation means for generating and outputting an output image by setting a pixel value of a pixel corresponding to the first signal based on the first signal when it is determined that the electromagnetic wave corresponding to the first signal generated sequentially is not due to specular reflection, and setting a pixel value of a pixel corresponding to the first signal based on a third signal generated in response to an electromagnetic wave from the same ground position as the electromagnetic wave corresponding to the first signal when it is determined that the electromagnetic wave corresponding to the first signal generated sequentially is due to specular reflection.

[0007] In addition, in the sensing system of the present invention, it is preferable that the sensing system further has a matching means for matching, among the first signals and third signals generated sequentially based on the difference in angle between the first direction and the second direction, the first signals and third signals generated in response to electromagnetic waves from the same ground position, and the generating means identifies, based on the matching result by the matching means, the third signal generated in response to electromagnetic waves from the same ground position as the electromagnetic waves corresponding to the first signal.

[0008] In addition, it is preferable that the sensing system of the present invention further has an acquisition means for acquiring information regarding the position and attitude of the flying object, and the matching means matches the first signal and the third signal further based on the position and attitude of the flying object.

[0009] In the sensing system according to the present invention, the first wavelength range is preferably a visible wavelength range, and the second wavelength range is preferably a near-infrared wavelength range.

[0010] In addition, in the sensing system of the present invention, it is preferable that the sensing system further has a calculation means for calculating the reflectance of the electromagnetic wave corresponding to the first signal at the object to be observed based on a second signal corresponding to the first signal that is generated sequentially, and the determination means determines whether or not the electromagnetic wave corresponding to the first signal is due to specular reflection based on the reflectance.

[0011] In addition, in the sensing system of the present invention, it is preferable that the second sensor is further configured to be capable of detecting electromagnetic waves incident from above the flying object, and the calculation means corrects the reflectance at the object to be observed based on a second signal corresponding to the electromagnetic waves incident from above the flying object.

[0012] The information processing device according to the present invention detects the first signal, the second signal and a corresponding signal from a sensing device having: a first sensor that is installed on a flying object and is arranged to be able to detect electromagnetic waves in a first wavelength range that are incident from a first direction that forms a predetermined pitch angle downward with respect to the traveling direction of the flying object and sequentially generates a first signal according to the detected electromagnetic waves; a second sensor that is arranged to be able to detect electromagnetic waves in a second wavelength range different from the first wavelength range that are incident from the first direction and sequentially generates a second signal according to the detected electromagnetic waves; and a third sensor that is arranged to be able to detect electromagnetic waves in the first wavelength range that are incident from the second direction that forms a pitch angle different from the predetermined pitch angle downward with respect to the traveling direction of the flying object and sequentially generates a third signal according to the detected electromagnetic waves. The image generating device has an acquisition means for acquiring three signals, a determination means for determining whether or not an electromagnetic wave corresponding to a first signal generated sequentially is due to specular reflection based on a second signal corresponding to the first signal, and a generation means for generating and outputting an output image by setting a pixel value of a pixel corresponding to the first signal based on the first signal when it is determined that the electromagnetic wave corresponding to the sequentially generated first signal is not due to specular reflection, and setting a pixel value of a pixel corresponding to the first signal based on a third signal generated in response to an electromagnetic wave from the same ground position as the electromagnetic wave corresponding to the first signal when it is determined that the electromagnetic wave corresponding to the sequentially generated first signal is due to specular reflection.

[0013] A sensing method according to the present invention includes a computer that detects first signals, from a first sensor that is installed on a flying object and is capable of detecting electromagnetic waves in a first wavelength range that are incident from a first direction that forms a predetermined pitch angle downward with respect to a traveling direction of the flying object and that sequentially generates a first signal in response to the detected electromagnetic waves, a second sensor that is capable of detecting electromagnetic waves in a second wavelength range different from the first wavelength range that are incident from the first direction and that sequentially generates a second signal in response to the detected electromagnetic waves, and a third sensor that is capable of detecting electromagnetic waves in the first wavelength range that are incident from the second direction that forms a pitch angle different from the predetermined pitch angle downward with respect to the traveling direction of the flying object and that sequentially generates a third signal in response to the detected electromagnetic waves. The method includes acquiring a second signal and a third signal, determining whether or not the electromagnetic waves corresponding to the sequentially generated first signal are due to specular reflection based on the second signal corresponding to the first signal, and when it is determined that the electromagnetic waves corresponding to the sequentially generated first signal are not due to specular reflection, setting a pixel value of a pixel corresponding to the first signal based on the first signal, and when it is determined that the electromagnetic waves corresponding to the sequentially generated first signal are due to specular reflection, setting a pixel value of a pixel corresponding to the first signal based on a third signal generated in response to electromagnetic waves from the same ground position as the electromagnetic waves corresponding to the first signal, thereby generating and outputting an output image.

[0014] The program according to the present invention is a program for controlling a computer, and includes a first sensor that is installed on a flying object and is capable of detecting electromagnetic waves in a first wavelength range that are incident from a first direction that forms a predetermined pitch angle downward with respect to a traveling direction of the flying object, and that sequentially generates a first signal according to the detected electromagnetic waves, a second sensor that is capable of detecting electromagnetic waves in a second wavelength range different from the first wavelength range that are incident from the first direction, and that sequentially generates a second signal according to the detected electromagnetic waves, and a third sensor that is capable of detecting electromagnetic waves in the first wavelength range that are incident from a second direction that forms a pitch angle different from the predetermined pitch angle downward with respect to the traveling direction of the flying object, and that sequentially generates a third signal according to the detected electromagnetic waves. The method causes a computer to acquire a second signal and a third signal, determine whether or not the electromagnetic waves corresponding to the first signal are due to specular reflection based on the second signal corresponding to the sequentially generated first signal, and if it is determined that the electromagnetic waves corresponding to the sequentially generated first signal are not due to specular reflection, set a pixel value of the pixel corresponding to the first signal based on the first signal, and if it is determined that the electromagnetic waves corresponding to the sequentially generated first signal are due to specular reflection, set a pixel value of the pixel corresponding to the first signal based on a third signal generated in response to electromagnetic waves from the same ground position as the electromagnetic waves corresponding to the first signal, thereby generating and outputting an output image. Effect of the Invention

[0015] The sensing system, information processing device, sensing method, and program according to the present invention are capable of performing appropriate observation even when specular reflection occurs on the observation target. [Brief description of the drawings]

[0016] [Figure 1] 1 is a diagram illustrating an example of a schematic configuration of a sensing system 1. FIG. [Diagram 2] 2 is a schematic diagram for explaining the internal configuration of the sensing device 2. FIG. [Diagram 3] 2 is a schematic diagram for explaining light incident on the sensing device 2. FIG. [Figure 4]2 is a block diagram showing a schematic configuration of a sensing device 2. FIG. [Diagram 5] 13 is a diagram illustrating an example of a data structure of sensor data D. [Figure 6] FIG. 2 is a block diagram showing a schematic configuration of an analysis device 3. [Figure 7] FIG. 11 is a flow diagram showing an example of the flow of an analysis process. [Figure 8] 13A and 13B are schematic diagrams for explaining association. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0017] Hereinafter, various embodiments of the present invention will be described with reference to the drawings. It should be noted that the technical scope of the present invention is not limited to these embodiments, but covers the inventions described in the claims and their equivalents.

[0018] FIG. 1 is a diagram showing an example of a schematic configuration of a sensing system 1 according to the present invention. The sensing system 1 includes a sensing device 2 and an analysis device 3. The sensing device 2 is a platform including a plurality of sensors, such as optical sensors, for observing an observation target, and a housing incorporating each sensor. The sensing device 2 is mounted on an air vehicle 4, such as a UAV (Unmanned Aerial Vehicle) or a helicopter. The sensing device 2 detects electromagnetic waves, such as visible light or near-infrared light, from an observation target below the air vehicle 4 while the air vehicle 4 is flying along a linear flight path at a preset altitude, and generates sensor data. In this embodiment, the sensing device 2 detects electromagnetic waves in the visible wavelength range and the near-infrared wavelength range. In this embodiment, the visible wavelength range and the near-infrared wavelength range refer to wavelength bands of 340 nm to 850 nm and 700 nm to 2500 nm, respectively. The visible wavelength range and the near-infrared wavelength range are examples of a first wavelength range and a second wavelength range, respectively.

[0019] The analysis device 3 is an information processing device such as a server or a PC (Personal Computer), etc. The analysis device 3 receives and displays the output image generated by the sensing device 2, making it available for use by a user.

[0020] Fig. 2 is a schematic diagram for explaining the internal configuration of the sensing device 2. The sensing device 2 has a first lens 201, a second lens 202, a diffuser 203, an optical fiber bundle 204, a third lens 205, a swing mirror 206, a first spectrometer 207, a second spectrometer 208, a first sensor 209, a second sensor 210, and a third sensor 211. In Fig. 2, arrows A1, A2, and A3 indicate the traveling (front-back) direction, left-right direction, and up-down direction of the flying object 4, respectively.

[0021] The first lens 201 and the second lens 202 are condensing lenses, and condense electromagnetic waves (light) incident from below in the up-down direction A3 toward the optical fiber bundle 204. The first lens 201 and the second lens 202 are telephoto lenses. The first lens 201 and the second lens 202 are provided so as to be able to condense electromagnetic waves from different positions in the left-right direction A2. As the first lens 201 and the second lens 202, a cylindrical lens extending in the left-right direction A2, or a plurality of lenses arranged along the left-right direction A2 so that their optical axes are parallel, is used. The first lens 201 is provided so as to be able to condense electromagnetic waves incident from a first direction that forms a first pitch angle downward with respect to the traveling direction A1 of the flying object 4. The second lens 202 is provided so as to be able to condense electromagnetic waves incident from a second direction that forms a second pitch angle different from the first pitch angle downward with respect to the traveling direction A1 of the flying object 4.

[0022] For example, the first pitch angle is set to 90°, and the second pitch angle is set in the range of 50° to 85° or 95° to 130°. In this case, the first direction is a substantially vertical direction, and the second direction is a direction tilted forward or backward with respect to the vertical direction. If the difference between the first pitch angle and the second pitch angle is too small, specular reflected light may be incident on both the first lens 201 and the second lens 202, and defects in the observation results may not be corrected. Also, if the difference between the first pitch angle and the second pitch angle is too large, the observed object may appear differently between the first lens 201 and the second lens 202, and the electromagnetic waves incident from each lens may not correspond appropriately. Therefore, it is preferable that the difference between the first pitch angle and the second pitch angle is set in the range of 5° to 40°.

[0023] Diffuser 203 is a cosine corrector that collects electromagnetic waves incident from above flying object 4, particularly from a viewing angle of 180° centered on a direction that is 90° upward relative to the traveling direction A1 of flying object 4, and outputs the waves to optical fiber bundle 204. As diffuser 203, for example, a cosine corrector such as CCSA1 (manufactured by Thorlabs) is used.

[0024] The optical fiber bundle 204 is a multi-core fiber and includes a plurality of optical fibers 204a-t. One end of each of the optical fibers 204a-t is disposed to face the lens surface of the third lens 205. One end of each of the optical fibers 204a-j and one end of each of the optical fibers 204k-t are disposed side by side along the left-right direction A2. One end of each of the optical fibers 204k-t is disposed side by side below each of the optical fibers 204a-j in the up-down direction A3.

[0025] The other ends of the optical fibers 204a-d, f-i are arranged side by side along the left-right direction A2 so as to face the lens surface of the first lens 201. The optical fibers 204a-d, f-i guide the electromagnetic waves incident from the first lens 201 and emit them toward the third lens 205. Similarly, the other ends of the optical fibers 204k-n, p-s are arranged side by side along the left-right direction A2 so as to face the lens surface of the second lens 202. The optical fibers 204k-n, p-s guide the electromagnetic waves incident from the second lens 202 and emit them toward the third lens 205. The number of optical fibers that emit the electromagnetic waves incident from the first lens 201 toward the third lens 205 and the number of optical fibers that emit the electromagnetic waves incident from the second lens 202 toward the third lens 205 are not limited to 8 and may be any number. By increasing the number of optical fibers in the left-right direction A2, it is possible to increase the measurement resolution. In addition, by arranging the optical fibers in an array to increase the number of optical fibers, it is possible to increase the measurement range in one measurement in the traveling direction A1 and improve the measurement efficiency. The mirror may be made of a metal reflector or a foamed resin reflector (for example, ultrafine foamed optical reflector (MCPET) made by Furukawa Electric). Note that a polygon mirror having a regular polygonal prism shape (a regular octagonal prism shape in an embodiment in which the number of optical fibers is eight) may be used instead of the swing mirror.

[0026] The other ends of the optical fibers 204j,t are disposed so as to face the lens surface of the diffuser 203. The optical fibers 204j,t guide the electromagnetic wave incident from the diffuser 203 and emit it toward the third lens 205. The other end of the optical fiber 204e is disposed so as to face the first spectrometer 207. The optical fiber 204e guides the electromagnetic wave incident from the third lens 205 and emits it toward the first spectrometer 207. The other end of the optical fiber 204o is disposed so as to face the second spectrometer 208. The optical fiber 204o guides the electromagnetic wave incident from the third lens 205 and emits it toward the second spectrometer 208.

[0027] The third lens 205 is a condenser lens, and condenses the electromagnetic waves incident from one end side of the optical fibers 204a to d, f to n, and p to t toward the swing mirror 206. The third lens 205 is a plano-convex lens.

[0028] The swing mirror 206 is disposed so as to face the optical fiber bundle 204 via the third lens 205. The swing mirror 206 is provided so as to be swingable (rotatable) within a predetermined range by a motor described later, with the vertical direction A3 as a swing axis (rotation axis). The swing mirror 206 is provided so as to be swingable from a first position where the reflection surface reflects electromagnetic waves from one end of the optical fibers 204a, k arranged at one end of the optical fiber bundle 204 toward one end of the optical fibers 204e, o to a second position where the reflection surface reflects electromagnetic waves from one end of the optical fibers 204j, t arranged at the other end of the optical fiber bundle 204 toward one end of the optical fibers 204e, o. The swing mirror 206 alternately repeats swinging (rotation) from the first position to the second position and swinging (rotation) from the second position to the first position at a constant period.

[0029] The first spectrometer 207 splits the incident electromagnetic wave into an electromagnetic wave in the visible wavelength range and an electromagnetic wave in the near-infrared wavelength range, and emits them respectively toward the first sensor 209 and the second sensor 210. The second spectrometer 208 splits the incident electromagnetic wave into an electromagnetic wave in the visible wavelength range and an electromagnetic wave in the near-infrared wavelength range, and emits the electromagnetic wave in the visible wavelength range toward the third sensor 211. That is, the electromagnetic waves in each wavelength range output from the first spectrometer 207 and the second spectrometer 208 include electromagnetic waves incident from below the flying object 4 and electromagnetic waves incident from above the flying object 4.

[0030] The first sensor 209 is provided to be able to detect electromagnetic waves in the visible wavelength range emitted from the first spectrometer 207, and generates and outputs a first signal in sequence in accordance with the detected electromagnetic waves at regular intervals. The first sensor 209 generates the first signal so that the signal value increases as the intensity of the detected electromagnetic waves increases. The second sensor 210 is provided to be able to detect electromagnetic waves in the near-infrared wavelength range emitted from the first spectrometer 207, and generates and outputs a second signal in sequence in accordance with the detected electromagnetic waves at regular intervals. The second sensor 210 generates the second signal so that the signal value increases as the intensity of the detected electromagnetic waves increases. The third sensor 211 is provided to be able to detect electromagnetic waves in the visible wavelength range emitted from the second spectrometer 208, and generates and outputs a third signal in sequence in accordance with the detected light at regular intervals. The third sensor 211 generates the third signal so that the signal value increases as the intensity of the detected electromagnetic waves increases.

[0031] For example, a C12880MA (manufactured by Hamamatsu Photonics) which is a spectrophotometer compatible with the visible light range is used as the first sensor 209 and the third sensor 211. For example, a C14486GA (manufactured by Hamamatsu Photonics) which is a spectrophotometer compatible with the near-infrared range is used as the second sensor 210.

[0032] 3 is a schematic diagram for explaining light incident on the sensing device 2. The first lens 201 receives sunlight reflected from an area S1 extending in the left-right direction A2 on the observation target F, which is located on a straight line extending in the first direction relative to the sensing device 2. The optical fibers 204a-d, f-i facing the first lens 201 receive sunlight reflected from corresponding areas S11-S18 in the area S1, which are respectively concentrated and incident on the optical fibers 204a-d, f-i facing the first lens 201. Similarly, the second lens 202 receives sunlight reflected from an area S2 extending in the left-right direction A2 on the observation target F, which is located on a straight line extending in the second direction relative to the sensing device 2. The optical fibers 204k-n, p-s facing the second lens 202 receive sunlight reflected from corresponding areas S21-S28 in the area S2, which are respectively concentrated and incident on the optical fibers 204k-n, p-s facing the second lens 202.

[0033] The size of each of the regions S1 and S2 is determined based on the distance from the lens to the observation target F and the focal length of the lens. When the first direction is the vertical direction, the second direction is a direction inclined with respect to the vertical direction, and the observation target F is a plane, the distance from the second lens 202 to the region S2 is greater than the distance from the first lens 201 to the region S1. As will be described later, the electromagnetic wave collected from the second lens 202 is used to replace the first signal generated according to the electromagnetic wave collected from the first lens 201. Therefore, the size of the region S2 needs to be determined to be the same as the size of the region S1. In order to make the size of the region S2 the same as the size of the region S1, the focal length of the second lens 202 is determined to be cosθ times the focal length of the first lens 201. Here, θ is the difference between the first pitch angle θ1 and the second pitch angle θ2, that is, the size of the angle between the first direction and the second direction.

[0034] On the other hand, as shown in FIG. 2, sunlight is incident on the diffuser 203, and the sunlight collected by the diffuser 203 is incident on the optical fibers 204j and 204t.

[0035] The electromagnetic waves emitted from the optical fibers 204a-d, f-j, k-n, p-t are focused toward the swing mirror 206 by the third lens 205. As described above, one end of each of the optical fibers 204k-t is arranged below each of the optical fibers 204a-j in the vertical direction A3, and the swing mirror 206 swings around the vertical direction A3 as a swing axis. With the swing of the swing mirror 206, electromagnetic waves from any pair of the optical fibers 204a-d, f-j and the optical fibers 204k-n, p-t arranged in the vertical direction A3 are selectively reflected toward the optical fiber 204e and the optical fiber 204o.

[0036] The electromagnetic waves emitted from the optical fiber 204e and the optical fiber 204o are incident on the first spectrometer 207 and the second spectrometer 208, respectively, and are split into electromagnetic waves in the visible wavelength range and electromagnetic waves in the near-infrared wavelength range. The electromagnetic waves in the visible wavelength range and the near-infrared wavelength range split by the first spectrometer 207 are incident on the first sensor 209 and the second sensor 210, respectively, and the electromagnetic waves in the visible wavelength range split by the second spectrometer 208 are incident on the third sensor 211. As a result, the electromagnetic waves in the visible wavelength range and the near-infrared wavelength range from the region S11, the region S12, the region S13, the region S14, the region S15, the region S16, the region S17, the region S18 or the diffuser 203 are selectively and sequentially incident on the first sensor 209 and the second sensor 210. Further, electromagnetic waves in the visible wavelength range from the area in area S2 corresponding to the area corresponding to the electromagnetic waves incident on the first sensor 209 or from the diffuser 203 are selectively and sequentially incident on the third sensor 211. The electromagnetic waves from each area and the diffuser 203 are sequentially incident on the first sensor 209, the second sensor 210, and the third sensor 211 while the swing mirror 206 swings between the first position and the second position, and the electromagnetic waves from each area and the diffuser 203 are repeatedly incident on the first sensor 209, the second sensor 210, and the third sensor 211 every time the swing mirror 206 swings between the first position and the second position.

[0037] That is, the first sensor 209 is provided so as to be capable of detecting electromagnetic waves in the visible wavelength range incident from a first direction. The second sensor 210 is provided so as to be capable of detecting electromagnetic waves in the near-infrared wavelength range incident from a first direction. The third sensor 211 is provided so as to be capable of detecting electromagnetic waves in the visible wavelength range incident from a second direction.

[0038] 4 is a block diagram showing a schematic configuration of the sensing device 2. In addition to the above-mentioned configuration, the sensing device 2 further includes a motor 221, a GPS (Global Positioning System) sensor 222, a six-axis sensor 223, a first communication unit 224, a first storage unit 225, a first processing unit 230, and the like.

[0039] The motor 221 swings the swing mirror 206 in accordance with a control signal from the first processing unit 230 .

[0040] The GPS sensor 222 includes an antenna and a receiving circuit for receiving radio waves from GPS satellites. The GPS sensor 222 receives the radio waves from the GPS satellites via the antenna in the receiving circuit, generates position information indicating the position of the sensing device 2 and time information indicating absolute time, and supplies the position information and time information to the first processing unit 230.

[0041] The six-axis sensor 223 includes an acceleration sensor and a gyro sensor corresponding to three mutually orthogonal axial directions. The six-axis sensor 223 acquires the acceleration and angular velocity of the sensing device 2 in the three axial directions at a predetermined period, generates attitude information (attitude change amount) and a movement vector (relative position) related to the attitude of the sensing device 2, and supplies them to the first processing unit 230.

[0042] The first communication unit 224 includes a communication interface circuit, such as a wired LAN (Local Area Network) or a USB (Universal Serial Bus), to enable the sensing device 2 to communicate with the analysis device 3. The first communication unit 224 transmits data supplied from the first processing unit 230 to the analysis device 3, and supplies data received from the analysis device 3 to the first processing unit 230.

[0043] The first storage unit 225 includes a semiconductor memory such as a Synchronous Dynamic Random Access Memory (SDRAM). The first storage unit 225 stores computer programs, data, and the like used in processing by the first processing unit 230. The computer programs may be installed in the storage device 140 from a computer-readable portable recording medium using a known setup program or the like. The portable recording medium is, for example, a compact disc read only memory (CD-ROM) or a digital versatile disc read only memory (DVD-ROM). The first storage unit 225 stores sensor data and the like as data. The sensor data will be described in detail later.

[0044] The first processing unit 230 includes, for example, a central processing unit (CPU). The first processing unit 230 may include, for example, a large scale integration (LSI), an application specific integrated circuit (ASIC), a digital signal processor (DSP), a field programmable gate array (FPGA), or the like. The first processing unit 230 is connected to the first sensor 209, the second sensor 210, the third sensor 211, the motor 221, the GPS sensor 222, the six-axis sensor 223, the first communication unit 224, the first storage unit 225, and the like, and controls each of these units. The first processing unit 230 reads a program stored in the first storage unit 225, and operates according to the read program, thereby functioning as an acquisition unit 231, a correlation unit 232, a calculation unit 233, a determination unit 234, an output unit 235, and a generation unit 236.

[0045] 5 is a diagram showing an example of the data structure of the sensor data D stored in the first storage unit 225. The sensor data D stores time, position, attitude, first signal value, second signal value, third signal value, etc. in association with one another. The sensor data D is generated by the first processing unit 230 storing in the first storage unit 225 each piece of information supplied to the first processing unit 230 at a predetermined period from each sensor while the flying object 4 carrying the sensing device 2 is flying, in association with one another.

[0046] The time is the time when each piece of information shown in the sensor data D was acquired, and is calculated based on the time information generated by the GPS sensor 222. The position is data indicating the position of the sensing device 2 at each time, calculated based on the position information generated by the GPS sensor 222 and the movement vector generated by the six-axis sensor 223, and includes, for example, latitude and longitude. The attitude is data indicating the attitude of the sensing device 2 at each time, calculated based on the attitude information generated by the six-axis sensor 223, and includes the acceleration and angular velocity in three axial directions. The first signal value is a set of signal values ​​of each first signal generated by the first sensor 209 in response to electromagnetic waves in the visible wavelength range from each of the optical fibers 204a-d, f-j at each time. The second signal value is a set of signal values ​​of each second signal generated by the second sensor 210 in response to electromagnetic waves in the near-infrared wavelength range from each of the optical fibers 204a-d, f-j at each time. The third signal values ​​are a set of signal values ​​of the third signals generated by the third sensor 211 in response to electromagnetic waves in the visible wavelength range from the optical fibers 204k to n, p to t at each time.

[0047] 6 is a block diagram showing a schematic configuration of the analysis device 3. The analysis device 3 includes a second communication unit 324, a second storage unit 325, a display unit 326, a second processing unit 330, and the like.

[0048] The second communication unit 324 includes a communication interface circuit such as a wired LAN or a USB to enable the analysis device 3 to communicate with the sensing device 2. The second communication unit 324 transmits data supplied from the second processing unit 330 to the sensing device 2, and supplies data received from the sensing device 2 to the second processing unit 330.

[0049] The second storage unit 325 includes, for example, a semiconductor memory. The second storage unit 325 stores computer programs, data, and the like used in the processing by the second processing unit 330. The programs and data may be installed in the storage device 140 from a computer-readable portable recording medium such as a CD-ROM or DVD-ROM using a known setup program or the like.

[0050] The display unit 326 has a display configured with liquid crystal, organic EL (Electro-Luminescence) or the like, and an interface circuit that outputs image data to the display. The display unit 326 displays various types of information on the display according to instructions from the second processing unit 330.

[0051] The second processing unit 330 includes, for example, a CPU. The second processing unit 330 may include an LSI, an ASIC, a DSP, an FPGA, etc. The second processing unit 330 is connected to the second communication unit 324, the second storage unit 325, the display unit 326, etc., and controls each of these units.

[0052] 7 is a flow diagram showing an example of the flow of observation processing executed by the sensing device 2. The observation processing is periodically executed while the sensing device 2 is mounted on the flying object 4 and flying.

[0053] First, the acquiring means 231 drives the motor 221 to swing the swing mirror 206, and acquires a first signal, a second signal, and a third signal generated in response to electromagnetic waves from each optical fiber from the first sensor 209, the second sensor 210, and the third sensor 211, respectively. The acquiring means 231 also acquires position information, time information, attitude information, and a movement vector from the GPS sensor 222 and the six-axis sensor 223, respectively. The acquiring means 231 associates the signal value of each acquired signal with the current time, current position, and current attitude calculated from the position information, time information, attitude information, and movement vector, and stores them in the first storage unit 225 as sensor data D (S101).

[0054] The time it takes for the swing mirror 206 to swing between the first position and the second position is shorter than the cycle at which the GPS sensor 222 generates position information and time information. Every time the GPS sensor 222 generates position information, the acquisition means 231 synchronizes the swing timing of the swing mirror 206 with the timing at which the GPS sensor 222 generates position information. The acquisition means 231 calculates, at the time indicated by the latest time information acquired from the GPS sensor 222, a sum of the number of times the swing mirror 206 has been swung since acquiring the time information and the product of the time it takes for the swing mirror 206 to swing between the first position and the second position, as the current time.

[0055] Moreover, the period in which the six-axis sensor 223 generates the attitude information and the movement vector is shorter than the period in which the GPS sensor 222 generates the position information and the time information. The acquisition means 231 synchronizes the timing at which the six-axis sensor 223 generates the attitude information and the movement vector with the timing at which the GPS sensor 222 generates the position information and the time information every time the GPS sensor 222 generates the position information. The acquisition means 231 calculates a position moved from the position indicated in the latest position information acquired from the GPS sensor 222 by the movement vector acquired from the six-axis sensor 223 after acquiring the time information. The acquisition means 231 also considers that the sensing device 2 is moving at a constant speed from the last time the six-axis sensor 223 generates the movement vector to the next time it generates it. The acquisition means 231 estimates the current position of the sensing device 2, which has moved from the calculated position to the current time, by linear interpolation of the movement vectors generated before and after the current time or by interpolation using a Kalman filter or the like.

[0056] Furthermore, the sensing device 2 sets an initial value for the attitude of the sensing device 2 at takeoff, and calculates the attitude by sequentially adding to the initial value the amount of attitude change indicated in the attitude information generated up to the present by the six-axis sensor 223. Furthermore, the acquisition means 231 considers the sensing device 2 to be moving at a constant speed during the period from when the six-axis sensor 223 last generated attitude information to when it next generates it. The acquisition means 231 estimates the current attitude, which has changed from the calculated attitude to the current time, by linear interpolation of the attitude information generated before and after the current time, or by interpolation using a Kalman filter or the like.

[0057] Next, the association means 232 associates the first signal and the third signal stored in the first storage unit 225 (step S102). Based on the difference in angle between the first direction and the second direction, the association means 232 associates the first signal and the third signal generated in response to electromagnetic waves from the same ground position among the first and third signals generated sequentially. When the second direction is a direction tilted forward with respect to the first direction, the association means 232 associates the newly acquired first signal with the previously acquired third signal. On the other hand, when the second direction is a direction tilted backward with respect to the first direction, the association means 232 associates the newly acquired third signal with the previously acquired first signal.

[0058] 8(a) and (b) are schematic diagrams for explaining the correspondence. FIG. 8(a) shows an example in which the second direction is tilted backward with respect to the first direction, and the flying object 4 (sensing device 2) flies parallel to the observation target F at a predetermined altitude h. In this case, a predetermined position P on the observation target F is located in the first direction with respect to the sensing device 2 at time t1, and is located in the second direction with respect to the sensing device 2 at time t2 a predetermined time later. Therefore, the first signal generated by the first sensor 209 at time t1 and the third signal generated by the third sensor 211 at time t2 are generated in response to electromagnetic waves from the same ground position. As shown in FIG. 8(a), the distance between the position of the flying object 4 at time t1 and the position of the flying object 4 at time t2 is h·tanθ. The correspondence means 232 refers to the position information, extracts the first signal and the third signal for which the above relationship holds from the sensor data D, and associates them with each other. In this way, the association means 232 associates the first signal with the third signal based on the position of the flying object 4. This allows the association means 232 to appropriately associate the first signal with the third signal even if the flight position (height) of the flying object 4 changes during flight.

[0059] The association means 232 may associate the first signal with the third signal further based on the attitude information. FIG. 8(b) shows an example in which the second direction is tilted backward with respect to the first direction, and the flying object 4 (sensing device 2) flies at a predetermined altitude h at an incline with respect to the observation target F. In this case, the distance between the position of the flying object 4 at time t1 and the position of the flying object 4 at time t2 is h·(tan(θ+φ2)-tanφ1). Here, φ1 is a pitch angle based on the first direction at time t1, and φ2 is a pitch angle based on the first direction at time t2. The association means 232 refers to the attitude information and calculates the pitch angles φ1 and φ2 for each sensor data D. The association means 232 refers to the position information, extracts the first signal and the third signal that satisfy the above relationship from the sensor data D, and associates them with each other. In this way, the association means 232 associates the first signal with the third signal further based on the attitude of the flying object 4. This allows the association means 232 to appropriately associate the first signal with the third signal even if the attitude of the flying object 4 changes during flight.

[0060] The association means 232 may also associate the first and third signals using an image processing technique. For example, the association means 232 executes a pattern matching process between an image including pixels whose pixel values ​​are the first signal values ​​of a plurality of first signals and an image including pixels whose pixel values ​​are the third signal values ​​of a plurality of third signals, and associates the first and third signals that correspond to pixels that match each other. Such association using an image processing technique is preferably used in terrestrial observations where the observation target includes feature points such as land features.

[0061] Next, the calculation means 233 calculates the reflectance of the electromagnetic wave corresponding to the first signal at the observation target based on the second signal corresponding to the first signal that is generated sequentially (S103). The calculation means 233 calculates the reflectance of the first signal that is newly associated with the third signal. The calculation means 233 calculates the reflectance by normalizing the second signal value stored in the first storage unit 225. The reflectance R is calculated by the following formula (1). R = (S - Sd) / (Sw - Sd) (1) Here, S is the second signal value, Sd is the dark current value, and Sw is the second signal value (hereinafter sometimes referred to as the reference value) when the electromagnetic wave is reflected by a standard white board placed on the observation target F.

[0062] As the dark current value Sd, for example, a second signal value detected in advance in a state where no light enters through the first lens 201 and the second lens 202 (for example, a state where caps are attached to the first lens 201 and the second lens 202) is used. Also, as the reference value Sw, a second signal value measured in advance by placing a standard white plate on the observation target F, such as when the flying object 4 starts flying, is used.

[0063] The reference value Sw may be corrected based on the intensity of light from above the flying object 4 in flight. The reference value Sw changes depending on the intensity of incident light (mainly sunlight from above) on the standard white plate. Therefore, it is preferable that the reference value Sw is measured simultaneously with the second signal value S of the observation target F. However, in order to do so, it is necessary to install a standard white plate over a wide range of the observation target F along the flight path of the flying object 4, which is difficult to realize. Therefore, the calculation means 233 corrects the reference value Sw measured in advance based on the second signal value corresponding to the electromagnetic waves from above the flying object 4, which are incident from the diffuser 203 and are measured simultaneously with the electromagnetic waves from the observation target F.

[0064] The reference value Sw is corrected by the following formula (2). Sw = Sw0 × (Sz / Sz0) (2) Here, Sw0 is a reference value measured in advance, Sz0 is a second signal value corresponding to the electromagnetic waves from above the flying object 4, measured in advance and simultaneously with Sw0, and Sz is a second signal value measured simultaneously with the electromagnetic waves from the observation target F. That is, the calculation means 233 corrects the reflectance at the observation target based on the second signal value corresponding to the electromagnetic waves from above the flying object 4. This allows the calculation means 233 to accurately determine whether the electromagnetic waves are due to specular reflection or not, even if the weather or the like changes during the flight of the flying object 4 and the intensity of the sunlight changes.

[0065] Next, the determination means 234 determines whether or not the electromagnetic wave corresponding to the first signal is due to specular reflection based on the reflectance calculated by the calculation means 233 (S104). If the reflectance is equal to or greater than a predetermined threshold, the determination means 234 determines that the electromagnetic wave corresponding to the first signal is due to specular reflection, and if the reflectance is less than the predetermined threshold, the determination means 234 determines that the electromagnetic wave corresponding to the first signal is not due to specular reflection. Usually, the reflectance of light diffused (not specularly reflected) in the observation target F is much smaller than the reflectance of a standard white board, whereas the reflectance of specularly reflected light may exceed the reflectance of a standard white board. Therefore, the predetermined threshold is set to, for example, 1.

[0066] Electromagnetic waves in the near-infrared wavelength range are more easily absorbed by water than electromagnetic waves in the visible wavelength range. Therefore, by using a wavelength band in the near-infrared range to determine specular reflection, the sensing device 2 can make a more accurate determination without being affected by diffusion in water or reflection on the riverbed or ocean bottom. In particular, by using electromagnetic waves in the wavelength range of 700 nm to 800 nm or 1500 nm to 1600 nm as electromagnetic waves in the near-infrared range, the sensing device 2 can more accurately determine whether the electromagnetic waves are due to specular reflection.

[0067] When it is determined that the electromagnetic wave corresponding to the first signal is not due to specular reflection (S104-No), the output means 235 outputs the first signal (S105). The output means 235 outputs the first signal, for example, by storing the first signal in the first storage unit 225 in association with the position information corresponding to the first signal.

[0068] On the other hand, if it is determined that the electromagnetic wave corresponding to the first signal is due to specular reflection (S104-No), the output means 235 outputs a third signal generated in response to an electromagnetic wave from the same terrestrial position as the electromagnetic wave corresponding to the first signal (S106). The output means 235 identifies a third signal generated in response to an electromagnetic wave from the same terrestrial position as the electromagnetic wave corresponding to the first signal based on the association result by the association means 232 in step S102. The output means 235 outputs the third signal by storing the third signal in the first storage unit 225 in association with the position information corresponding to the first signal.

[0069] Next, the first processing unit 230 determines whether or not the processing has been performed for a predetermined number of first signals (S107). If the processing has not yet been performed for the predetermined number of first signals (S107-No), the first processing unit 230 repeats the processing of S101 to S107.

[0070] On the other hand, if the processing has been performed for a predetermined number of first signals (S107-Yes), the generating means 236 generates an output image based on the first and third signals output by the output means 235 (S108), and ends the series of steps. The generating means 236 generates an output image including a plurality of pixels corresponding to the position information corresponding to each first signal. When the generating means 236 determines that the electromagnetic waves corresponding to the sequentially generated first signals are not due to specular reflection, the generating means 236 sets the pixel value of the pixel corresponding to the first signal based on the first signal. On the other hand, when the generating means 236 determines that the electromagnetic waves corresponding to the sequentially generated first signals are due to specular reflection, the generating means 236 sets the pixel value of the pixel corresponding to the first signal based on the third signal generated according to the electromagnetic waves from the same ground position as the electromagnetic waves corresponding to the first signal.

[0071] For example, the generating means 236 sets the pixel value of each pixel to a value obtained by normalizing (converting) the first signal value or the third signal value so that it falls within a predetermined gradation range. Note that the generating means 236 may correct the first signal value or the third signal value based on the dark current value, the reference value, and the signal value corresponding to the electromagnetic wave from above the flying object 4, as in the process of S103, and then set the pixel value of each pixel to a value obtained by normalizing (converting) it so that it falls within a predetermined gradation range.

[0072] When the flight of the flying object 4 is completed, the generation means 236 outputs each generated output image by transmitting it to the analysis device 3 via the first communication unit 224. The analysis device 3 receives the output image from the sensing device 2 via the second communication unit 324, and displays the received output image on the display unit 326, or outputs it by transmitting it to another device via the second communication unit 324.

[0073] The process of S103 may be omitted, and in S104, the determination means 234 may determine whether the electromagnetic wave corresponding to the first signal is due to specular reflection or not based on whether the second signal value exceeds a predetermined threshold. In this case, the determination means 234 may set the predetermined threshold based on the second signal value generated in response to the electromagnetic wave from above the flying object 4. For example, the determination means 234 sets the predetermined threshold so that the greater the intensity of the electromagnetic wave from above, the greater the threshold value becomes. This allows the sensing device 2 to accurately determine whether the electromagnetic wave corresponding to the first signal is due to specular reflection or not without calculating the reflectance.

[0074] Furthermore, in S104 and S105, the output means 235 may output the first signal and the third signal by transmitting them to an external device via the first communication unit 224. In this case, the external device generates an output image based on the output first signal and the third signal.

[0075] In the above description, the sensing device 2 executes the analysis process, but the analysis process may be executed by the analysis device 3. In that case, the second processing unit 330 of the analysis device 3 has each unit that the first processing unit 230 of the sensing device 2 has. The first processing unit 230 of the sensing device 2 executes the process of S101 in FIG. 7 during the flight of the flying object 4, and transmits the sensor data D including the time, position, attitude, first signal, second signal, and third signal to the analysis device 3 via the first communication unit 224 when the flight of the flying object 4 is completed. Meanwhile, in S101 in FIG. 7, the acquisition means 231 acquires the sensor data D including the time, position, attitude, first signal, second signal, and third signal from the sensing device 2 that has completed the flight of the flying object 4 via the second communication unit 324, and stores it in the second storage unit 325. The second processing unit 330 sequentially executes the processes of S102 to S108 for each acquired sensor data D in order from the oldest sensor data D. The generating means 236 outputs the generated output image by displaying it on the display unit 326 or transmitting it to another device via the second communication unit 324.

[0076] The analysis device 3 may be wirelessly connected to the sensing device 2 mounted on the flying object 4, sequentially acquire sensor data while the flying object 4 is flying, and execute observation processing on the acquired data. The analysis device 3 may be wirelessly connected to the sensing device 2 mounted on the flying object 4, sequentially acquire sensor data while the flying object 4 is flying, and execute observation processing on the acquired data.

[0077] In the above description, an example has been shown in which there is one set of optical systems that detects electromagnetic waves in the visible light wavelength range from the second direction, and a third signal related to one direction is acquired. However, the sensing device 2 may define the second direction as a plurality of different directions, include a set of optical systems for the number of directions, and acquire a third signal including a signal related to each direction. In that case, the sensing device 2 may acquire a third signal in four directions by defining the second direction as four directions, i.e., front, back, left, and right, and by using four sets of optical systems with lenses facing each direction, so as to function as an oblique camera, for example. This allows the sensing device 2 to more reliably replace the first signal.

[0078] In addition, in the above explanation, the GPS sensor 222 is exemplified as using GPS as a positioning means for acquiring the position together with the time, however, instead of the GPS 222 sensor, the positioning means may be configured with a sensor that uses other GNSS such as GLONASS (GLObal'naya NAvigatsionnaya Sputnikovaya Sistema), or the positioning means may be configured with a sensor that uses two or more types of GNSS in combination.

[0079] As described above, when the incident electromagnetic wave is the result of specular reflection, the sensing system 1 generates an output image by replacing a signal generated in response to the electromagnetic wave with a signal generated in response to an electromagnetic wave incident from a different direction from the electromagnetic wave. This enables the sensing system 1 to generate an appropriate output image even when specular reflection occurs on the observation target.

[0080] That is, when specularly reflected light is incident on the first sensor 209, the electromagnetic waves diffused in the observation target are buried in the specularly reflected light, and the first signal cannot be used as the observation result, resulting in a loss of the observation result. In this case, the sensing system 1 uses a third signal corresponding to the same point on the observation target instead of the first signal, thereby preventing a loss of the observation result.

[0081] In aerial photography, it is assumed that specular reflection of sunlight can only occur from vertically below when the observation target is horizontal. Therefore, even if electromagnetic waves incident from vertically below are due to specular reflection, electromagnetic waves incident from a different direction are likely not due to specular reflection. Therefore, the sensing system 1 can generate an appropriate output image even if specular reflection occurs on the observation target.

[0082] Although a correction may be considered in which the specular reflection component is estimated from the first signal and then subtracted from it, in that case, it is difficult to estimate the specular reflection component with high accuracy because of fluctuations in the state of water vapor and aerosols in the atmosphere.On the other hand, sensing system 1 performs substitution with observation results that do not actually include specular reflection light, and therefore can generate a correct output image with high accuracy.

[0083] Furthermore, the first sensor 209 and the second sensor 210 detect electromagnetic waves incident on the first lens 201 and the second lens 202, respectively. Since the first lens 201 and the second lens 202 are provided so that their optical axes are inclined relative to each other, specularly reflected light does not normally enter both the first lens 201 and the second lens 202. Therefore, the sensing system 1 can more reliably prevent the occurrence of deficiencies in the observation results.

[0084] It should be understood by those skilled in the art that various changes, substitutions, and modifications can be made to the present invention without departing from the spirit and scope of the present invention. For example, the processes of the above-described parts may be executed in different orders as appropriate within the scope of the present invention. Furthermore, the above-described embodiments and modifications may be combined as appropriate within the scope of the present invention. [Explanation of symbols]

[0085] 2. Sensing equipment 209 First Sensor 210 Second Sensor 211 3rd Sensor 231 Acquisition means 232 Correspondence means 233 Figure out the means 234 Judgment Method 235 Means of contribution 236 Generation means 3 Analysis device

Claims

1. A sensing system installed in an airborne vehicle, a first sensor that is provided to be capable of detecting electromagnetic waves in a first wavelength range that are incident from a first direction that forms a first pitch angle downward with respect to a traveling direction of the flying object, and that sequentially generates first signals corresponding to the detected electromagnetic waves; a second sensor that is provided to be capable of detecting electromagnetic waves in a second wavelength range different from the first wavelength range and that is incident from the first direction, and that sequentially generates second signals in accordance with the detected electromagnetic waves; a third sensor that is provided to be capable of detecting electromagnetic waves in the first wavelength range that are incident from a second direction that forms a second pitch angle different from the first pitch angle downward with respect to the traveling direction of the flying object, and that sequentially generates a third signal according to the detected electromagnetic waves; a determination means for determining whether or not an electromagnetic wave corresponding to a first signal is due to specular reflection, based on a second signal corresponding to the first signal that is sequentially generated; a generating means for generating and outputting an output image by setting a pixel value of a pixel corresponding to the first signal based on the first signal when it is determined that the electromagnetic waves corresponding to the sequentially generated first signals are not due to specular reflection, and setting a pixel value of a pixel corresponding to the first signal based on a third signal generated in response to electromagnetic waves from the same ground position as the electromagnetic waves corresponding to the first signal when it is determined that the electromagnetic waves corresponding to the sequentially generated first signals are due to specular reflection, The difference between the first pitch angle and the second pitch angle is set in a range of 5° to 40°. A sensing system comprising:

2. a correlation unit that correlates, among the first and third signals that are generated sequentially, the first and third signals that are generated in response to electromagnetic waves from the same ground position, based on an angle difference between the first direction and the second direction; The generating means identifies a third signal generated in response to electromagnetic waves from the same ground position as the electromagnetic waves corresponding to the first signal, based on a result of the association by the associating means. The sensing system of claim 1 .

3. The system further includes an acquisition unit for acquiring information regarding a position and an attitude of the flying object, The association means associates the first signal with the third signal further based on a position and an attitude of the flying object. The sensing system of claim 2 .

4. the first wavelength range is a visible wavelength range, The second wavelength range is a near-infrared wavelength range. The sensing system according to any one of claims 1 to 3.

5. a calculation unit that calculates a reflectance of an electromagnetic wave corresponding to a first signal at an observation target based on a second signal corresponding to the first signal that is sequentially generated; The determination means determines whether or not the electromagnetic wave corresponding to the first signal is due to specular reflection based on the reflectance. The sensing system according to any one of claims 1 to 4.

6. The second sensor is further provided to be capable of detecting electromagnetic waves incident from above the flying object, The calculation means corrects the reflectance at the observation target based on a second signal corresponding to an electromagnetic wave incident from above the flying object. The sensing system according to claim 5 .

7. an acquisition means for acquiring the first signal, the second signal, and the third signal from a sensing device having: a first sensor that is installed on the flying object and is adapted to detect electromagnetic waves in a first wavelength range that are incident from a first direction that forms a first pitch angle downward with respect to a traveling direction of the flying object, and that sequentially generates a first signal according to the detected electromagnetic waves; a second sensor that is adapted to detect electromagnetic waves in a second wavelength range different from the first wavelength range that are incident from the first direction, and that sequentially generates a second signal according to the detected electromagnetic waves; and a third sensor that is adapted to detect electromagnetic waves in the first wavelength range that are incident from a second direction that forms a second pitch angle downward with respect to the traveling direction of the flying object, and that sequentially generates a third signal according to the detected electromagnetic waves; a determination means for determining whether or not an electromagnetic wave corresponding to a first signal is due to specular reflection, based on a second signal corresponding to the first signal that is sequentially generated; a generating means for generating and outputting an output image by setting a pixel value of a pixel corresponding to the first signal based on the first signal when it is determined that the electromagnetic waves corresponding to the sequentially generated first signals are not due to specular reflection, and setting a pixel value of a pixel corresponding to the first signal based on a third signal generated in response to electromagnetic waves from the same ground position as the electromagnetic waves corresponding to the first signal when it is determined that the electromagnetic waves corresponding to the sequentially generated first signals are due to specular reflection, The difference between the first pitch angle and the second pitch angle is set in a range of 5° to 40°.

13. An information processing device comprising:

8. The computer the first signal, the second signal, and the third signal are obtained from a first sensor that is installed on the flying object and is capable of detecting electromagnetic waves in a first wavelength range that are incident from a first direction that forms a first pitch angle downward with respect to a traveling direction of the flying object, and that sequentially generates a first signal according to the detected electromagnetic waves; a second sensor that is capable of detecting electromagnetic waves in a second wavelength range different from the first wavelength range that are incident from the first direction, and that sequentially generates a second signal according to the detected electromagnetic waves; and a third sensor that is capable of detecting electromagnetic waves in the first wavelength range that are incident from a second direction that forms a second pitch angle downward with respect to the traveling direction of the flying object, and that sequentially generates a third signal according to the detected electromagnetic waves, determining whether or not the electromagnetic wave corresponding to the first signal is due to specular reflection based on the second signal corresponding to the sequentially generated first signal; generating and outputting an output image by setting a pixel value of a pixel corresponding to the first signal based on the first signal when it is determined that the electromagnetic waves corresponding to the sequentially generated first signals are not due to specular reflection, and setting a pixel value of a pixel corresponding to the first signal based on a third signal generated in response to electromagnetic waves from the same ground position as the electromagnetic waves corresponding to the first signal when it is determined that the electromagnetic waves corresponding to the sequentially generated first signals are due to specular reflection, The difference between the first pitch angle and the second pitch angle is set in a range of 5° to 40°. A sensing method comprising:

9. A program for controlling a computer, the first signal, the second signal, and the third signal are obtained from a first sensor that is installed on the flying object and is capable of detecting electromagnetic waves in a first wavelength range that are incident from a first direction that forms a first pitch angle downward with respect to a traveling direction of the flying object, and that sequentially generates a first signal according to the detected electromagnetic waves; a second sensor that is capable of detecting electromagnetic waves in a second wavelength range different from the first wavelength range that are incident from the first direction, and that sequentially generates a second signal according to the detected electromagnetic waves; and a third sensor that is capable of detecting electromagnetic waves in the first wavelength range that are incident from a second direction that forms a second pitch angle downward with respect to the traveling direction of the flying object, and that sequentially generates a third signal according to the detected electromagnetic waves, determining whether or not the electromagnetic wave corresponding to the first signal is due to specular reflection based on the second signal corresponding to the sequentially generated first signal; causing the computer to generate and output an output image by setting a pixel value of a pixel corresponding to the first signal based on the first signal when it is determined that the electromagnetic waves corresponding to the sequentially generated first signals are not due to specular reflection, and setting a pixel value of a pixel corresponding to the first signal based on a third signal generated in response to electromagnetic waves from the same ground position as the electromagnetic waves corresponding to the first signal when it is determined that the electromagnetic waves corresponding to the sequentially generated first signals are due to specular reflection; The difference between the first pitch angle and the second pitch angle is set in a range of 5° to 40°. A program characterized by:

Citation Information

Patent Citations

  • Analytical method for growth condition of crop

    JP2006250827A

  • Spectroscopic camera and spectroscopic image processing method

    JP2014163721A

  • Ground surface water content information acquisition method, ground surface water content information acquisition device and program

    JP2016212000A

  • Sensing device and sensing method

    WO2015068395A1