Luminance distribution estimation system, luminance distribution estimation method, and luminance distribution estimation program

Abstract

[Problem] To easily estimate the celestial sphere luminance distribution of sunlight. [Solution] Provided is a luminance distribution estimation system 1000 for estimating the luminance distribution of the sky, said system comprising: an image sensor 211 in which a plurality of sensor elements 211a are provided side by side in a planar manner; a shielding object 213 that is placed above a detection target surface of the image sensor, shields the light flux incident toward the image sensor to generate a shadow on the detection target surface, and has a known object surface reflectance; and a luminance estimation unit 220 that assumes that there are many virtual light sources discretely present in the sky and estimates the luminance of each virtual light source contributing to the pixel values of the sensor elements on the basis of the pixel values observed by at least a portion of the plurality of sensor elements and the incidence effectiveness indicating whether or not the light from each of the virtual light sources is incident on the sensor elements.

Description

Luminance distribution estimation system, luminance distribution estimation method, and luminance distribution estimation program

[0001] The present invention relates to a luminance distribution estimation system, a luminance distribution estimation method, and a luminance distribution estimation program.

[0002] When analyzing crop growth conditions from images, it is necessary to consider the influence of sunlight. This is particularly important when analyzing based on specific wavelengths, such as multispectral images. In particular, the distribution of reflectance characteristics of crop canopies changes depending on weather conditions, such as sunny or cloudy. This is because the direct and scattered components of incident sunlight, i.e., the luminance distribution, change depending on the weather. Therefore, there is a need for a technology that can easily estimate the celestial luminance distribution of sunlight in the area captured by an image.

[0003] In this regard, Patent Document 1 discloses a multi-sensor irradiance estimation device that is mounted on a drone or the like and estimates or determines irradiance using an irradiance detection device having multiple optical sensors with different orientations. Patent Document 2 discloses an optical sensor that includes a detection unit that has multiple detection elements that output the intensity of received light as a detection signal and a light-shielding unit that determines the angle of incidence of light with respect to the multiple detection elements.

[0004] However, none of these inventions allowed for a simple estimation of the celestial luminance distribution of sunlight.

[0005] Patent Publication No. 2020-515809 Patent Publication No. 2018-004317

[0006] One of the objectives is to easily estimate the celestial luminance distribution of sunlight.

[0007] In order to achieve the above-mentioned object, a luminance distribution estimation system according to one aspect of the present invention is a luminance distribution estimation system that estimates the luminance distribution of the sky, and includes an image sensor having a plurality of sensor elements arranged side by side in a plane, a shielding object with a known object surface reflectance that is placed above a detection target surface of the image sensor and blocks light beams incident toward the image sensor, thereby generating shadows on the detection target surface, and a luminance estimation unit that assumes that there are a large number of virtual light sources discretely located in the sky and estimates the luminance of each of the virtual light sources that contributes to the pixel values ​​of the sensor elements based on pixel values ​​observed by at least some of the plurality of sensor elements and incidence effectiveness that indicates whether light beams from each of the virtual light sources are incident on the sensor elements.

[0008] The image sensor may further include a diffuse transmission plate disposed on the detection target surface of the image sensor, and the obstructing object may be disposed above the diffuse transmission plate.

[0009] The occluding object may be a sphere.

[0010] The image sensor may further include a tilt sensor for acquiring a tilt direction and a tilt amount of the image sensor, and the brightness estimation unit may estimate the brightness of each of the virtual light sources in consideration of the tilt direction and the tilt amount.

[0011] The device may further include a notification unit that issues an alert when the celestial luminance distribution cannot be estimated based on the measurement results from the image sensor.

[0012] In order to achieve the above-mentioned object, a luminance distribution estimation method according to another aspect of the present invention is a luminance distribution estimation method that estimates the luminance distribution of the sky using a luminance distribution estimation system that includes an image sensor having a plurality of sensor elements arranged side by side in a plane, and a shielding object with a known object surface reflectance that is placed above a detection target surface of the image sensor and blocks light beams incident toward the image sensor, thereby generating shadows on the detection target surface.The method assumes that there are a large number of discrete virtual light sources in the sky, and includes a luminance estimation step that estimates the luminance of each of the virtual light sources that contributes to the pixel values ​​of the sensor elements based on pixel values ​​observed by at least some of the plurality of sensor elements and incidence effectiveness that indicates whether light beams from each of the virtual light sources are incident on the sensor elements.

[0013] To achieve the above object, according to yet another aspect of the present invention, there is provided a luminance distribution estimation program for estimating the luminance distribution of the sky using a luminance distribution estimation system including an image sensor having a plurality of sensor elements arranged side by side in a plane, and a shielding object having a known object surface reflectance, mounted above a detection target surface of the image sensor and shielding a light beam incident on the image sensor to generate a shadow on the detection target surface, the program causing a computer to execute a luminance estimation instruction to estimate the luminance of each of the virtual light sources contributing to the pixel value of the sensor element based on pixel values ​​observed by at least some of the plurality of sensor elements and incidence validity, which indicates whether a light beam from each of the virtual light sources is incident on the sensor element. Note that the computer program can be provided by downloading via a network such as the Internet, or by recording it on various computer-readable recording media such as a CD-ROM.

[0014] The celestial luminance distribution of sunlight can be easily estimated.

[0015] 8A is a schematic perspective view of a drone equipped with at least a portion of the configuration of a luminance distribution estimation system according to a first embodiment of the present invention. It is a diagram showing an example of a schematic hardware configuration diagram of the drone and a sensor unit included in the luminance distribution estimation system. It is a diagram showing the sensor unit, in which (a) is a schematic right side view and (b) is a schematic plan view. It is a conceptual diagram showing how a light beam from a light source passes through a diffuse transmissive plate and is incident on an image sensor in the sensor unit, in which (a) is a conceptual diagram showing how the light beam from the light source is incident on a one-dimensional image sensor, (b) is a conceptual diagram showing how the sensor element receives the light beam of FIG. 8A, and (c) is a conceptual diagram showing how the light beam is incident from a position with a large angle of incidence. It is a functional block diagram of the drone, the sensor unit, a user interface device, and a luminance distribution estimation device. It is a conceptual diagram of the celestial sphere luminance distribution estimated by the luminance distribution estimation system, in which (a) is a conceptual diagram showing how direct light and scattered light are incident from regions on the celestial sphere, and (b) is a conceptual diagram showing how the celestial sphere is likened to a geodesic dome. It is a flowchart showing how the luminance distribution estimation system estimates the celestial sphere luminance distribution.

[0016] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS The following detailed description will be described with reference to the drawings. All figures are illustrative. In the following detailed description, for purposes of explanation, certain specific details are set forth in order to provide a thorough understanding of the disclosed embodiments. However, the embodiments are not limited to these specific details. Additionally, well-known structures and devices are schematically depicted to simplify the drawings.

[0017] First, we will explain the configuration of a drone that can be equipped with a part of the brightness distribution estimation system according to the present invention. In this specification, a drone refers to any flying object with multiple rotors, regardless of the power source (electricity, prime mover, etc.) or the control method (wireless or wired, autonomous flight type or manual flight type, etc.).

[0018] As shown in FIG. 1 , rotors 101-1a, 101-1b, 101-2a, 101-2b, 101-3a, 101-3b, 101-4a, and 101-4b (also referred to as rotors) are used to fly the drone 100. Eight rotors (four sets of two-stage rotors) are provided, taking into consideration a balance between flight stability, aircraft size, and power consumption. Each rotor 101 is positioned on all four sides of the housing 103 of the drone 100 by an arm extending from the housing 103. Specifically, rotors 101-1a and 101-1b are positioned at the rear left of the direction of travel, rotors 101-2a and 101-2b at the front left, rotors 101-3a and 101-3b at the rear right, and rotors 101-4a and 101-4b at the front right. Note that the +x direction in FIG. 1 is the direction of travel for the drone 100.

[0019] A lattice-shaped propeller guard is provided on the outer periphery of each set of rotor blades 101 to prevent the rotor blades 101 from interfering with foreign objects.

[0020] The motors 102-1a, 102-1b, 102-2a, 102-2b, 102-3a, 102-3b, 102-4a, and 102-4b are means (typically electric motors, but may also be engines, etc.) for rotating the rotors 101-1a, 101-1b, 101-2a, 101-2b, 101-3a, 101-3b, 101-4a, and 101-4b, and one motor is provided for each rotor. The motor 102 is an example of a propulsion device. The upper and lower rotors (e.g., 101-1a and 101-1b) in a set and their corresponding motors (e.g., 102-1a and 102-1b) have axes that are on the same straight line and rotate in opposite directions to ensure the drone's flight stability, etc.

[0021] As shown in FIG. 2 , the housing 103 contains, for example, a flight controller 501, which is a component responsible for overall control of the drone 100. Specifically, the flight controller 501 may be an embedded computer including a CPU, memory, related software, and the like. The flight controller 501 controls the rotation speeds of the motors 102-1a, 102-1b, 102-2a, 102-2b, 102-3a, 102-3b, 104-a, and 104-b via control means such as an ESC (Electronic Speed ​​Control) based on input information received from a controller and input information obtained from various sensors (described below), thereby controlling the flight of the drone 100. The actual rotation speeds of the motors 102-1a, 102-1b, 102-2a, 102-2b, 102-3a, 102-3b, 104-a, and 104-b are fed back to the flight controller 501, allowing monitoring of whether the motors are rotating normally. Alternatively, an optical sensor or the like may be provided on the rotor 101 so that the rotation of the rotor 101 is fed back to the flight controller 501.

[0022] The drone 100 also includes a hardware configuration, such as a growth diagnosis camera 502, that measures the field and acquires data for growth diagnosis. The field may be a rice paddy, a field, or the like, that is the subject of imaging by the drone 100. The growth diagnosis camera 502 is, for example, a camera that receives visible light. The growth diagnosis camera 502 may also be a multispectral camera that receives multiple light beams with different wavelengths. The multiple light beams may be light beams in any frequency range, such as red light (wavelength approximately 650 nm) and near-infrared light (wavelength approximately 774 nm). Because red light corresponds to the absorption spectrum of chlorophyll contained in plants, a configuration that receives and captures red light allows for special imaging of plants. Data captured by the growth diagnosis camera 502 is transmitted to an appropriate growth diagnosis device via a WiFi slave function 503.

[0023] The configuration of drone 100 is not limited to the above configuration, and may include other appropriate components such as an obstacle detection sensor, a tank for storing the pesticide to be sprayed on the field, and a spray nozzle.

[0024] The drone 100 is also provided with a sensor unit 20 that mainly measures information for estimating the luminance distribution of the celestial sphere. The sensor unit 20 is mounted in a position where it is not blocked by sunlight irradiated from above, and is exposed, for example, on the top side (+z side) of the drone 100. The sensor unit 20 may be provided, for example, above the motors 102-1a, 102-2a, 102-3a, and 104-a, above the GNSS receiver units 105 arranged in pairs on the left and right, or on the top surface of the housing 103.

[0025] The sensor unit 20 may be connected to the flight controller 501 of the drone 100 via an appropriate interface circuit, or may be independent of the drone 100. When the sensor unit 20 and the flight controller 501 are connected, for example, the sensor unit 20 may be powered by the flight controller 501. Furthermore, the sensor unit 20 may transmit the measurement results to the flight controller 501, and then transmit the measurement results to the luminance distribution estimation device 200 via a WiFi slave function 503 that the drone 100 has. If the sensor unit 20 is connected to the flight controller 501 and can share some of its functions, the sensor unit 20 can be made smaller.

[0026] Overview of sensor unit 20 The sensor unit 20 mainly comprises, as its hardware configuration, an illuminance sensor 21, an inclination sensor 22, a position sensor 23, an orientation sensor 24, and a timing unit 25. Furthermore, when the sensor unit 20 is configured independently of the drone 100, it is appropriately provided with a battery and a communication processing unit for supplying power to the above configuration and transmitting measurement results to the luminance distribution estimation device 200. A configuration in which the sensor unit 20 operates independently of the drone 100 allows it to be mounted on an appropriate drone or moving object, making it highly versatile.

[0027] The illuminance sensor 21 shown in FIGS. 3( a ) and 3 ( b ) is a device that measures the illuminance of a light beam irradiated onto a surface to be measured, and mainly includes an image sensor 211 , a diffuse transmission plate 212 , and a shielding object 213 .

[0028] As shown in Figures 3(a) and (b), the image sensor 211 is a substantially flat member, and in Figure 3(a) a plurality of sensor elements 211a are arranged side by side in a plane on the upper side of the image sensor 211. That is, in Figure 3(a) the upper side of the image sensor 211 is the detection target surface. The sensor elements 211a are photoelectric conversion elements, and each of the sensor elements 211a corresponds to one pixel in the image sensor 211. The plurality of sensor elements 211a measure pixel values ​​of irradiated light beams independently of each other. It is preferable that the sensor elements 211a of the image sensor 211 are exposed on the surface.

[0029] By arranging bandpass filters for various wavelength bands in the image sensor 211 in a mosaic pattern, it is possible to obtain the celestial luminance distribution for each of a plurality of wavelength bands.

[0030] The plurality of sensor elements 211a may be modularized as a sensor package 214. The effects of the present invention can be achieved even if the sensor package 214 is a small component, for example, about 0.5 inches square. However, the size of the sensor package 214 is not limited to this.

[0031] The diffuse transparent plate 212 is a substantially flat plate-shaped member of a size corresponding to the size of the image sensor 211. It is disposed on the detection target surface of the image sensor 211 and diffuses and transmits the light beam from above, guiding it to the image sensor 211. It is desirable that the diffuse transparent plate 212 be located above the image sensor 211 in order to capture light from a light source with a large angle of incidence. Because the brightness on the celestial sphere varies greatly between the sun, blue sky, and clouds, an image sensor with a wide dynamic range would be required if an image were to be formed on an image sensor using a fisheye lens or the like to measure the brightness directly. In this regard, a configuration in which the diffuse transparent plate 212 adjusts the brightness and guides it to the image sensor 211 allows the use of an image sensor with a general dynamic range.

[0032] It is desirable that the diffuse transmitting plate 212 be thin and highly diffusive. The diffuse transmitting plate 212 and the sensor element 211a are close to each other, and it is desirable that they be in close contact with each other. If it is difficult to make them in close contact with each other, it is desirable to insert an imaging optical system such as a relay lens or a fiber bundle made up of many very thin optical fibers as an optical system that transfers the image that appears on the lower surface of the diffuse transmitting plate 212 to the sensor element 211a. A configuration that includes an imaging optical system that has a mirror or a prism and refracts the optical path allows the thickness of the illuminance sensor 21 to be reduced.

[0033] 4A, when light from a light source i is incident on only one point on the upper surface of the diffusely transmitting plate 212, the transmitted light is spread unevenly by the diffusely transmitting plate 212 and reaches the lower surface of the diffusely transmitting plate 212. This figure schematically shows the directly downward component of the light emitted from the lower surface of the diffusely transmitting plate 212. In reality, the light that reaches the lower surface of the diffusely transmitting plate 212 is emitted with a predetermined directionality.

[0034] As shown in FIG. 4B, when the downward component of the light emitted from the lower surface of the diffuse transmission plate 212 is photoelectrically converted by the pixels of the image sensor 211, i.e., the sensor elements 211a, the point spread function H i When the light source i is at infinity, the point spread function H i is shift-invariant. An image obtained from light incident from a light source i onto the entire upper surface of the diffuse transmission plate 212 is obtained by adding a point spread function H i can be thought of as being folded.

[0035] The diffuse transparent plate 212 has the property of blurring the bright and dark image that appears on its upper surface as it is guided to the lower surface. On the other hand, as shown in FIG. 4( c), incident light with a large angle of incidence can be guided to a certain extent to the lower surface of the diffuse transparent plate 212, where it can be received by the image sensor 211. In other words, a configuration including the diffuse transparent plate 212 makes it possible to estimate the luminance distribution of the celestial sphere even when the angle of incidence of sunlight is large. A detailed method for estimating the luminance distribution of the celestial sphere will be described later.

[0036] The shielding object 213 is an object placed above the detection target surface of the image sensor 211, and shields the light beam incident on the image sensor 211 to generate a shadow 213a on the detection target surface. In the figure, a dark shadow 213b is generated by direct light L10. In addition, scattered light L11, which is generated when sunlight is scattered by blue sky or clouds and has a lower brightness than direct light L10, is irradiated onto the shielding object 213 from multiple directions, generating a relatively light shadow 213c.

[0037] The occluding object 213 is an object whose surface reflectance is known. The occluding object 213 is, for example, a sphere, but is not limited to a sphere as long as its shape is known. By configuring the occluding object 213 as a sphere, the shadow 213a on the detection target surface can generate a shadow with gradation on a flat surface at any solar altitude. This ultimately enables more accurate estimation of the celestial luminance distribution.

[0038] Furthermore, the occluding object 213 is, for example, completely black. With this configuration, the reflectance is 0, which simplifies the luminance calculation described below and allows for accurate estimation of the celestial sphere luminance distribution. Note that the color of the occluding object 213 does not have to be completely black. By setting the reflectance of the occluding object 213 to be greater than 0 and estimating the celestial sphere luminance distribution taking the reflectance into consideration, more accurate estimation can be achieved.

[0039] The shielding object 213 is disposed above the diffuse transmission plate 212. The shielding object 213 may be in close contact with the diffuse transmission plate 212 or may be fixed thereto.

[0040] The diffuse transmission plate 212 and the shielding object 213 may be covered with a cover 215. The cover 215 is, for example, a colorless and transparent hemispherical member that integrally covers the diffuse transmission plate 212 and the shielding object 213. This configuration can prevent the diffuse transmission plate 212 and the shielding object 213 from being soiled or damaged. The shielding object 213 may be fixed to the inner surface of this cover 215 instead of being configured to be in close contact with the diffuse transmission plate 212.

[0041] 2 is a sensor for acquiring the tilt direction and tilt amount of the detection target surface of the image sensor 211 relative to the horizontal. With this configuration, even if the image sensor 211 can tilt, the acquired data can be calibrated taking into account the tilt direction and tilt amount, and the luminance distribution of the celestial sphere can be accurately estimated. In particular, for example, if the image sensor 211 is mounted on the drone 100, the luminance distribution of the celestial sphere can be accurately estimated even if the drone 100 is tilted in the pitch direction or roll direction.

[0042] The position sensor 23 is a sensor that measures the position coordinates of the point where measurement is performed by the illuminance sensor 21, and is configured by, for example, a GNSS receiving unit.

[0043] The orientation sensor 24 is a sensor for acquiring the orientation of the rows and columns of the sensor elements 211a. With this configuration, even when the image sensor 211 is used while being moved, the luminance distribution of the celestial sphere can be accurately estimated by correcting the orientation. In particular, for example, when the image sensor 211 is mounted on the drone 100, the luminance distribution of the celestial sphere can be accurately estimated even when the drone 100 rotates in a yaw direction.

[0044] The timer 25 is configured to acquire the measurement time, and may be configured to store the acquisition time of the pixel values ​​used to estimate the celestial luminance distribution in association with the pixel values.

[0045] In addition, instead of being provided in the sensor unit 2, the tilt sensor 22, the position sensor 23, the orientation sensor 24 and the timing unit 25 may be provided in an appropriate configuration connected to the sensor unit 2, such as the drone 100, and may be configured to be able to record the measured pixel values ​​in correspondence with them.

[0046] Overview of the Luminance Distribution Estimation System The luminance distribution estimation system 1000 shown in FIG. 5 is a system that estimates the luminance distribution of the sky. The luminance distribution estimation system 1000 includes, for example, a sensor unit 20 and a luminance distribution estimation device 200, but may also include a drone 100 or a user interface device 300. The drone 100, the sensor unit 20, the luminance distribution estimation device 200, and the user interface device 300 are communicably connected to each other via a network NW. The luminance distribution estimation device 200 may be configured as hardware, or may be partially or entirely configured on the cloud. The drone 100, the luminance distribution estimation device 200, and the user interface device 300 may be connected to each other wirelessly, or may be partially or entirely connected by wire.

[0047] 5 is an example, and a certain component may include another component, and a functional unit of each component may be possessed by another component. For example, some or all of the functions of the luminance distribution estimation device 200 or the sensor unit 20 may be mounted on the drone 100. Furthermore, some or all of the functions of the luminance distribution estimation device 200 may be mounted on the user interface device 300.

[0048] The user interface device 300 may have, for example, an input unit and a display unit for an operator, and may also have the functions of an operator for the drone 100. The user interface device 300 may also be a personal computer, and information may be input to and displayed on a UI on the web via a web browser installed on the personal computer.

[0049] Functional Units of the Drone 100 The drone 100 has at least a flight control unit 110 and an imaging unit 120 as software resources, as shown in FIG. 5, for example, by a flight controller 501.

[0050] The flight control unit 110 is a functional unit that operates the motor 102 and controls the flight and takeoff and landing of the drone 100. The flight control unit 110 is realized by, for example, a flight controller 501, and controls the flight altitude, flight speed, and flight path to fly the drone 100 above the field.

[0051] The imaging unit 120 is a functional unit that controls the growth diagnosis camera 502 and acquires images of crops growing in the field while the drone 100 is flying above the field. The imaging unit 120 can acquire, for example, multispectral images of the crops growing in the field.

[0052] The sensor unit 20 has, for example, a sensor control unit 201 as software resources, and controls an illuminance sensor 21, a tilt sensor 22, a position sensor 23, a direction sensor 24, a timer unit 25, and the like.

[0053] Functional Units of the Luminance Distribution Estimation Device 200 The luminance distribution estimation device 200 shown in Fig. 5 is a device that estimates the luminance distribution of the celestial sphere based on information acquired by a sensor unit 20. The luminance distribution estimation device 200 includes a calculation unit such as a CPU (Central Processing Unit) for executing information processing, and storage devices such as RAM (Random Access Memory) and ROM (Read Only Memory), and thus includes at least a data acquisition unit 210, a luminance estimation unit 220, a notification unit 230, and a storage unit 250 as software resources.

[0054] The data acquisition unit 210 acquires information acquired by the sensor unit 20 .

[0055] The brightness estimation unit 220 is a functional unit that estimates the celestial luminance distribution based on at least the information acquired by the image sensor 211 .

[0056] Here, a method for estimating the luminance distribution on the celestial sphere will be described in detail. Fig. 6(a) is a conceptual diagram showing a celestial sphere T centered on a measurement target point P, and the appearance of direct light L10 and scattered light L11. The celestial sphere T is a hemisphere centered on the measurement target point P. On the celestial sphere T, there is an area A10 where direct light L10 from the sun is incident toward the measurement target point P, and an area A11 where the direct light is scattered by clouds, the sky, etc., and scattered light L11 is incident toward the measurement target point P from a position other than the sun.

[0057] Therefore, we consider the celestial sphere T as a geodesic dome G, as shown in Figure 6(b), which has the property that triangular patches T1 of equal solid angles are arranged inscribed on the spherical surface, and assume that each triangular patch T1 has a number of discrete virtual light sources. Therefore, the representative point of each light source is the center of gravity of each triangular patch T1. Point P1 in the figure is a point on the geodesic dome G, and is the intersection of the geodesic dome G and the line connecting the center of gravity of the triangular patch T1 and the light source. The luminance estimation unit 220 then estimates the luminance of each virtual light source.

[0058] In the case of a configuration with a diffuse transmission plate 212, the luminance distribution can be calculated as follows, taking advantage of the fact that the output value of the image sensor 211 is expressed as the convolution integral of the incident light and the point spread function. Here, in order to simplify the formulas and schematic diagrams, the explanation will be given using a model in which the dimensions are reduced from three-dimensional space to a two-dimensional plane. In this model, the surfaces of the diffuse transmission plate and the image sensor are reduced to one dimension, and the image sensor is considered to have pixels arranged in one dimension. In addition, all light sources are considered to exist within a two-dimensional plane. Furthermore, the luminance of N light sources i is expressed as L i For M pixels j of the image sensor 211, an index j is also assigned to the points on the upper and lower surfaces of the diffuser / transmitting plate 212 directly above the center point of the pixels j.

[0059] The light source i is considered to be at infinity from the point on the diffuse transmission plate 212, and its position is defined only by the direction to the light source i. The incidence effectiveness, which indicates whether or not the light flux from the light source i is incident on the point j on the diffuse transmission plate 212 due to the shielding object 213, is expressed as V i,j Only at point j on the diffuse transmission plate 212, the luminance L iWhen there is incident light only from the light source i, where j = 1, the light is received by the image sensor 211 through the diffuse transmission plate 212 and output as a point image. The point image is cut out in the range of ±W centered on point j and is expressed as the diffuse transmission point spread function H i,k The incident effectiveness V i,j and the point spread function H i,k can be kept constant by calibration. i,j and the point spread function H i,k According to the configuration in which the above equations are stored in advance, the calculation can be simplified.

[0060] The incidence effectiveness V from the light source i to the point j on the diffuse transmission plate 212 i,j Incident light I taking into account i,j is expressed as the following equation (1). ...(1) In equation (1), c is a constant. The output value of the image sensor 211 at pixel point j directly below point j, i.e., the pixel value E i,j is the incident light I as shown in the following equation (2): i,j and the point spread function H i,k This is the convolution integral of ... (2) Incident light I in the above formula i,j and the point spread function H i,k The convolution integral of the incident effective diffuse transmission image D is given by the following equation (3): i,j It can be summarized as: ... (3) This gives the luminance L of the light source i. i and pixel value E i,j The relationship between these two can be simplified as shown in the following equation (4). ...(4) Pixel value E for all light sources i,j is the luminance of the light source L i This can be reduced to a set of M simultaneous equations with unknowns. ... (5)

[0061] Although the estimation method has been explained above on a two-dimensional plane, it can be naturally extended to three-dimensional space. In this case, the incident effective diffuse transmission image and sensor pixel values ​​can be rearranged row-sequentially into one-dimensional vectors, which can then be incorporated into simultaneous equations.

[0062] Using a sufficient number of sensor elements 211a to solve the simultaneous linear equations, the simultaneous linear equations can be solved as an inverse problem. That is, the luminance estimation unit 220 can estimate the light source luminance distribution on the celestial sphere T that contributes to the pixel values ​​of the sensor elements 211a by estimating N unknown light source luminances. In this embodiment, the luminance estimation unit 220 uses pixel values ​​acquired by all of the sensor elements 211a in the simultaneous linear equations.

[0063] The illuminance sensor 21 of the present invention may be configured without the diffuse transmission plate 212. In this case, the point spread function H i,k By setting the value W representing the range to 0, it is possible to estimate the luminance in a configuration that does not have the diffuse transmission plate 212.

[0064] Furthermore, the illuminance sensor 21 may have a micro-collector with wide-angle directivity disposed on each sensor element 211a of the image sensor 211, instead of the diffuse transmission plate 212. An appropriate wavelength selection filter may be disposed between this on-chip micro-collector and the sensor element 211a. In this case, too, the point spread function H i,k is the same as equation (1) and equation (2) with W=0, and H i,0 Only has value.

[0065] The brightness estimation unit 220 may correct the celestial luminance distribution using the measurement results of the tilt sensor 22 or the orientation sensor 24.

[0066] The brightness estimation unit 220 also stores the celestial luminance distribution in the storage unit 250 in association with the position information obtained by the position sensor 23. This celestial luminance distribution can be used to correct an image of a farm field, such as a multispectral image, taken at the same position.

[0067] According to the above-described configuration, the celestial luminance distribution can be estimated by measuring the shadow created by the occluding object 213 with the image sensor 211 and calculating the direction and luminance of the direct light L10 and scattered light L11 that create the shadow. Consequently, by correcting the multispectral image of the field with the celestial luminance distribution, it is possible to more accurately diagnose the growth of crops growing in the field.

[0068] The notification unit 230 may output the celestial luminance distribution estimated by the luminance estimation unit 220 to the user interface device 300. When the celestial luminance distribution cannot be estimated by the luminance estimation unit 220, the notification unit 230 may issue an alert to that effect to the user interface device 300. The notification unit 230 may detect that the celestial luminance distribution cannot be estimated based on measurement results from one or more of the illuminance sensor 21, tilt sensor 22, position sensor 23, and orientation sensor 24, and issue an alert. The notification unit 230 may issue an alert, for example, when the angle measured by the tilt sensor 22 is within a predetermined range and the illuminance sensor 21 is not pointing upward. The notification unit 230 may also issue an alert when the average illuminance measured by the illuminance sensor 21 is below a predetermined level. This is because it is assumed that the illuminance sensor 21 is not receiving sufficient sunlight.

[0069] Flowchart for Estimating the Celestial Luminance Distribution As shown in FIG. 7, first, the drone 100 flies over the field, measures the field using the sensor unit 20 mounted on the drone 100, and acquires pixel values ​​from each sensor element 211a. The data acquisition unit 210 also acquires pixel values ​​from each sensor element 211a (step S11). Next, the luminance estimation unit 220 estimates the incident effective diffuse transmission image D i,j Using the above, the luminance L i and pixel value E i,j(Step S12). Next, an integral equation is generated that expresses the relationship between the pixel value of the sensor element 211a and the luminance of the contributing virtual light source, taking into account all N virtual light sources (Step S13). The processes of Steps S12 and S13 are repeatedly performed for all the sensor elements 211a to be used, and M equations shown in Equation (5) are formulated (Step S15).

[0070] Next, by solving the inverse problem of the simultaneous equations shown in Equation (5), the luminance L of the virtual light source is calculated. i are estimated (step S16).

[0071] (Technically Significant Effects of the Present Invention) The luminance distribution estimation system according to the present invention makes it possible to easily estimate the celestial luminance distribution of sunlight.

Claims

1. A luminance distribution estimation system for estimating the luminance distribution of the sky, comprising: an image sensor having a plurality of sensor elements arranged side by side in a plane; a shielding object with known object surface reflectance that is placed above a detection target surface of the image sensor and blocks light beams incident on the image sensor to generate shadows on the detection target surface; and a luminance estimation unit that assumes that there are a large number of discrete virtual light sources in the sky and estimates the luminance of each of the virtual light sources that contributes to the pixel values ​​of the sensor elements based on pixel values ​​observed by at least some of the plurality of sensor elements and incidence effectiveness that indicates whether light beams from the virtual light sources are incident on the sensor elements.

2. The luminance distribution estimation system according to claim 1, further comprising a diffuse transmission plate disposed on the detection target surface of the image sensor, and the obstructing object is disposed above the diffuse transmission plate.

3. The luminance distribution estimation system according to claim 1, wherein the occluding object is a sphere.

4. The luminance distribution estimation system according to claim 1, further comprising an inclination sensor that acquires the inclination direction and amount of inclination of the image sensor, and the luminance estimation unit estimates the luminance of each of the virtual light sources taking into account the inclination direction and amount of inclination.

5. The luminance distribution estimation system according to claim 1, further comprising a notification unit that issues an alert when the celestial luminance distribution cannot be estimated based on the measurement results from the image sensor.

6. A luminance distribution estimation method for estimating the luminance distribution of the sky using a luminance distribution estimation system comprising: an image sensor having a plurality of sensor elements arranged side by side in a plane; and a shielding object with known object surface reflectance that is placed above a detection target surface of the image sensor and blocks light beams incident on the image sensor to generate shadows on the detection target surface, wherein the luminance distribution estimation method comprises: a luminance estimation step, executed by a computer, of assuming that there are a large number of discrete virtual light sources in the sky, and estimating the luminance of each of the virtual light sources that contributes to the pixel values ​​of the sensor elements based on pixel values ​​observed by at least some of the plurality of sensor elements and incidence validity that indicates whether light beams from each of the virtual light sources are incident on the sensor elements.

7. A luminance distribution estimation program for estimating the luminance distribution of the sky using a luminance distribution estimation system comprising: an image sensor having a plurality of sensor elements arranged side by side in a plane; and a shielding object with known object surface reflectance that is placed above a detection target surface of the image sensor and blocks light beams incident on the image sensor to generate shadows on the detection target surface, the luminance distribution estimation program causing a computer to execute the following luminance estimation instructions: assuming that there are a large number of discrete virtual light sources in the sky, and estimating the luminance of each of the virtual light sources that contributes to the pixel values ​​of the sensor elements based on pixel values ​​observed by at least some of the plurality of sensor elements and incidence effectiveness that indicates whether light beams from each of the virtual light sources are incident on the sensor elements.

Images