Cloud height measurement system and cloud height measurement method

Abstract

To enable accurate measurement of cloud height for clouds with a few patterns, even using a stereo camera.SOLUTION: A cloud height measurement system includes: a stereo camera that acquires a parallax image according to the parallax with a predetermined length by imaging the same cloud by a plurality of imaging units arranged to be apart from each other by the predetermined length; a cloud base information acquisition device that acquires cloud base information about the cloud base of the cloud; and a computer that calculates the cloud height being the altitude of the cloud base of the cloud. The computer includes: a first altitude calculation unit that specifies feature extraction information including information on feature portions being contour portions of the cloud and intermediate regions surrounded by the feature portions from the parallax image acquired by the stereo camera and calculates a first altitude for the feature portions; a second altitude calculation unit that calculates a second altitude for the intermediate region surrounded by the feature portions on the basis of the cloud base information acquired by the cloud base information acquisition device and the feature extraction information; and an altitude interpolation unit that interpolates the cloud height on the basis of the first altitude and the second altitude.SELECTED DRAWING: Figure 5

Description

[Technical field]

[0001] The present invention relates to a cloud height measurement system and a cloud height measurement method, and is suitable for application to a cloud height measurement system relating to a technique for measuring the cloud height using a stereo camera when measuring the altitude of the clouds from the ground, for example. [Background technology]

[0002] Weather information is generally obtained from image data captured by meteorological satellites and weather forecasts based on general meteorology, and is disclosed not only to domestic users but also to the entire world. Some weather information is obtained using measurement methods specific to each type of weather information, and the measurement costs can be high. This weather information is widely used by businesses and individuals alike, and its applications are diverse. Generally, the weather information disclosed is for a wide area, covering an area of ​​several square kilometers, and is therefore effective for predicting weather phenomena on a regional basis.

[0003] On the other hand, there are many cases where local weather information at a specific location is required. For example, in the case of aircraft traffic management, pinpointing information on clouds and air currents on the aircraft's path allows for safe and efficient operation. In particular, to ensure the safety of aircraft during takeoff and landing, it is extremely important to observe weather information above airports and around runways. Airports are generally equipped with a wide variety of observation devices, and observers also constantly conduct visual observations. One of these observation items is the height from the ground to the cloud base (hereinafter referred to as "cloud height"), which plays an important role in determining whether or not an aircraft can take off or land. Currently, laser ranging devices such as ceilometers are widely used as a standard means of measuring cloud height.

[0004] A ceilometer is an optical device that uses a laser. Specifically, a ceilometer emits a powerful laser beam, detects the optical signal scattered and reflected at, for example, the cloud base, and can measure the cloud height to an accuracy of several meters from the arrival time and signal strength. However, since a ceilometer can only measure a narrow range directly above, it is not suitable for measuring the entire sky, including directly above and around airports and runways. Although it is possible to expand the measurement range by installing multiple such ceilometers, this is not practical because ceilometers are generally expensive.

[0005] Therefore, a technology is required to measure the cloud height information of the whole sky at any point with high accuracy. Patent Document 1 discloses a technology to measure the cloud height, the cloud movement speed, and the wind speed by directing a stereo camera consisting of two wide-angle cameras toward the zenith. In the technology described in Patent Document 1, the stereo camera is configured by two cameras whose installation positions have known altitude, longitude, and latitude, and the azimuth angle and zenith angle of the subject are obtained from multiple images taken by each camera at the same time. Furthermore, in the technology described in Patent Document 1, the altitude of the subject is assumed, the longitude and latitude are calculated based on the assumed altitude, and the process of assuming a new altitude is repeated until the values ​​match for each camera or the difference between them falls within a preset tolerance, thereby measuring the cloud height. [Prior art documents] [Patent documents]

[0006] [Patent Document 1] JP 2019-60754 A Summary of the Invention [Problem to be solved by the invention]

[0007] The technology described in Patent Document 1 requires two images in which at least a part of a cloud is commonly captured, and Patent Document 1 cites the problem that the outline of a cloud is generally vague, and it is difficult to identify a specific part of a cloud that spreads over an entire surface. However, in the technology described in Patent Document 1, when photographing clouds using a stereo camera, if it is not easy to identify a part of the cloud that is commonly captured in two images, for example, when the cloud has few patterns, there are areas in which the stereo camera cannot detect parallax, and it is difficult to measure the cloud height of a cloud with few patterns using the stereo camera alone.

[0008] The present invention has been made in consideration of the above points, and aims to propose a cloud height measurement system and a cloud height measurement method that can accurately measure the cloud height for clouds with little pattern even when using a stereo camera. [Means for solving the problem]

[0009] In order to solve such problems, the present invention includes a stereo camera that captures images of the same cloud using a plurality of imaging units arranged at a predetermined distance apart to capture a parallax image corresponding to the predetermined length of parallax, a cloud base information acquisition device (e.g., an infrared camera) that is a means for acquiring cloud base information related to the cloud base of the cloud, and a computer that calculates a cloud height that is the altitude of the cloud base of the cloud. The computer is equipped with a first altitude calculation unit that identifies feature extraction information including information on a characteristic part that is a contour part of the cloud and information on an intermediate area surrounded by the characteristic part from the parallax image acquired by the stereo camera and calculates a first altitude for the characteristic part, a second altitude calculation unit that calculates a second altitude for the intermediate area based on the cloud base information and the feature extraction information acquired by the cloud base information acquisition device, and an altitude interpolation unit that interpolates the cloud height based on the first altitude corresponding to the characteristic part and the second altitude corresponding to the intermediate area.

[0010] The present invention also includes a photographed image acquisition step of acquiring a plurality of photographed images of the same cloud by a stereo camera having a plurality of imaging units arranged at a predetermined distance apart, a parallax image acquisition step of acquiring a parallax image according to the predetermined length from the plurality of photographed images, a cloud base information acquisition step of acquiring cloud base information relating to the cloud base of the cloud by a cloud base information acquisition device, and a cloud height calculation step of calculating a cloud height which is the height of the cloud base of the cloud by a computer, the cloud height calculation step being a characteristic step in which the computer recognizes a characteristic portion which is a contour portion of the cloud from the parallax images acquired by the stereo camera. The method includes a part recognition step, an intermediate area recognition step in which the computer recognizes an intermediate area surrounded by the characteristic part, a first altitude calculation step in which the computer identifies feature extraction information which is information on the characteristic part and the intermediate area and calculates a first altitude for the characteristic part, a second altitude calculation step in which the computer calculates a second altitude for the intermediate area based on the cloud base information and the feature extraction information acquired by the cloud base information acquisition device, and an altitude interpolation step in which the computer interpolates the cloud height based on the first altitude and the second altitude. Effect of the Invention

[0011] According to the present invention, the cloud height can be measured with high accuracy even when a stereo camera is used, even for clouds with little pattern. [Brief description of the drawings]

[0012] [Figure 1] FIG. 1 is a diagram showing the principle of measuring cloud altitude using a stereo camera. [Diagram 2] FIG. 1 is a diagram showing an example of an arrangement in which a stereo camera is fixed to the ground. [Figure 3A] FIG. 13 is a diagram showing an example of an arrangement in which a stereo camera is fixed to a tripod. [Figure 3B] FIG. 13 is a diagram showing an example of an arrangement in which a stereo camera is fixed to a tripod. [Figure 4A] FIG. 1 is a diagram showing an example of a layout pattern of stereo cameras for measuring cloud heights over the entire sky. [Figure 4B] FIG. 1 is a diagram showing an example of a layout pattern of stereo cameras for measuring cloud heights over the entire sky. [Figure 4C] FIG. 1 is a diagram showing an example of a layout pattern of stereo cameras for measuring cloud heights over the entire sky. [Figure 4D] FIG. 1 is a diagram showing an example of a layout pattern of stereo cameras for measuring cloud heights over the entire sky. [Diagram 5] FIG. 1 is a diagram illustrating an example of the configuration of a cloud height measurement system. [Figure 6] 1 is a diagram showing an example of the arrangement of sensors provided in a cloud height measurement system according to a first embodiment; [Figure 7] 5 is a flowchart illustrating an example of a process of a first altitude calculation unit in the first embodiment. [Figure 8] 5 is a flowchart illustrating an example of processing by a second altitude calculation unit in the first embodiment. [Figure 9A] 1 is a graph showing an example of temperature distribution at different altitudes in the sky. [Figure 9B] 1 is a graph showing an example of temperature distribution at different altitudes in the sky. [Figure 9C] 1 is a graph showing an example of temperature distribution at different altitudes in the sky. [Figure 10A] FIG. 1 is a diagram showing an example of a cloud temperature image taken by an infrared camera. [Figure 10B] FIG. 1 is a diagram showing an example of a cloud altitude image taken by an infrared camera. [Figure 11] 5 is a flowchart illustrating an example of processing by an advanced interpolation unit in the first embodiment. [Figure 12] 5 is a flowchart illustrating an example of processing by an advanced interpolation unit in the first embodiment. [Figure 13] FIG. 1 is a diagram showing a case where the cloud is a single layer. [Figure 14] FIG. 13 is a diagram showing a case where clouds have multiple layers. [Figure 15] This is a diagram showing a case where blue sky is present above multiple clouds. [Figure 16]FIG. 4 is a diagram showing an example of a procedure of altitude interpolation processing by an altitude interpolation unit according to the first embodiment. [Figure 17] 11 is a flowchart showing an example of a procedure for cloud height measurement processing. [Figure 18] 5 is a flowchart showing an example of a procedure for cloud height measurement processing in the first embodiment. [Figure 19] FIG. 11 is a diagram illustrating an example of a sensor arrangement in the second embodiment. [Figure 20] 10 is a flowchart illustrating an example of processing by an advanced interpolation unit in the second embodiment. [Figure 21] FIG. 11 is a diagram showing an example of a procedure of altitude interpolation processing by an altitude interpolation unit in the second embodiment. [Figure 22] FIG. 13 is a diagram illustrating an example of the arrangement of sensors for measuring cloud height in the third embodiment. [Figure 23] 13 is a flowchart illustrating an example of processing by an advanced interpolation unit in the third embodiment. [Figure 24] 13 is a flowchart showing an example of a procedure for cloud height measurement processing in the third embodiment. [Figure 25A] FIG. 13 is a diagram illustrating an example of a sensor arrangement in the fourth embodiment. [Figure 25B] FIG. 13 is a diagram illustrating an example of a sensor arrangement in the fourth embodiment. [Figure 26] FIG. 13 is a diagram illustrating an example of the configuration of a cloud height measurement system according to a fourth embodiment. [Figure 27A] 13 is a flowchart illustrating an example of processing by a second altitude calculation unit in the fourth embodiment. [Figure 27B] 13 is a flowchart illustrating an example of processing by a second altitude calculation unit in the fourth embodiment. [Figure 28] 13 is a flowchart illustrating an example of processing by an advanced interpolation unit in the fourth embodiment. [Figure 29A] FIG. 13 is a diagram showing an example of altitude interpolation processing for an invalid area when multiple layers of clouds are present. [Figure 29B]FIG. 13 is a diagram showing an example of altitude interpolation processing for an invalid area when multiple layers of clouds are present. [Figure 30A] 13 is a flowchart showing an example of a procedure for cloud height measurement processing in the fourth embodiment. [Figure 30B] 13 is a flowchart showing an example of a procedure for cloud height measurement processing in the fourth embodiment. [Diagram 31] FIG. 13 is a diagram illustrating an example of a sensor arrangement in the fifth embodiment. [Diagram 32] 13 is a flowchart showing an example of a procedure for cloud height measurement processing in the fifth embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0013] Hereinafter, an embodiment of the present invention will be described in detail with reference to the drawings.

[0014] (1) First embodiment Figure 1 shows the principle of measuring cloud altitude using a stereo camera 3. The stereo camera 3 captures images of the same cloud C using multiple cameras (imaging units) arranged at a distance of a baseline length B (a predetermined length) to obtain a parallax image according to the parallax of the baseline length B.

[0015] The distance Z from the stereo camera 3 to the cloud base is calculated from the relationship between parallax and distance using the following formula (1). Z=B / tanθ=Bf / d=BF / pD (1)

[0016] where B is the baseline length between the multiple cameras (hereinafter also referred to as "imaging units") 3a and 3b included in the stereo camera 3, f is the focal length, θ is the angle, d is the parallax, p is the pixel pitch, and D is the parallax per pixel. Note that the baseline length B between the cameras 3a and 3b is preferably about 10 m to achieve the accuracy required for clouds in a scheduled aviation weather report (METAR).

[0017] 2 shows an example of the arrangement of the stereo camera 3 fixed on the ground. In this embodiment, cameras using a wide-angle lens (hereinafter also referred to as "wide-angle camera") may be used as the cameras 3a and 3b of the stereo camera 3, and cameras using a fisheye lens (hereinafter also referred to as "fisheye camera") may be used. In the former case, an area that fits within the viewing angle of the wide-angle lens is captured with one shutter, and the entire sky can be captured by, for example, releasing the shutter multiple times. In the latter case, the entire sky that fits within the viewing angle of the fisheye lens can be captured with one shutter.

[0018] 3A and 3B each show an example of the arrangement of the stereo camera 3 fixed to a tripod 5. The cameras 3a and 3b of the stereo camera 3 are attached near both ends of a rail 7 provided on the upper part of the tripod 5, and the elevation angles of the cameras 3a and 3b are adjustable. The orientations of the cameras 3a and 3b are adjusted so that they have the same elevation angle.

[0019] Figure 3A shows a case where the elevation angle of the cameras 3a and 3b is lowered to capture images of clouds closer to the horizon. On the other hand, Figure 3B shows a case where the elevation angle is higher to capture images of clouds directly above. Note that in Figure 3A, the distance from the cameras 3a and 3b to the clouds is longer, so the elevation angle must be more accurate.

[0020] 4A to 4D show examples of arrangement patterns of stereo cameras 3 for measuring cloud heights over the entire sky. Fig. 4A shows a stereo camera 3 using two cameras 3a and 3b equipped with fisheye lenses, which can observe the entire sky with a single shutter.

[0021] FIG. 4B illustrates, at its upper side, a stereo camera 3 in which four cameras 3a, 3b, 3c, and 3d equipped with fisheye lenses are arranged at each end of a cross, and, at its lower side, a stereo camera 3 in which three cameras 3a, 3b, and 3c equipped with fisheye lenses are arranged at each end of an L shape.

[0022] 4C illustrates a stereo camera 3 using cameras 3a, 3b, 3c, 3d, 3e, and 3f equipped with wide-angle lenses with a viewing angle of 120°. The stereo camera 3 can cover the entire sky with three sets of six cameras 3a, 3b, 3c, 3d, 3e, and 3f.

[0023] 4D illustrates a stereo camera 3 using cameras 3a to 3t equipped with a wide-angle lens with a field angle of 60°. Ten sets of two cameras, totaling 20 cameras 3a to 3t, can cover the entire sky.

[0024] Here, an overview of the cloud height measurement system according to this embodiment will be described. FIG. 5 is a block diagram showing an example of the configuration and functions of the cloud height measurement system according to this embodiment. The cloud height measurement system according to this embodiment includes any one of the stereo cameras 3 described above, a cloud base information acquisition device 2 that acquires cloud base information related to the cloud base, and a computer 1 that calculates the altitude of the cloud base (hereinafter also referred to as "cloud height"). The cloud base information acquisition device in this embodiment is, for example, an infrared camera. The computer 1 includes a first altitude calculation unit 11 that specifies feature extraction information 43 including information on a characteristic part, which is a contour part of a cloud, and an intermediate area surrounded by the characteristic part from a parallax image 41 acquired by the stereo camera, and calculates a first altitude 45 for the characteristic part, a second altitude calculation unit 13 that calculates a second altitude 46 for the intermediate area surrounded by the characteristic part based on the cloud base information 42 and the feature extraction information 43 acquired by the cloud base information acquisition device 2, and an altitude interpolation unit 14 that interpolates the cloud height 47 of the cloud based on the first altitude and the second altitude. Details of the functions of the computer will be described later. In this embodiment, the stereo camera 3 and the infrared camera (and the ceilometer in other embodiments described later) are also collectively referred to as "sensors."

[0025] The computer 1 of this cloud height measurement system includes a first altitude calculation unit 11, a second altitude calculation unit 13, and an altitude interpolation unit 14 that executes altitude interpolation processing.

[0026] The first altitude calculation unit 11 is an example of a first altitude calculation unit, which receives a parallax image 41 acquired by the stereo camera 3, identifies feature extraction information 43 from this parallax image, which includes information on a characteristic part, for example, the outline of a cloud, and an intermediate area surrounded by the characteristic part, and calculates an altitude (first altitude) for the characteristic part, for example, with high accuracy.

[0027] The second altitude calculation unit 13 is an example of a second altitude calculation unit, and identifies cloud layers and determines the area of ​​the cloud base for each cloud layer based on cloud base information 42 and feature extraction information 43 acquired by an infrared camera 9 as an example of a cloud base information acquisition device 2. In addition, for an intermediate area of ​​the cloud surrounded by the feature portion, the second altitude calculation unit 13 calculates an altitude (second altitude) with low accuracy, for example.

[0028] The altitude interpolation unit 14 interpolates the altitude of the cloud base by altitude interpolation processing based on the first altitude corresponding to the characteristic portion and the second altitude corresponding to the intermediate region, and calculates the accurate altitude (second altitude) of the cloud base. That is, the altitude interpolation unit 14 interpolates the altitude (second altitude) of the missing part (hereinafter also referred to as "altitude data missing part") of the stereo camera altitude map based on the parallax image acquired by the stereo camera 3, i.e., the invalid area corresponding to the intermediate region, by the altitude interpolation processing, creates an interpolated altitude map, and outputs the cloud height.

[0029] Fig. 6 shows an example of the arrangement of sensors provided in the cloud height measurement system according to this embodiment. The cloud height measurement system according to the first embodiment includes a computer (not shown), a stereo camera 3 having two cameras 3a and 3b, and one infrared camera 9. The arrangement pattern of the stereo camera 3 is not limited to that shown in Fig. 4A, and may be any of the arrangement patterns shown in Figs. 4B to 4D.

[0030] The infrared camera 9 is disposed, for example, on a baseline length B between the cameras 3a and 3b of the stereo camera 3. Imaging methods using the infrared camera 9 include (a) a method in which one infrared camera 9 is mounted on a PTZ head that can adjust the pan angle and tilt angle, and the pan angle and tilt angle are changed at regular intervals to capture multiple infrared images to capture an image of the entire sky, and (b) a method in which an infrared image of the entire sky is captured using an infrared camera equipped with a fisheye lens.

[0031] In addition, in the imaging method (a), instead of using a PTZ head, multiple infrared cameras 9 may be fixed at an angle to capture the entire sky as shown in Fig. 4C or 4D. It is desirable to install the infrared camera 9 near the main camera (e.g., camera 3a) of the stereo camera 3 in order to align it with the image by the stereo camera 3. In Fig. 6, the infrared camera 9 is placed on an extension line of the base line length B connecting the two cameras 3a and 3b of the stereo camera 3, but it does not have to be on the same straight line as the base line length B.

[0032] 7 is a flow chart showing an example of the processing of the first altitude calculation unit 11 among the processing of the computer 1 provided in the cloud height measurement system. The first altitude calculation unit 11 executes a feature part recognition step of recognizing a feature part, which is a cloud outline part, from the parallax image 41 of the stereo camera 3, and then executes an intermediate area recognition step of recognizing an intermediate area surrounded by the feature part, and then specifies feature extraction information 43 including information on the feature part and the intermediate area (step S41), and calculates a first altitude for the feature part (step S42).

[0033] FIG. 8 is a flow chart showing an example of the processing of the second altitude calculation unit 13 among the processing of the calculator 1 included in the cloud height measurement system. The second altitude calculation unit 13 creates a temperature map based on an infrared image acquired by an infrared camera (step S51), identifies cloud layers based on the temperature map (step S52), and specifies the area of ​​the cloud base for each cloud layer (step S53). Steps S51 to S53 are an example of the processing of the cloud base processing unit 12 included in the second altitude calculation unit 13. In addition, the second altitude calculation unit 13 converts the temperature acquired by the infrared camera into altitude using a method described later (step S54), and calculates a second altitude for the intermediate area based on the feature extraction information 43 specified by the first altitude calculation unit (step S55).

[0034] 9A to 9C are examples of temperature distributions at different altitudes in the sky. First, a method for measuring the surface temperature of a cloud using an infrared camera 9 will be described. The infrared camera 9 is, for example, an infrared thermography camera. Clouds are formed when air containing water vapor near the ground rises, the water vapor gradually cools, and the temperature reaches the dew point. In other words, when the surface temperature of the bottom of a cloud is measured from the ground, the altitude at which that temperature reaches the dew point coincides with the cloud height.

[0035] Therefore, if temperature data for each altitude in the sky is obtained and the relationship between temperature and altitude is graphed, the temperature measured by the infrared camera 9 can be converted to altitude. The above-mentioned infrared thermography camera is a device that detects infrared radiation energy emitted from an object, and converts it into a temperature distribution and displays it by utilizing the relationship that the amount of radiation is proportional to the temperature of the object.

[0036] In order to convert the detected infrared radiation energy into temperature, it is necessary to set the reflectance, reflected temperature, distance to the object, atmospheric temperature, and relative humidity of the object in advance as object parameters. By converting the temperature distribution of the detected clouds into altitude using the temperature data for each altitude in the sky, the altitude distribution of the clouds can be obtained.

[0037] Temperature data for each altitude in the sky can be obtained, for example, from the Mesoscale Model for Mesoscale Weather Forecasting (MSM), which is periodically published by the Japan Meteorological Agency. In the MSM, numerical analysis is performed using the four-dimensional variation method to forecast altitudes and atmospheric temperatures for each pressure surface on a mesh with a grid spacing of 5 km square.

[0038] 9A to 9C are graphs showing an example of temperature distribution at each altitude in the sky. FIGS. 9A and 9B are examples of the relationship between temperature and altitude in the sky above Kokubunji City, Tokyo, obtained from the MSM of numerical weather forecast data created by the Japan Meteorological Agency. In the case of the graph shown in FIG. 9A, the temperature (air temperature) and altitude correspond uniquely, so the cloud temperature measured by the infrared camera 9 can be converted to altitude. On the other hand, when a Z-shape is drawn as in the graph shown in FIG. 9B, the temperature and altitude do not correspond uniquely. This phenomenon occurs, for example, when an inversion layer exists.

[0039] In this embodiment, the infrared camera 9 is used to interpolate a portion where the altitude of the clouds is difficult to obtain in the parallax image obtained by the stereo camera 3 (hereinafter, also referred to as an "altitude data missing portion"). Therefore, it is sufficient to know the relative altitude with respect to the temperature, and the exact altitude is not required. Therefore, in this embodiment, as shown in the graph showing the general altitude vs. temperature characteristic shown in FIG. 9C, a specific relationship may be used, for example, in which the temperature decreases by 2°C for every certain distance (for example, 1000 ft) above the ground. This makes it possible to convert the cloud temperature measured by the infrared camera 9 into a relative altitude even when the temperature and altitude do not correspond uniquely as in FIG. 9B.

[0040] FIG. 10A and FIG. 10B show examples of an infrared temperature image and an altitude image of a cloud taken by an infrared camera 9, respectively. The infrared temperature image shown in FIG. 10A captures the temperature distribution of the cloud. Also shown is a line profile, which is the temperature distribution along a line L shown by a dashed line on the infrared temperature image of FIG. 10A. According to the line profile on line L, there is a part where the temperature is flat in the high temperature area, and it can be determined that this part is the cloud base. On the other hand, the altitude image shown in FIG. 10B shows data converted from temperature to altitude by a predetermined calculation for converting from temperature to altitude based on a specific relationship as shown in FIG. 9C. The axis on the right side of FIG. 10B shows the altitude converted from temperature.

[0041] FIG. 11 is a flow chart showing an example of the process of the altitude interpolation unit 14 in this embodiment. The altitude interpolation unit 14 receives a first altitude 45 and a second altitude 46, and outputs a cloud height 47. The first altitude includes feature extraction information 43 calculated by the first altitude calculation unit 11, and the second altitude includes the result of the cloud layer identification in the second altitude calculation unit 13. The altitude interpolation unit 14 uses the result of the cloud layer identification in the second altitude calculation unit 13 to associate the characteristic part of the cloud and the intermediate region with each other, and judges whether there is an area where the characteristic part and the intermediate region belong to the same cloud layer (step S61). That is, for each pixel of the stereo camera altitude map 30, it judges whether there is an area where the parallax cannot be detected, that is, an invalid area 32. This is to judge whether the above-mentioned altitude data missing part occurs in the stereo camera altitude map 30, since parallax cannot be detected in a patternless part of the cloud.

[0042] If the determination result indicates that the intermediate region belongs to the same cloud layer as the characteristic portion, the second altitude of the intermediate region is interpolated based on the first altitude of the characteristic portion that belongs to the same cloud layer as the intermediate region (step S62). By performing the above process, even if the characteristic portion belongs to multiple cloud layers, the altitude of the intermediate region can be accurately interpolated by interpolating the altitude of the intermediate region based on the altitude of the characteristic portion of the region that belongs to the same cloud layer as the intermediate region.

[0043] On the other hand, the altitude interpolation unit determines whether there is an area where the feature part and the intermediate area belong to the same cloud layer, and if the intermediate area does not belong to any cloud layer, it determines that the intermediate area is an area of ​​blue sky without clouds, and does not interpolate the altitude of the intermediate area (step S63).

[0044] FIG. 12 is a flowchart showing an example of a process for determining whether or not to perform altitude interpolation processing in the altitude interpolation unit 14 according to this embodiment. The first altitude input to the altitude interpolation unit 14 includes information on the area in which parallax cannot be detected in the parallax image 41 as feature extraction information. When the stereo camera altitude map 30 is generated from the stereo camera captured image, the altitude interpolation unit 14 determines whether or not the area in which parallax cannot be detected in the parallax image 41 exists at a predetermined rate (for example, 10%) or more (step S64). When the area in which parallax cannot be detected in the parallax image 41 exists at a predetermined rate (for example, 10%) or more, it is determined that there is an invalid area 32, and the altitude (second altitude) of the middle area of ​​the cloud corresponding to the area in which parallax cannot be detected is interpolated (step S65). On the other hand, when the area in which parallax cannot be detected in the parallax image 41 is less than a predetermined rate (for example, 10%), it may be determined that there is no invalid area 32, and the altitude (second altitude) of the middle area of ​​the cloud corresponding to the area in which parallax cannot be detected is not interpolated (step S66).

[0045] 13 to 15 show an example of a method for determining cloud height by combining the stereo camera 3 and the infrared camera 9. FIG. 13 shows a case where the clouds are a single layer, FIG. 14 shows a case where the clouds are multiple layers, and FIG. 15 shows a case where the clouds are multiple layers and blue sky exists above the clouds. In the illustrated example, the thick line shows an example of the altitude measured using the stereo camera 3, and the dashed line shows an example of the altitude measured using the infrared camera 9. In this embodiment, a region where parallax can be detected in a parallax image in which parallax is reflected based on a plurality of images captured by the cameras 3a and 3b of the stereo camera 3, that is, a region where a pattern can be identified, is referred to as a "valid region 31", and a region where parallax cannot be detected in the parallax image, that is, a region where a pattern cannot be identified, is referred to as an "invalid region 32".

[0046] Here, while it is possible to grasp the cloud altitude (first altitude) 31a from the parallax image for the valid region 31, it is difficult to grasp the cloud altitude 32a from the parallax image for the invalid region 32. Therefore, in this embodiment, as an example, an altitude interpolation process to be described later is performed for the invalid region.

[0047] The cloud contours are areas where patterns can be identified, and correspond to the valid areas described above. The areas between the cloud contours, i.e., the intermediate areas surrounded by the cloud contours, are areas where parallax cannot be detected and patterns cannot be identified, and correspond to the invalid areas described above. In this embodiment, the cloud contours, which are areas where parallax can be detected, will be described as an example of a characteristic part of a cloud.

[0048] In this embodiment, in the altitude interpolation process, the cloud layer to which the cloud characteristic portion and the intermediate region belong is determined based on an infrared camera temperature map as an example of a temperature map, and if the cloud characteristic portion and the intermediate region belong to the same cloud layer, the altitude 32a (second altitude) of the intermediate region is interpolated based on the altitude 31a (first altitude) of the cloud characteristic portion shown in Fig. 13. That is, in this embodiment, the altitude of a patternless region (invalid region) surrounded by a patterned region (valid region) in the same cloud layer can be mechanically interpolated by a method such as interpolation.

[0049] When there are multiple layers of clouds as shown in Figure 14, care must be taken with the altitude interpolation process for the parts of each cloud layer that are not surrounded by valid areas (corresponding to the "invalid area A" and "invalid area B" in the figure), i.e., the invalid area clouds. This is because if the area indicated by the arrow (corresponding to the "invalid area B" in the figure) is mechanically interpolated, it will be interpolated to connect the first layer set C1 and the second layer cloud C2, resulting in a misidentification of the cloud layer.

[0050] To prevent such misrecognition, a temperature map, which is a temperature distribution based on an infrared image acquired by infrared camera 9, is referred to to perform cloud layer separation to determine which cloud layer a patternless area (invalid area) in each of multiple cloud layers at different altitudes belongs to (whether it is "invalid area A" or "invalid area B" in the figure), and then the altitude of the invalid area is interpolated to correctly interpolate the altitude of the cloud layer.

[0051] In this embodiment, the cloud base processing unit 12 included in the second altitude calculation unit 13 determines the cloud layer to which the characteristic parts of the cloud and the intermediate region belong based on an infrared camera temperature map as an example of a temperature map, and if the intermediate region does not belong to any cloud layer, the altitude interpolation unit 14 determines that the intermediate region is a blue sky area between multiple clouds and does not interpolate the altitude of the intermediate region.

[0052] For example, the blue sky area BS shown in Fig. 15 is recognized as having no pattern by the stereo camera 3, so that the parallax cannot be detected and it is regarded as an invalid area (corresponding to the "invalid area C" in the figure). Furthermore, according to the infrared camera 9, this blue sky area BS is below the lower limit of temperature, making it difficult to distinguish between the blue sky area BS and upper clouds higher than the second layer clouds C2. For such blue sky area BS, advanced interpolation processing can be used to combine the color of the visible light image captured by the stereo camera 3 and the infrared camera temperature map acquired by the infrared camera 9 to determine whether or not it is a blue sky area BS (hereinafter also referred to as "blue sky determination").

[0053] 16 is a diagram showing an example of the procedure of altitude interpolation processing by the altitude interpolation unit 14. Inputs to the calculator 1 are a parallax image 41 created from two captured images 4a, 4b acquired by the cameras 3a, 3b of the stereo camera 3, and an infrared camera temperature map 9a acquired by the infrared camera 9.

[0054] The first altitude calculation unit 11 calculates the altitude (first altitude) from the parallax image 41 shown in the upper left of Fig. 16, and generates the stereo camera altitude map 30 shown in the upper center of Fig. 16. This stereo camera altitude map 30 includes a valid area 31 that is a cloud outline portion having a pattern as an example of a characteristic portion of a cloud, and an invalid area 32 that is an intermediate area surrounded by the valid area 31 and has no pattern. As described above, the invalid area 32 is an area where parallax cannot be detected.

[0055] For each pixel in the stereo camera altitude map 30, a determination is made as to whether or not there is an area where parallax cannot be detected, i.e., an invalid area 32. This is because parallax cannot be detected in patternless parts of the clouds, and therefore, it is necessary to determine whether or not there is a missing part of altitude data in the stereo camera altitude map 30 as described above.

[0056] In addition, the second altitude calculation unit 13 generates an infrared camera altitude map 40 shown in the lower center of Figure 16 from an infrared camera temperature map 9a shown in the lower left of Figure 16, for example, taking into account general altitude vs. temperature characteristics (see Figure 9C).

[0057] Therefore, the altitude interpolation unit 14 refers to the infrared camera altitude map 40 obtained by calculating the altitude from the infrared camera temperature map 9a, and outputs the altitude map 50 generated by interpolating the altitude for the invalid area 32 of the stereo camera altitude map 30.

[0058] In this embodiment, for patternless clouds, the valid area 31 is small, so that almost the entire area of ​​the stereo camera altitude map 30 may become the invalid area 32. In such a case, the altitude interpolation unit 14 may use the infrared camera altitude map 40 to interpolate the invalid area 32 of the stereo camera altitude map 30, and adopt the altitude obtained based on the infrared camera altitude map 40 as the altitude of the invalid area 32.

[0059] 17 is a flowchart showing an example of the procedure of the cloud height measurement process in this embodiment. First, an overview of the cloud height measurement method of the cloud height measurement system according to this embodiment will be described. The cloud height measurement method includes a photographed image acquisition step of acquiring a plurality of photographed images of the same cloud by a stereo camera 3 having a plurality of cameras arranged at a predetermined distance apart, a parallax image acquisition step (step S1) of acquiring a parallax image according to a predetermined length from the plurality of photographed images, a cloud base information acquisition step (step S2) of acquiring cloud base information related to the cloud base by an infrared camera 9 as an example of a cloud base information acquisition device 2, and a cloud height calculation step (step S3) of calculating the cloud height, which is the altitude of the cloud base, by a computer 1. Among these, the cloud height calculation step includes a feature part recognition step in which the first altitude calculation unit 11 of the computer 1 recognizes a feature part, which is a contour part of a cloud, from a parallax image acquired by the stereo camera 3; an intermediate area recognition step in which the first altitude calculation unit 11 of the computer 1 recognizes an intermediate area surrounded by the feature part; a first altitude calculation step (step S4) in which the first altitude calculation unit 11 of the computer 1 identifies feature extraction information 43 including information on the feature part and the intermediate area, and calculates an altitude (first altitude) for the feature part; a second altitude calculation step (step S5) in which the second altitude calculation unit 13 of the computer 1 identifies a cloud layer based on the cloud base information and feature extraction information 43 acquired by the infrared camera 9, and calculates an altitude (second altitude) for the intermediate area surrounded by the feature part; and an altitude interpolation step (step S6) in which the altitude interpolation unit 14 of the computer 1 interpolates the altitude of the intermediate area based on the first altitude corresponding to the feature part and the second altitude corresponding to the cloud base of the cloud. To explain more specifically, it is as follows.

[0060] In step S1, the stereo camera 3 first obtains the captured images 4a and 4b captured by the cameras 3a and 3b of the stereo camera 3, detects parallax by comparing the captured images 4a and 4b, and creates a parallax image 41.

[0061] In step S2, the infrared camera 9, which is an example of the cloud base information acquisition device 2, creates an infrared camera temperature map 9a from the captured infrared image, and acquires cloud base information 42 relating to the cloud base.

[0062] In step S3, the calculator 1 receives the parallax image 41 and the cloud base information 42 as input, calculates a first altitude, calculates a second altitude, and interpolates the altitude, and outputs the cloud height 47.

[0063] In step S4, the first altitude calculation unit 11 creates a stereo camera altitude map 30 capable of estimating the altitude of the cloud layer based on the parallax image 41, identifies feature extraction information 43 from the parallax image, which includes information on characteristic parts, which are the contours of the cloud, and intermediate areas surrounded by the characteristic parts, and calculates a first altitude 45 for the characteristic parts.

[0064] In step S5, the second altitude calculation unit 13 performs cloud layer separation to identify the cloud layers of each cloud from the infrared camera temperature map 9a obtained from the infrared image captured by the infrared camera 9, for example, based on the temperature distribution shown in Fig. 10A, and identifies, for each cloud layer, for example, the region where the temperature is maximum in the cloud layer as the cloud base. Also, from the infrared camera temperature map 9a obtained from the infrared image captured by the infrared camera 9, for example, based on the general altitude vs. temperature characteristics shown in Fig. 9C, the second altitude 46 is calculated for the intermediate region based on the infrared camera altitude map 40 and the feature extraction information 43.

[0065] In step S6, the altitude interpolation unit 14 determines whether or not there is an area where parallax cannot be detected based on the feature extraction information 43. If there is an area where parallax cannot be detected, the altitude interpolation unit 14 refers to the result of the cloud layer separation described above, and uses the altitude corresponding to the cloud layer of the cloud to which the invalid area 32 belongs to, to interpolate the altitude of the intermediate area, thereby generating an altitude map 50. If the invalid area 32 does not belong to any cloud layer, the invalid area is determined to be an empty area, and no interpolation is performed. On the other hand, if there is no area where parallax cannot be detected, the altitude interpolation unit 14 generates the altitude map 50 without performing altitude interpolation processing. The cloud height 47 is calculated based on the altitude map 50, and output.

[0066] FIG. 18 is a flow chart showing an example of the procedure of the cloud height measurement process in this embodiment. The sensors used in this cloud height measurement process are two types: a stereo camera 3 and an infrared camera 9. This cloud height measurement process corresponds to a cloud height measurement method of the cloud height measurement system according to this embodiment. A parallax image 41 is acquired from the stereo camera 3 (step S1), and the first altitude calculation unit 11 creates a stereo camera altitude map 30 from the parallax image 41 and calculates a first altitude 45 (step S4). Also, an infrared camera temperature map 9a is acquired from the infrared camera 9 as cloud base information 42 (step S29), and the cloud base processing unit 12 included in the second altitude calculation unit 13 identifies the cloud layer by cloud layer separation (step S52), and the second altitude calculation unit 13 creates an infrared camera altitude map 40 from the infrared camera temperature map 9a and calculates a second altitude 46 (step S5). In step S64, the presence or absence of an invalid area 32 is determined based on the stereo camera altitude map 30, and if an invalid area is present, the altitude interpolation unit 14 interpolates the altitude of the intermediate area based on the first altitude 45 (step S6). If there is no invalid area, no altitude interpolation is performed. In addition, the altitude interpolation unit 14 calculates and outputs the cloud height 47 from the altitude map 50 after the altitude interpolation process has been applied (step S7).

[0067] In addition, the computer 1 of the cloud height measurement system according to this embodiment combines information about clouds such as cloud height, cloud layer, information about visibility observed by other sensors not shown (e.g., the maximum distance visible to the naked eye), and observation information such as the current weather, to generate a METAR statement, which is a specified format for METAR.

[0068] As described above, the cloud height measurement system according to the present embodiment includes a stereo camera 3 that captures images of the same cloud using a plurality of imaging units arranged at a predetermined distance from each other to obtain a parallax image 41 corresponding to the predetermined length of parallax, a cloud base information acquisition device 2 that acquires cloud base information 42 related to the cloud base, and a computer 1 that calculates a cloud height 47 that is the altitude of the cloud base of the cloud. The computer 1 includes a first altitude calculation unit 11 that identifies, from the parallax image 41 acquired by the stereo camera 3, a characteristic part that is a contour part of the cloud and feature extraction information 43 including information on an intermediate area surrounded by the characteristic part, and calculates an altitude (first altitude 45) for the characteristic part, a second altitude calculation unit 13 that calculates a second altitude 46 for the intermediate area surrounded by the characteristic part of the cloud based on the cloud base information 42 and the feature extraction information 43 acquired by the cloud base information acquisition device 2, and an altitude interpolation unit 14 that interpolates the cloud height based on the altitude (first altitude) corresponding to the characteristic part of the cloud and the altitude (second altitude) corresponding to the intermediate area (see, for example, FIG. 5).

[0069] The cloud height measurement method of the cloud height measurement system according to this embodiment includes a disparity image acquisition step of acquiring a disparity image corresponding to a predetermined length of disparity by imaging the same cloud with a stereo camera 3 having multiple cameras arranged a predetermined length apart, a cloud base information acquisition step of acquiring cloud base information related to the cloud base by an infrared camera 9 as an example of a cloud base information acquisition device, and a cloud height calculation step of calculating the cloud height by a computer. Among these, the cloud height calculation step includes a feature part recognition step in which the first altitude calculation unit 11 of the computer recognizes a feature part, which is a contour part of a cloud, from a parallax image acquired by the stereo camera 3; an intermediate area recognition step in which the first altitude calculation unit 11 of the computer 1 recognizes an intermediate area surrounded by the feature part; a first altitude calculation step in which the first altitude calculation unit 11 of the computer 1 identifies feature extraction information 43, which is information on the feature part and the intermediate area, and calculates an altitude (first altitude) for the feature part of the cloud; a second altitude calculation step in which the second altitude calculation unit 13 of the computer 1 calculates an altitude (second altitude) for the intermediate area based on the cloud base information and feature extraction information 43 acquired by the infrared camera 9; and an altitude interpolation step in which the altitude interpolation unit 14 of the computer 1 interpolates the altitude of the intermediate area based on the first altitude corresponding to the feature part of the cloud and the second altitude corresponding to the cloud base of the cloud (see FIG. 17, for example).

[0070] In this way, even if a stereo camera 3 that has difficulty measuring the cloud height of clouds with little pattern on its own is used, the altitude (first altitude) corresponding to the characteristic parts of the cloud is obtained based on the parallax image acquired from the multiple captured images obtained in the captured image acquisition step, and the altitude of the intermediate area surrounded by the characteristic parts of the cloud is interpolated, thereby making it possible to accurately measure the cloud height even for clouds with little pattern.

[0071] In this embodiment, the cloud base information acquisition device 2 is an infrared camera 9, and the second altitude calculation unit 13 creates an infrared camera temperature map based on an infrared image acquired by the infrared camera 9, and calculates the altitude (second altitude) for the intermediate region based on the infrared camera temperature map (the infrared camera altitude map 40 obtained from the infrared camera temperature map). In this way, the cloud height can be measured with high accuracy (see, for example, FIG. 8) while assuming the temperature in the sky according to a general altitude vs. temperature characteristic (see FIG. 9C). In addition, the second altitude calculation unit 13 creates an infrared camera temperature map based on an infrared image acquired by the infrared camera 9, identifies the cloud layers of the cloud based on the infrared camera temperature map (the infrared camera altitude map 40 obtained from the infrared camera temperature map), and determines the cloud base for each cloud layer of the cloud (see, for example, FIG. 8). In this way, the cloud height can be measured with high accuracy.

[0072] In this embodiment, the altitude interpolation unit 14 determines the cloud layer to which the cloud characteristic portion and the intermediate region belong based on an infrared camera temperature map as an example of a temperature map, and if the cloud characteristic portion and the intermediate region belong to the same cloud layer, it interpolates the altitude (second altitude) of the intermediate region based on the altitude (first altitude) of the cloud characteristic portion. In this way, since it has been confirmed that the cloud characteristic portion and the intermediate region belong to the same cloud layer, it is possible to accurately interpolate the altitude of the intermediate region and measure the cloud height.

[0073] In this embodiment, the first altitude includes the feature extraction information 43, and the second altitude includes information on the cloud layer of the cloud. The altitude interpolation unit 14, for example, determines whether there is an area where the feature part and the intermediate area belong to the same cloud layer based on an infrared camera temperature map, and if there is an area where the feature part and the intermediate area belong to the same cloud layer, interpolates the second altitude of the intermediate area based on the first altitude of the feature part. If the intermediate area does not belong to any cloud layer, it determines that the intermediate area is a blue sky area BS between multiple clouds, and does not interpolate the altitude (second altitude) of the intermediate area (see, for example, FIG. 11). In this way, it is possible to avoid measuring the altitude of the blue sky area BS without clouds.

[0074] In this embodiment, the first altitude includes feature extraction information 43, which includes information on the area in the parallax image where the parallax cannot be detected, and the altitude interpolation unit 14 determines that there is an invalid area 32 when the area in the parallax image 41 where the parallax cannot be detected is equal to or exceeds a predetermined rate (e.g., 10%), and interpolates the altitude (second altitude) of the intermediate area corresponding to the area where the parallax cannot be detected (see, e.g., FIG. 12). In this way, when there are many areas where the parallax cannot be detected, i.e., when there is a cloud without a pattern, the cloud height can be measured with high accuracy.

[0075] In this embodiment, the cameras 3a and 3b of the stereo camera 3 each include a fisheye lens or a wide-angle lens. In this way, the number of times the stereo camera 3 captures images can be adjusted according to the type of lens used.

[0076] (2) Second embodiment The second embodiment will be described with reference to Figures 19 to 21. In the second embodiment, the description of the same configuration and operation as in the first embodiment will be omitted, and the following description will focus on the differences from the first embodiment. In the second embodiment, the cloud height is measured for patternless clouds that cover the entire sky or clouds such as dense fog.

[0077] In the second embodiment, as in the first embodiment, a stereo camera 3 is used, and in areas where the clouds have no pattern and no parallax can be detected at all, a laser ranging device as an example of a cloud base information acquisition device 2 is used, as described below.

[0078] In the case of patternless clouds (or thick fog) like this, no parallax can be detected at all from the parallax image 41 captured by the cameras 3a and 3b of the stereo camera 3, so when converted into the stereo camera altitude map 30, there are missing areas of altitude data over almost the entire area.

[0079] In the first embodiment, the altitude map 50 was obtained by interpolating the stereo camera altitude map 30 using the infrared camera altitude map 40. However, the altitude obtained using an infrared thermography camera as an example of an infrared camera 9 has low absolute accuracy, and if pixels of the entire sky are interpolated using the infrared camera altitude map 40, the accuracy of the cloud altitudes that can be grasped by the altitude map 50 may deteriorate.

[0080] Therefore, in the second embodiment, as a method for improving the absolute accuracy of cloud height even for clouds including areas where parallax cannot be detected by the stereo camera 3, altitude is replaced using a laser ranging device as an example of the cloud base information acquisition device 2. In this embodiment, the laser ranging device is, for example, a ceilometer 21.

[0081] The ceilometer 21 is an optical device that uses a laser. Specifically, the ceilometer 21 is a device that irradiates a powerful laser beam, detects an optical signal scattered or reflected at the cloud base, and can measure the cloud height with an accuracy of several meters from the arrival time and signal strength.

[0082] Fig. 19 shows an example of the arrangement of sensors for measuring cloud height in the second embodiment. In the example shown, the sensors include a stereo camera 3 equipped with two cameras 3a and 3b as in the first embodiment, and one ceilometer 21. As described above, a description of the same configuration as in the first embodiment will be omitted.

[0083] The ceilometer 21 measures the cloud height, for example, in a narrow range directly above. In the illustrated example, the infrared camera 9 and the ceilometer 21 are disposed on the baseline length B of the stereo camera 3, but their positions do not have to be on the baseline length B.

[0084] 20 is a flow chart showing an example of the process of the altitude interpolation unit 14 using the ceilometer 21. As an input to the altitude interpolation unit 14, the first altitude 45 includes feature extraction information 43, and the feature extraction information 43 includes information on the area in which parallax cannot be detected in the parallax image 41. First, to determine whether or not there is a valid area 31 in which parallax can be detected, it is determined whether or not the area in which parallax cannot be detected is a predetermined ratio (for example, 90%) or more (step S67). If the area in which parallax cannot be detected is a predetermined ratio or more, it is determined that there is no valid area 31 and the entire area is an invalid area 32, and the altitude of the entire area is replaced by the cloud height measurement value measured by the ceilometer 21 (step S68). The replaced altitude is output as the cloud height 47.

[0085] In this embodiment, the first altitude includes feature extraction information 43, and the feature extraction information 43 includes information regarding regions where parallax cannot be detected in the parallax image 41. When creating the stereo camera altitude map 30 from the parallax image 41, the altitude interpolation unit 14 may determine that there is no valid region 31 if the regions where parallax cannot be detected in the parallax image 41 exist at a predetermined rate (e.g., 90%) or more, and may use the cloud height measurement value measured by the ceilometer 21 to interpolate the altitude (second altitude) of the intermediate region of the cloud corresponding to the region where parallax cannot be detected.

[0086] Fig. 21 shows an example of cloud height interpolation processing using the ceilometer 21. In the stereo camera altitude map 30 (see the upper center of Fig. 21) obtained from the parallax image 41 acquired by the cameras 3a and 3b of the stereo camera 3 shown in the upper left of Fig. 21, parallax cannot be detected, so the entire area is an invalid area 32. That is, in the case of clouds that are made up of areas without patterns and where no parallax can be detected at all, the invalid area 32 (area where altitude data is missing) occupies a large area in the stereo camera altitude map 30.

[0087] At this time, when the entire cloud or most of the cloud falls within the invalid area 32, specifically when the entire sky is covered with patternless clouds or there is thick fog, the altitude interpolation unit 14 interpolates the altitude using the cloud height measurement value measured by the ceilometer 21.

[0088] 21, the inputs to Calculator 1 are the parallax image 41 from the stereo camera and the cloud height measurement value from the ceilometer 21. For areas where no parallax can be detected, the altitude interpolation unit 14 performs a process of replacing the altitude data for the entire area with the cloud height measurement value from the ceilometer 21, and outputs a replaced altitude map 50. As a result, the cloud height 47 output by Calculator 1 matches the altitude of the cloud base measured by the ceilometer 21.

[0089] In this embodiment, the cloud base information acquisition device 2 is a ceilometer 21, the first altitude includes feature extraction information 43, the feature extraction information 43 includes information on regions in the parallax image 41 where parallax cannot be detected, and the altitude interpolation unit 14 determines that there is no valid region 31 when the parallax image 41 includes regions where parallax cannot be detected at a predetermined rate (e.g., 90%) or more, and uses the cloud height measurement value measured by the ceilometer 21 to interpolate the altitude of an intermediate region corresponding to the region where parallax cannot be detected (e.g., see FIG. 20). In this way, the cloud height can be accurately measured for clouds with few patterns where the region where parallax cannot be detected is equal to or greater than the predetermined rate.

[0090] (3) Third embodiment The third embodiment will be described with reference to Figures 22 to 24. In the third embodiment, the description of the same configuration and operation as in the first and second embodiments will be omitted, and the following description will focus on the differences from the first and second embodiments. In the third embodiment, in addition to the clouds that cover part of the sky in the first embodiment, the cloud height is also measured for clouds that include an area in which the parallax cannot be detected by the stereo camera 3, such as patternless clouds or dense fog that cover the sky in the second embodiment.

[0091] The third embodiment is a combination of the first and second embodiments, and uses a stereo camera 3, as well as an infrared camera 9 and a ceilometer 21, which is a laser distance measuring device, as an example of a cloud base information acquisition device 2.

[0092] Fig. 22 shows an example of the arrangement of sensors for measuring cloud height in the third embodiment. In the example shown, the sensor is a stereo camera 3 equipped with two cameras 3a and 3b, and the cloud base information acquisition device 2 is equipped with not only one infrared camera 9 but also one ceilometer 21. As described above, the description of the same configuration as the first embodiment will be omitted.

[0093] 23 shows an example of the process of the altitude interpolation unit 14 in the third embodiment. In step S67, the altitude interpolation unit 14 judges whether or not there is a valid area 31 in the stereo camera altitude map 30. If it is judged in step S31 that there is a valid area, that is, for an area where disparity can be detected, the following flow chart is the same as that of the first embodiment, and therefore the description thereof will be omitted.

[0094] On the other hand, if it is determined in step S67 that there is no valid area, i.e., there is no area where parallax can be detected at all, partial interpolation using infrared camera altitude map 40 as in the first embodiment is difficult, so as in the second embodiment, first, the cloud height directly above is measured using ceilometer 21 to obtain a cloud height measurement value. In this case, altitude interpolation unit 14 replaces stereo camera altitude map 30 with the cloud height measurement value measured by ceilometer 21 to obtain an interpolated altitude map 50 (step S68).

[0095] Fig. 24 is a flowchart showing an example of the procedure of cloud height measurement processing in the third embodiment. Three types of sensors are provided as sensors for measuring cloud height: a stereo camera 3, an infrared camera 9, and a ceilometer 21. In the flowchart shown in Fig. 24, explanations of the same procedures as those in the flowchart shown in Fig. 18 in the first embodiment will be omitted.

[0096] In step S67, the altitude interpolation unit 14 determines whether or not there is a valid area 31 in the parallax image 41 or the stereo camera altitude map 30. If it is determined in step S67 that there is a valid area, that is, for an area where parallax can be detected, the following flow chart is the same as that of the first embodiment, and therefore the description thereof will be omitted.

[0097] On the other hand, if it is determined in step S67 that there is no valid area, that is, if there is no area in which disparity can be detected at all, the flow chart thereafter is the same as that of the second embodiment, and therefore the description thereof will be omitted.

[0098] In the third embodiment, for the invalid area, one of three types of processing is performed: no altitude interpolation, interpolation based on information measured by the infrared camera 9, or interpolation based on information measured by the ceilometer 21, and the cloud height 47 is calculated and output (step S7).

[0099] In this embodiment, the cloud base information acquisition device 2 is an infrared camera 9 and a ceilometer 21, the first altitude includes an effective area and an invalid area as the feature extraction information 43, and the altitude interpolation unit 14 judges the presence or absence of an effective area 31 as an area where parallax can be detected in the parallax image 41, and if there is no effective area 31, replaces the altitude of the entire photographed area with the cloud height measurement value measured by the ceilometer 21, and judges the presence or absence of an invalid area 32 as an area where parallax cannot be detected in the parallax image 41, and if there is an invalid area 32, interpolates the altitude of the invalid area 32 based on the infrared camera altitude map 40 measured by the infrared camera 9 (see, for example, FIG. 23). In this way, the cloud height can be measured even in a situation where the clouds covering the entire sky have no pattern or in a situation where the cloud pattern cannot be identified such as thick fog.

[0100] (4) Fourth embodiment The fourth embodiment will be described with reference to Figs. 25 to 30. In the fourth embodiment, the description of the same configuration and operation as the first to third embodiments will be omitted, and the following description will focus on the differences from the first to third embodiments. In the fourth embodiment, in addition to cloud height measurement, cloud shape classification (hereinafter also referred to as "cloud shape recognition") is performed. In the fourth embodiment, unlike the first and third embodiments, instead of using the infrared camera 9, a case where a visible light camera 23 is used to obtain a visible light image as a sensor for cloud shape recognition, and a case where an imaging unit 3a constituting a stereo camera 3 is used instead of using this visible light camera 23, will be described.

[0101] 25A and 25B show examples of sensor arrangement in the fourth embodiment. In the example of Fig. 25A, a configuration is shown that includes a stereo camera 3 having two cameras 3a and 3b, one ceilometer 21, and one visible light camera 23. Here, the visible light camera 23 has a wide-angle lens, is installed on, for example, a PTZ head, and captures an image of the entire sky by adjusting the pan angle and tilt angle.

[0102] For example, when a wide-angle lens with a viewing angle of 60° is used for the visible light camera 23, an imaging pattern can be constructed in which the pan angle is changed by 360° in 45° increments and the tilt angle is changed by 15°, 60°, and 75°, so that the entire sky can be covered with multiple visible light images taken by the visible light camera 23, and the entire sky can be imaged in 1 to 2 minutes.

[0103] In the example of Fig. 25B, compared to the sensor arrangement example shown in Fig. 25A, the visible light camera 23 is omitted, and the function of the visible light camera 23 is substituted by a stereo camera 3. In the configuration shown in Fig. 25B, one stereo camera is used for two purposes, and a visible light image acquired by the imaging unit 3a of the stereo camera 3 is used for cloud shape recognition, and a parallax image 41 acquired by the stereo camera 3 is used for cloud height measurement.

[0104] 26 is a block diagram showing an example of the configuration and functions of a cloud height measurement system according to the fourth embodiment. The cloud height measurement system according to this embodiment includes a stereo camera 3, a cloud base information acquisition device 2 that acquires cloud base information related to the cloud base, and a computer 1 that calculates the cloud height. The cloud base information acquisition device in this embodiment is, for example, an imaging unit 3a constituting the stereo camera 3, and this imaging unit 3a has both a function of acquiring a parallax image by the stereo camera 3 and a function of acquiring a visible light image as cloud base information that is the output of the cloud base information acquisition device 2. The computer 1 is equipped with a first altitude calculation unit 11 that identifies feature extraction information 43 including information on a characteristic part, which is the outline of a cloud, and an intermediate area surrounded by the characteristic part from a parallax image 41 acquired by a stereo camera, and calculates a first altitude 45 for the characteristic part, a second altitude calculation unit 13 that calculates a second altitude 46 for the intermediate area surrounded by the characteristic part based on the cloud base information 42 and the feature extraction information 43 acquired by the cloud base information acquisition device 2, and an altitude interpolation unit 14 that interpolates a cloud height 47 of the cloud based on the first altitude and the second altitude.

[0105] Fig. 27A and Fig. 27B are flowcharts showing an example of the processing of the second altitude calculation unit 13 in the fourth embodiment. Fig. 27A corresponds to the sensor configuration of Fig. 25A and is a flowchart when a visible light image used for cloud shape recognition is obtained by the visible light camera 23. Fig. 27B corresponds to the sensor configuration of Fig. 26B and is a flowchart when a visible light image used for cloud shape recognition is obtained by the imaging unit 3a constituting the stereo camera 3.

[0106] In the cloud base processing unit 12 included in the second altitude calculation unit 13, cloud shape recognition is performed to estimate the cloud shape of the cloud included in the visible light image (step S56). The cloud base processing unit 12 uses a cloud shape recognition model created in advance by machine learning, for example, to estimate the cloud shape of the cloud included in the visible light image and generate a cloud shape map indicating the position and shape of the cloud. Here, the cloud shape is a classification of clouds according to their shape. For example, in a classification called 10 types of cloud shapes, the shape of the cloud is classified into any of the following cloud shapes: cirrus, cirrocumulus, cirrostratus, altocumulus, altostratus, nimbostratus, stratocumulus, stratus, cumulus, and cumulonimbus. In addition, as a method of cloud shape recognition, for example, the method described in JP 2023-56092 A can be adopted.

[0107] In this cloud shape recognition, clouds contained in a visible light image are classified into one of the cloud shapes for each pixel. When multiple clouds are contained in a visible light image and the cloud shapes are different, they are classified into the cloud shapes of each cloud. In other words, if the cloud shapes are different, the cloud layers can be separated even if multiple clouds overlap. On the other hand, when multiple clouds are contained in a visible light image and the same cloud shape overlaps in two or more layers, the cloud shape recognition may recognize the same cloud shape as a single mass, and overlapping clouds of the same cloud shape at different altitudes cannot be separated even if the cloud shape recognition is applied alone.

[0108] As explained in the first embodiment, the cloud height can be obtained by performing cloud height measurement using the stereo camera 3. In the fourth embodiment, when clouds of the same cloud type but at different heights overlap, the cloud height can be measured after separating cloud layers by combining the results with the cloud height measurement results by the stereo camera 3 described above.

[0109] As described above, the first altitude calculation unit 11 obtains altitude information by creating the stereo camera altitude map 30 based on the parallax image 41 acquired by the stereo camera 3 (step S4). The second altitude calculation unit 13 integrates the cloud type classification based on the visible light image and the altitude information based on the parallax image to identify the cloud layer (step S57). The following flow chart is the same as that of the first embodiment, so the description thereof will be omitted.

[0110] 28 shows an example of the process of the altitude interpolation unit 14 in the fourth embodiment. Steps S31, S32, and S33 are the same as those in the third embodiment, and therefore their explanation will be omitted. If it is determined in step S33 that there is an invalid area, the altitude of the invalid area is interpolated based on the result of cloud layer separation using the result of cloud shape recognition by the visible light camera (step S44).

[0111] 29A and 29B each show an example of altitude interpolation processing for an invalid region 32 when multiple layers of clouds exist. In the illustrated example, the dashed lines show an example of the interpolated altitude. In the fourth embodiment, since the infrared camera 9 is not used as described above, altitude interpolation using the infrared camera altitude map 40 using the infrared camera 9 is not performed, but the altitude interpolation unit 14 performs altitude interpolation as follows.

[0112] In this embodiment, the cloud base processing unit 12 included in the second altitude calculation unit 13 classifies cloud shapes from visible light images captured by the visible light camera 23 (or the stereo camera 3 as a substitute thereof) and performs cloud layer separation to determine which cloud layer each of the cloud's characteristic parts and intermediate region belongs to. The altitude interpolation unit 14 interpolates the altitude (second altitude) of the intermediate region based on the altitude (first altitude) corresponding to the cloud's characteristic parts determined to belong to the same cloud layer as the result of this cloud layer separation.

[0113] The example shown in Fig. 29A shows a case where two cloud layers have different cloud shapes. In this case, if low clouds CL and high clouds CH have different cloud shapes, cloud layer separation can be performed by cloud shape recognition. For low clouds CL, the effective area 31 at altitude 31a and the invalid area 32 at altitude 32a (corresponding to the "invalid area A" shown in the figure) are classified as the same cloud shape, so the altitude interpolation unit 14 may mechanically interpolate the altitude of the invalid area 32a by a method such as, for example, interpolation so as to connect the altitudes 31a on both sides of the effective area outside the effective area.

[0114] On the other hand, for high clouds CH, if they are classified as clouds of a different cloud type from low clouds CL by cloud type recognition, the altitude interpolation unit 14 may interpolate the invalid area 32 of altitude 32a (corresponding to the "invalid area B" shown in the figure) with the same altitude 31a as the valid area 31 classified as the same cloud type. Therefore, the altitude interpolation unit 14 can interpolate the altitude 32a for the invalid area 32 as shown by the dashed line in the figure.

[0115] The example shown in Fig. 29B shows an example in which two cloud layers have the same cloud shape, resulting in a misidentification of altitude, unlike this embodiment. In this case, in cloud shape recognition, since the two clouds have the same cloud shape, they are recognized as a single mass of clouds of the same cloud shape. As in this embodiment, the altitude interpolation unit 14 may mechanically interpolate altitude 32a for low-level clouds CL by connecting the invalid area 32 of altitude 32a (corresponding to the "invalid area A" shown in the figure) to the valid area altitude 31a outside of it.

[0116] On the other hand, for high clouds CH, it is possible to recognize that the altitude 31a of the effective area is higher than the altitude of low clouds CL, but it is not possible to determine that the clouds are separated into two layers. In other words, if left as is, the altitude interpolation unit 14 will interpolate the altitude of the invalid area (corresponding to the "invalid area B" in the figure) as shown by a dashed line, and it will be classified as a single cloud mass. In this way, when cloud height measurement and cloud shape recognition are performed using the stereo camera 3 or the visible light camera 23 without using the infrared camera 9, there is a possibility that cloud layers will be erroneously recognized for multiple clouds CL and CH at different altitudes as shown in Figure 29B.

[0117] Figures 30A and 30B are flowcharts showing an example of the procedure of the cloud height measurement method in the fourth embodiment. Note that in the flowcharts shown in Figures 30A and 30B, differences from the flowcharts shown in Figures 18 and 24 described above will be mainly described, and descriptions of similar points will be omitted as appropriate.

[0118] FIG. 30A corresponds to FIG. 25A and shows a case where cloud shape recognition is performed based on a visible light image acquired using a visible light camera 23, and cloud height measurement is performed from a parallax image 41 acquired using a stereo camera 3.

[0119] In step S1, the stereo camera 3 acquires images 4a and 4b captured by the cameras 3a and 3b of the stereo camera 3, detects the parallax, and creates a parallax image 41. In step S4, the first altitude calculation unit 11 calculates a first altitude by obtaining a stereo camera altitude map 30 as an example of altitude distribution from the parallax image 41. In addition, in step S21, the ceilometer 21 acquires a cloud height measurement value by measuring the cloud height of the cloud directly above.

[0120] In step S67, the altitude interpolation unit 14 determines whether or not there is an effective area 31, in the same manner as described above, and if there is no effective area 31, replaces the altitude with the cloud height measurement value of the ceilometer 21 (step S68).

[0121] The visible light camera 23 captures the entire sky and acquires a visible light image (step S23). The second altitude calculation unit 13 classifies the cloud shape by performing cloud shape recognition from the visible light image acquired by the visible light camera 23 (step S56). The second altitude calculation unit 13 identifies the cloud layers by performing the cloud layer separation based on the result of the cloud shape recognition and the stereo camera altitude map 30 (step S57). The second altitude calculation unit 13 estimates the cloud shape and second altitude for each cloud layer using the result of this cloud layer separation (step S5).

[0122] In step S74, the altitude interpolation unit 14 determines whether or not there is an invalid area 32, and if there is an invalid area 32, it interpolates the altitude for areas in the clouds where parallax cannot be detected, in combination with the results of the cloud shape recognition and cloud layer separation described above (step S69).

[0123] As a result, in step S7, the altitude interpolation unit 14 calculates the cloud height for each cloud layer.

[0124] Fig. 30B corresponds to Fig. 25B, and shows a case where the stereo camera 3 plays two roles: altitude measurement and cloud shape recognition. In the flowchart shown in Fig. 30A described above, the visible light camera 23 is used to acquire a visible light image, whereas in the flowchart shown in Fig. 30B, the imaging unit 3a constituting the stereo camera 3 is used instead of the visible light camera 23. The flowchart shown in Fig. 30B is the same as the flowchart shown in Fig. 30A, and therefore a description of similar processes will be omitted.

[0125] 30B, the stereo camera 3 releases the shutter twice, for example, with different exposure times, for the visible light image for cloud shape recognition and the parallax image 41 for cloud height measurement. Alternatively, the cameras 3a and 3b of the stereo camera 3 may take the visible light image for cloud shape recognition and the parallax image 41 for cloud height measurement by releasing the shutter once.

[0126] The second altitude calculation unit 13 performs cloud shape recognition to classify cloud shapes based on the visible light image captured by the imaging unit 3a constituting the stereo camera 3 (step S56). Similarly, the second altitude calculation unit 13 performs cloud layer separation to identify cloud layers based on the result of the cloud shape recognition and the stereo camera altitude map 30 (step S57). The second altitude calculation unit 13 uses the result of this cloud layer separation to estimate the cloud shape and second altitude of the cloud for each cloud layer (step S6).

[0127] In this embodiment, the cloud base information acquisition device is a visible light camera 23, the feature extraction information 43 includes altitude information based on the parallax image 41 acquired by the stereo camera 3, the second altitude calculation unit 13 classifies the cloud shape from the visible light image acquired by the visible light camera 23 or the imaging unit 3a constituting the stereo camera 3 to perform cloud shape recognition, and performs cloud layer separation based on the result of the cloud shape recognition and the stereo camera altitude map 30, and the altitude interpolation unit 14 estimates the cloud shape and cloud height of the cloud for each cloud layer using the result of this cloud layer separation (for example, see FIG. 27). In this way, it is possible to accurately classify the cloud shape from the visible light image acquired by the visible light camera 23 provided independently of the stereo camera 3 and perform cloud shape recognition.

[0128] In this embodiment, the cloud base information acquisition device 2 is the imaging unit 3a constituting the stereo camera 3, the feature extraction information 43 includes altitude information based on the parallax image 41 acquired by the stereo camera 3, and the second altitude calculation unit 13 classifies the cloud shape from the visible light image acquired by the imaging unit 3a constituting the stereo camera 3, and identifies the cloud layer from the distribution of the cloud shape and the altitude information based on the parallax image 41 acquired by the imaging unit 3a constituting the stereo camera 3 (see, for example, Figures 26 and 27B). In this way (when one stereo camera 3 plays two roles as shown in Figure 30B), the visible light camera 23 is not required, and the number of sensors to be installed can be minimized.

[0129] In this embodiment, the cloud base information acquisition device is a visible light camera 23 and a ceilometer 21, the first altitude includes a valid area and an invalid area as feature extraction information 43, and the altitude interpolation unit 14 determines whether or not there is a valid area in the parallax image 41 where parallax can be detected, and if there is no valid area, replaces the cloud height with the cloud height measurement value measured by the ceilometer 21, and determines whether or not there is an invalid area in the parallax image 41 where parallax cannot be detected, and if there is an invalid area, interpolates the altitude of the invalid area based on the cloud layer calculated by the visible light camera 23 (see, for example, Figure 28).

[0130] (5) Fifth embodiment The fifth embodiment will be described with reference to Figures 31 and 32. In the fifth embodiment, the description of the same configuration and operation as the first to fourth embodiments will be omitted, and the following description will focus on the differences from the first to fourth embodiments. In the fifth embodiment, in addition to the method of recognizing cloud shapes using visible light images obtained using the visible light camera 23 in the fourth embodiment, the cloud height estimation accuracy will be improved by performing cloud layer separation using an infrared camera 9.

[0131] In the case where the infrared camera 9 is not used as in the fourth embodiment, the altitude interpolation by the altitude interpolation unit 14 may erroneously recognize the cloud layer as described above when clouds of the same cloud shape overlap in multiple layers as shown in Fig. 29B. Therefore, in the fifth embodiment, when there is an invalid area 32, the altitude interpolation unit 14 refers to the result of cloud layer separation using the infrared camera 9, determines which cloud layer the invalid area 32 corresponds to, and then interpolates the altitude of the corresponding cloud. By performing the task of collating with the result of cloud layer separation using such an infrared camera 9, the altitude interpolation unit 14 can avoid the risk of erroneously recognizing the cloud layer when performing the altitude interpolation.

[0132] This method is necessary when clouds of the same cloud type overlap and the height of that part needs to be interpolated. Regarding the frequency of occurrence of such a situation, since the phenomenon in which multiple layers of the same cloud type overlap is common among the 10 types of cloud types mentioned above, stratus clouds, it is expected that the method of using an infrared camera9 in combination will improve the accuracy of estimating cloud height.

[0133] Fig. 31 shows an example of the arrangement of sensors in the fifth embodiment. In the example shown, a stereo camera 3 having two cameras 3a and 3b, an infrared camera 9, a visible light camera 23, and a ceilometer 21 are provided. The stereo camera 3 and the infrared camera 9 are equipped with, for example, a wide-angle lens. Note that the infrared camera 9 and the visible light camera 23 may be equipped with a fisheye lens instead of a wide-angle lens.

[0134] The infrared camera 9 and the visible light camera 23 are arranged, for example, on a PTZ head, and capture an image of the whole sky by adjusting the pan angle and tilt angle. Note that the infrared camera 9 and the visible light camera 23 may be arranged on the same PTZ head.

[0135] Fig. 32 is a flowchart showing an example of the procedure of the cloud height measurement method in the fifth embodiment. In the illustrated example, the cloud height is measured based on the stereo camera altitude map 30 using the stereo camera 3, and the cloud shape is measured based on a visible light image using the visible light camera 23. Note that in the flowchart shown in Fig. 32, explanations of processes similar to those in the flowcharts shown in Figs. 17 and 18 in the first embodiment, the flowchart shown in Fig. 24 in the third embodiment, and the flowchart shown in Fig. 30A in the fourth embodiment are omitted as appropriate.

[0136] In the fifth embodiment, similarly to the third embodiment, the altitude interpolation unit 14 determines whether or not there is an effective area 31 as a result of the above-mentioned cloud layer separation (step S67), and if it determines that there is no effective area 31, it interpolates the altitude of the invalid area 32 by replacing the altitude with the cloud height measurement value directly above measured by the ceilometer 21, similarly to the above-mentioned embodiment (step S68).

[0137] In addition, the altitude interpolation unit 14 determines whether or not there is an invalid area 32 (step S64), and if it determines that there is an invalid area 32, it interpolates the altitude of the invalid area 32 based on the infrared camera altitude map 40, which is an example of an altitude map acquired using the infrared camera 9 (step S6).

[0138] In this embodiment, the cloud base information acquisition device 2 is an infrared camera 9, a visible light camera 23, and a ceilometer 21, the first altitude includes valid areas and invalid areas as feature extraction information 43, and the second altitude calculation unit 13 classifies the cloud shape from the visible light image captured by the visible light camera 23, creates an infrared temperature map based on the infrared image acquired by the infrared camera 9, and calculates the second altitude based on the infrared temperature map (see, for example, Figure 32).

[0139] It should be noted that the present invention is not limited to the above-described embodiments, and includes various modifications and equivalent configurations within the spirit and scope of the appended claims. For example, the above-described embodiments have been described in detail to clearly explain the present invention, and the present invention is not necessarily limited to those including all of the described configurations.

[0140] In the above embodiment, the outline of the cloud is given as an example of a characteristic part of the cloud, but other parts of the cloud may be given as examples. In the above embodiment, a ceilometer is given as an example of a laser distance measuring device, but any device capable of measuring cloud height, such as a radar echo or an ultrasonic distance measuring device, may be adopted. In addition, in the above embodiment, a camera capable of capturing an image of the entire sky is given as an example of the sensor, but a fisheye camera capable of capturing an image of the entire sky at once may be used, or the entire sky may be divided and captured, or only a part of the entire sky where cloud height measurement is required may be captured. There is no particular limitation as long as the means is capable of measuring clouds. [Industrial Applicability]

[0141] The present invention can be applied to a cloud height measurement system relating to a technique for measuring the cloud height using a stereo camera when measuring the altitude of the clouds from the ground. [Explanation of symbols]

[0142] 1……Calculator 2...Cloud base information acquisition device 3...Stereo camera 3a, 3b, ..., 3t ... multiple imaging units 4a: Image captured by the imaging unit 3a 4b: Image captured by the imaging unit 3b 5...Tripod 7...Rail 8... Cloud height measurement system 9...Infrared camera 9a……Infrared camera temperature map 11...First altitude calculation section 12... Cloud base processing section 13...Second altitude calculation section 14...Altitude Interpolation Section 15……Cloud height calculation section 21...Ceilometer 30...Stereo camera altitude map 31... Effective Area 31a……Cloud altitude of effective area 32……Void area 32a……Cloud height of invalid area 40...Infrared camera altitude map 41... Parallax image 42... Cloud base information 43...Feature extraction information 45...First altitude 46...Second Altitude 47……Cloud height 50...Altitude Map B……baseline length C……cloud

Claims

1. A stereo camera that acquires a parallax image corresponding to the parallax of a predetermined length by imaging the same cloud with multiple imaging units arranged at a predetermined distance apart, A cloud base information acquisition device that acquires cloud base information relating to the cloud base of the aforementioned cloud, The system includes a computer for calculating cloud height, which is the altitude of the cloud base of the aforementioned cloud, The aforementioned computer is A first altitude calculation unit identifies feature extraction information from the disparity image acquired by the stereo camera, which includes information on the feature portion that is the outline of the cloud and the intermediate region surrounded by the feature portion, and calculates a first altitude for the feature portion. A second altitude calculation unit calculates a second altitude for the intermediate region based on the cloud base information and the feature extraction information, An altitude interpolation unit that interpolates the cloud height based on the first altitude and the second altitude, A cloud height measurement system characterized by having the following features.

2. The cloud base information acquisition device is an infrared camera. The second advanced computing unit described above is: A temperature map is created based on the infrared image acquired by the infrared camera, and the second altitude is calculated based on the temperature map. The cloud height measurement system according to feature 1.

3. The second advanced computing unit described above is: A temperature map is created based on the infrared image acquired by the infrared camera, and the cloud layer of the cloud is identified based on the temperature map. The cloud base is determined for each cloud layer of the aforementioned cloud. The cloud height measurement system according to claim 2, characterized by the features described above.

4. The advanced interpolation unit is Determine whether there is a region in which the aforementioned characteristic portion and the aforementioned intermediate region belong to the same cloud layer. If there is a region in which the feature portion and the intermediate region belong to the same cloud layer, the second altitude of the intermediate region is interpolated based on the first altitude of the feature portion. If the intermediate region does not belong to any cloud layer, it is determined that the intermediate region is a region of blue sky between multiple clouds, and the second altitude of the intermediate region is not interpolated. The cloud height measurement system according to claim 2, characterized by the features described above.

5. The feature extraction information includes information about the region in the disparity image where the disparity cannot be detected, The aforementioned advanced interpolation unit is If a predetermined percentage or more of the area in the disparity image where the disparity cannot be detected exists, it is determined that there is an invalid area, and the second height corresponding to the area where the disparity cannot be detected is interpolated. The cloud height measurement system according to claim 2, characterized by the features described above.

6. The cloud base information acquisition device is a ceilometer. The feature extraction information includes information about the region in the disparity image where the disparity cannot be detected. The aforementioned advanced interpolation unit is If a predetermined percentage or more of the parallax image contains areas where parallax cannot be detected, it is determined that there is no effective area, and the second altitude corresponding to the areas where parallax cannot be detected is interpolated using the cloud height measurement value measured by the siemeter. The cloud height measurement system according to feature 1.

7. The cloud base information acquisition device consists of an infrared camera and a ceilometer. The aforementioned advanced interpolation unit is In the parallax image, it is determined whether or not there is an effective region in which the parallax can be detected. If there is no effective region, the cloud height is replaced with the cloud height measurement value measured by the siemeter. The system determines whether there is an invalid region in the parallax image where the parallax cannot be detected, and if there is an invalid region, it interpolates the altitude of the invalid region based on the cloud temperature map measured by the infrared camera. The cloud height measurement system according to feature 1.

8. The cloud base information acquisition device is a visible light camera. The feature extraction information includes altitude information based on the disparity image acquired by the stereo camera, The second advanced computing unit described above is: The visible light image acquired by the visible light camera is used to classify the cloud shape of the clouds, and the cloud layer is identified from the distribution of the cloud shapes and the altitude information based on the disparity image acquired by the stereo camera. The cloud height measurement system according to feature 1.

9. The cloud base information acquisition device is the imaging unit that constitutes the stereo camera. The feature extraction information includes altitude information based on the disparity image acquired by the stereo camera, The second advanced computing unit described above is: The system classifies the cloud shape from the visible light image acquired by the imaging unit of the stereo camera, and identifies the cloud layer from the distribution of the cloud shape and altitude information based on the disparity image acquired by the imaging unit of the stereo camera. The cloud height measurement system according to feature 1.

10. The cloud base information acquisition device consists of a visible light camera and a ceilometer. The aforementioned advanced interpolation unit is In the aforementioned disparity image, determine whether or not there is an effective region in which the disparity can be detected. If the effective area is not available, the cloud height is replaced with the cloud height measurement value measured by the siemeter, The system determines whether there is an invalid region in the parallax image where the parallax cannot be detected, and if there is an invalid region, it interpolates the altitude of the invalid region based on the cloud layer of the cloud calculated by the visible light camera. The cloud height measurement system according to feature 1.

11. The cloud base information acquisition device comprises an infrared camera, a visible light camera, and a siemeter. The second advanced computing unit described above is: The cloud shape of the cloud is classified from the visible light image acquired by the visible light camera. A temperature map is created based on the infrared image acquired by the infrared camera, and the second altitude is calculated based on the temperature map. The cloud height measurement system according to feature 1.

12. Each of the multiple imaging units in the stereo camera is equipped with either a fisheye lens or a wide-angle lens. The cloud height measurement system according to feature 1.

13. Image acquisition step: Acquire multiple captured images of the same cloud using a stereo camera equipped with multiple imaging units arranged at a predetermined distance apart. A parallax image acquisition step involves acquiring a parallax image corresponding to the predetermined length of parallax from the plurality of captured images, A cloud base information acquisition step in which a cloud base information acquisition device acquires cloud base information relating to the cloud base of the cloud, The computer includes a cloud height calculation step that calculates the cloud height, which is the altitude of the cloud base of the cloud, The cloud height calculation step described above is: A feature recognition step in which a computer recognizes a feature portion which is the outline portion of the cloud from the disparity image acquired by the stereo camera, An intermediate region recognition step in which a computer recognizes an intermediate region enclosed by the feature portion, A first altitude calculation step in which the computer identifies feature extraction information which is information of the feature portion and the intermediate region, and calculates a first altitude for the feature portion, A second altitude calculation step in which the computer calculates a second altitude for the intermediate region based on the cloud base information and feature extraction information acquired by the cloud base information acquisition device, The computer performs an altitude interpolation step in which it interpolates the cloud height based on the first altitude and the second altitude, A method for measuring cloud height, characterized by including the following:

Images