System, computer, and program

The system uses a stereo camera and ceilometer integration with satellite data to enhance cloud height measurement accuracy, addressing limitations in existing technologies and improving weather reporting precision.

JP2024171672AInactive Publication Date: 2024-12-12HITACHI LTD

Patent Information

Application Number
JP2023088812
Authority / Receiving Office
JP · JP
Patent Type
Applications
Current Assignee / Owner
Filing Date
2023-05-30
Publication Date
2024-12-12
Estimated Expiration
Not applicable · inactive patent

AI Technical Summary

Technical Problem

Existing cloud height measurement technologies, such as those using ceilometers and far-infrared cameras, struggle with accuracy and limited horizontal measurement ranges, particularly in measuring cloud heights over the entire sky.

Method used

A system utilizing a stereo camera to capture stereo images, calculate parallax for cloud heights, and integrate ceilometer measurements with satellite data to correct cloud heights, generating precise weather reports.

Benefits of technology

Enables high-precision measurement of cloud heights across the entire sky, improving the accuracy of weather reporting systems like METAR.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 2024171672000001_ABST
    Figure 2024171672000001_ABST
Patent Text Reader

Abstract

To measure cloud height over the whole sky with high accuracy.SOLUTION: A computer including one or more processors and one or more memory resources, a program, or a system comprising the computer and a stereo camera is provided. The processor acquires a stereo image obtained by imaging a sky in which a cloud may exist from the stereo camera, extracts a parallax at a corresponding point of the stereo image to acquire a first cloud height representing a distance to the cloud on the basis of the parallax, determines a fourth cloud height on the basis of at least one of the first cloud height, a second cloud height representing cloud bottom height, and a third cloud height representing cloud top height, and generates a weather report formula including the fourth cloud height.SELECTED DRAWING: Figure 1
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] The present invention relates to a system, a computer, and a program. [Background technology]

[0002] For example, when controlling the takeoff and landing of aircraft at an airport, it is necessary to have constantly changing weather information around the airport. METAR (Meteorological Aerodrome Report) is a type of weather report format for reporting aviation weather information, and is used to grasp the weather conditions at airports, air bases, etc.

[0003] METAR includes information on various observational items such as wind direction, wind speed, visibility, weather, cloud height, etc. Of these observational items, the measurement of cloud height in particular is based on the observer's experience, and automation has not progressed much.

[0004] Conventionally, there have been devices for measuring cloud height, such as a ceilometer that uses a laser. However, although the ceilometer can measure the cloud height directly above, it cannot measure the cloud height over the entire sky because its horizontal measurement range is narrow.

[0005] Regarding cloud height measurement, for example, Patent Document 1 describes an all-sky cloud height measurement device comprising: a local cloud height measurement unit that receives data from a laser and measures multiple cloud base heights in a local range; a low-accuracy cloud height estimation unit that receives all-sky image data from a sensor and estimates the relative heights of the multiple cloud bases based on the image data; and a cloud height correction unit that accepts the intersection of the laser measurement range and the multiple cloud bases, and corrects the multiple cloud bases estimated by the low-accuracy cloud height estimation unit using the intersection and the multiple cloud base heights measured by the local cloud height measurement unit. [Prior art documents] [Patent documents]

[0006] [Patent Document 1] Patent Publication No. 2022-13152 Summary of the Invention [Problem to be solved by the invention]

[0007] The technology described in Patent Document 1 is a technology that combines a far-infrared camera (infrared thermo camera) and a ceilometer, but there is room for improvement in the accuracy of cloud height measurement.

[0008] The present invention has been made in consideration of the above-mentioned points, and has an object to make it possible to measure cloud heights over the entire sky with high accuracy. [Means for solving the problem]

[0009] The present application includes a number of means for solving at least some of the above problems, examples of which are as follows.

[0010] In order to solve the above problem, a computer according to one aspect of the present invention comprises: A computer having one or more processors and one or more memory resources; has. And, the processor: Acquire, from the stereo camera, a stereo image of the sky where clouds may exist; extracting a disparity between corresponding points of the stereo images, and obtaining a first cloud height representing a distance to the cloud based on the disparity; Determine the fourth cloud height based on: The first cloud height, and At least one of a second cloud height representing a cloud base height and a third cloud height representing a cloud top height; A weather report formula including the fourth cloud height is generated. From another viewpoint, such a program executable by a computer is another aspect of the present invention, and a system including such a computer and a stereo camera is yet another aspect of the present invention. Effect of the Invention

[0011] According to the present invention, it is possible to measure cloud heights over the entire sky with high accuracy.

[0012] Problems, configurations, and effects other than those described above will become apparent from the following description of the embodiments. [Brief description of the drawings]

[0013] [Figure 1] FIG. 1 is a diagram showing an example of the configuration of a cloud height measurement system according to an embodiment of the present invention. [Diagram 2] FIG. 2 is a diagram illustrating an example of the configuration of a typical computer. [Diagram 3] FIG. 3 is a flowchart illustrating an example of the weather information processing. [Figure 4] FIG. 4 is a diagram for explaining a specific example of the process in step S10. [Diagram 5] FIG. 5 is a diagram for explaining a specific example of the process in step S10. [Figure 6] FIG. 6 is a diagram for explaining a specific example of the process in step S10. [Figure 7] FIG. 7 is a diagram for explaining a specific example of the process in step S10. [Figure 8] FIG. 8 is a diagram for explaining a specific example of the process in step S10. [Figure 9] FIG. 9 is a diagram showing an example of a weather report format. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0014] Hereinafter, an embodiment of the present invention will be described with reference to the drawings. The embodiment is an example for explaining the present invention, and is omitted and simplified as appropriate for clarity of explanation. The present invention can be implemented in various other forms. Unless otherwise limited, each component may be singular or plural. The position, size, shape, range, etc. of each component shown in the drawings may not represent the actual position, size, shape, range, etc. in order to facilitate understanding of the invention. Therefore, the present invention is not necessarily limited to the position, size, shape, range, etc. disclosed in the drawings. As examples of various information, descriptions may be given using expressions such as "table", "list", and "queue", but various information may be expressed in other data structures. For example, various information such as "XX table", "XX list", and "XX queue" may be expressed as "XX information". When explaining identification information, expressions such as "identification information", "identifier", "name", "ID", and "number" are used, but these can be replaced with each other. In all drawings for explaining the embodiment, the same members are generally given the same symbols, and repeated explanations are omitted. In the following embodiments, the components (including element steps, etc.) are not necessarily essential, unless otherwise specified or considered to be clearly essential in principle. When "consisting of A," "made of A," "having A," or "including A" is used, other elements are not excluded, unless otherwise specified to mean only that element. Similarly, in the following embodiments, when referring to the shape, positional relationship, etc. of components, etc., it includes things that are substantially similar or similar to that shape, etc., unless otherwise specified or considered to be clearly not essential in principle. In addition, "obtain" is intended to include at least the subject generating, calculating, and receiving from outside, as specific examples.

[0015] <Configuration example of cloud height measurement system 10 according to one embodiment of the present invention> FIG. 1 shows an example of the configuration of a cloud height measurement system 10 according to an embodiment of the present invention.

[0016] The cloud height measurement system 10 includes a weather information processing device 20, a stereo camera 30, a visible light camera 31, a ceilometer 32, and an external device 40. The weather information processing device 20 is connected to the stereo camera 30, the visible light camera 31, the ceilometer 32, and the external device 40 via a network N. The network N is a two-way communication network such as the Internet. Note that the weather information processing device 20 and the stereo camera 30, the visible light camera 31, and the ceilometer 32 may be directly connected to each other using, for example, a dedicated cable without going through the network N.

[0017] The weather information processing device 20 calculates the distance to clouds in the sky (cloud height) based on two images (stereo images) of the sky captured simultaneously by the stereo camera 30, and generates a weather report format (METAR, etc.) including the cloud height. Note that the stereo image is a set of images (for convenience, referred to as sub-stereo images) captured by each of the multiple (typically two) cameras included in the stereo camera 30, or an image obtained by combining multiple sub-stereo images.

[0018] The weather information processing device 20 includes a processing unit 21, a storage unit 22, a communication unit 23, a UI (User Interface) unit 24, and an internal clock 25. The processing unit 21 has the following functional blocks: a cloud height calculation unit 211, a cloud height correction unit 212, a weather report formula generation unit 213, an information acquisition unit 214, and a weather report formula output unit 215. The weather information processing device 20 is composed of a general computer such as a personal computer or a server computer. Note that when the weather information processing device 20 is realized by a computer, the computer does not necessarily have to be dedicated to weather information processing.

[0019] 2 shows an example of the configuration of a general computer 100 constituting the weather information processing device 20. The computer 100 includes a processor 101 such as a CPU (Central Processing Unit) or a GPU (Graphics Processing Unit), a memory 102 such as a DRAM (Dynamic Random Access Memory), a storage 103 such as an HDD (Hard Disk Drive) or an SSD (Solid State Drive), an input device 104 such as a keyboard, a mouse, a touch panel, an output device 105 such as a display, and a communication module 106 such as a NIC (Network Interface Card).

[0020] The computer 100 executes a program by the processor 101, and performs processing defined by the program while using storage resources (memory 102 and storage 103) and a communication module 106. Therefore, the subject of the processing performed by executing the program may be the processor 101. Similarly, the subject of the processing performed by executing the program may be a controller, device, system, computer, or node having a processor. These entities may be considered as one form of a computer. The subject of the processing performed by executing the program may be a calculation unit, and the calculation unit may include a dedicated circuit that performs a specific process. Here, the dedicated circuit is, for example, a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), or a complex programmable logic device (CPLD). From the viewpoint of executing a program and performing some kind of processing, an FPGA, an ASIC, or a CPLD may be considered as a processor.

[0021] The program may be installed in the computer 100 from a program source. The program source may be, for example, a program distribution server or a storage medium (typically non-volatile) readable by the computer 100. In addition, a program generally includes a plurality of instructions to be given to a processor. When the program source is a program distribution server, the program distribution server may include a processor and a storage resource for storing a program to be distributed, and the processor of the program distribution server may distribute the program to be distributed to other computers. In addition, in an embodiment, two or more programs may be realized as one program, and one program may be realized as two or more programs.

[0022] For example, the computer 100 realizes each of the functional blocks of the processing unit 21, namely, the cloud height calculation unit 211, the cloud height correction unit 212, the weather report formula generation unit 213, the information acquisition unit 214, and the weather report formula output unit 215, by the processor 101 executing a predetermined program loaded into the memory. However, some or all of these functional blocks may be realized as hardware using an integrated circuit or the like. Furthermore, these functional blocks may be realized by one computer 100 or by multiple computers 100. When realized by multiple computers 100, the multiple computers 100 may be connected via a network N, or may be distributed and located in remote locations.

[0023] Returning to Fig. 1, the cloud height calculation unit 211 uses stereo images obtained from the stereo camera 30 to calculate the cloud height (corresponding to the first cloud height of the present invention) based on the parallax. The cloud height correction unit 212 corrects the cloud height calculated by the cloud height calculation unit 211 based on the cloud base altitude directly above the ceilometer 32 measured by the ceilometer 32 (corresponding to the second cloud height of the present invention) and the cloud top altitude based on the satellite image (corresponding to the third cloud height of the present invention). The processing of the cloud height calculation unit 211 and the cloud height correction unit 212 will be described in detail later.

[0024] The weather report form generating unit 213 generates a weather report form such as METAR based on the cloud height corrected by the cloud height correcting unit 212 (corresponding to the fourth cloud height of the present invention) and the wind direction, wind speed, visibility, etc. acquired from the external device 40. Note that, in addition to METAR, the weather report form may be, for example, SPECI (Special Airport Actual Weather Report Form), TAF (Operational Airport Forecast Weather Report Form), TREND (Landing Airport Forecast Weather Report Form), VOLMET (Operational Airport Forecast Weather Report Form for Volmet Broadcasting), SCAN (Aviation Weather Observatory Actual Weather Report Form), etc.

[0025] The information acquisition unit 214 controls the stereo camera 30 via the communication unit 23 and the network N to acquire stereo images, and stores them in the storage unit 22 as image information 221. The information acquisition unit 214 also controls the visible light camera 31 via the communication unit 23 and the network N to acquire visible light images, and stores them in the storage unit 22 as image information 221. The information acquisition unit 214 also controls the ceilometer 32 via the communication unit 23 and the network N to acquire the cloud base altitude directly above the ceilometer 32, and stores them in the storage unit 22 as cloud base altitude information 225. The information acquisition unit 214 also connects to the external device 40 via the communication unit 23 and the network N, acquires wind direction and speed information 222, visibility information 223, and cloud top altitude information 224, and stores them in the storage unit 22.

[0026] The weather report form output unit 215 outputs the generated weather report form. A specific example of outputting the weather report form is to display the weather report form using the UI unit 24, or to transmit the weather report form to an external device of the weather information processing device 20 (using the communication unit 23). Note that the weather report form output unit 215 does not necessarily need to be executed every time a weather report form is generated. Also, the program that realizes the weather report form output unit 215 may be separate from the programs that realize the functional blocks other than the weather report form output unit 215.

[0027] The storage unit 22 is realized by storage resources such as the memory 102 and storage 103 included in the computer 100. The storage unit 22 stores image information 221, wind direction and speed information 222, visibility information 223, cloud top altitude information 224, cloud base altitude information 225, and a weather report format 226.

[0028] The image information 221 is composed of stereo images captured simultaneously by the stereo camera 30, which may have parallax, and a visible light image captured by the visible light camera 31 simultaneously with the stereo camera 30. The wind direction and speed information 222 is information indicating the wind direction and wind speed at the measurement point, obtained from the external device 40. The visibility information 223 is information indicating the visibility at the measurement point, obtained from the external device 40. The cloud top height information 224 is information indicating the cloud top height (highest height of clouds) based on a satellite image at the measurement point, obtained from the external device 40. However, there may be a time lag between the imaging time of the satellite image on which the cloud top height information 224 is based and the imaging time of the stereo image. The cloud base height information 225 indicates the cloud base height (lowest height of clouds) directly above it, measured by the ceilometer 32. The weather report formula 226 is a METAR statement or the like generated by the weather report formula generation unit 213.

[0029] The communication unit 23 is realized by a communication module 106 included in the computer 100. The communication unit 23 receives stereo images, visible light images, and cloud base altitudes via the network N. In addition, the communication unit 23 receives various information transmitted by the external device 40 via the network N.

[0030] The UI unit 24 is realized by an input device 104 and an output device 105 provided in the computer 100. The UI unit 24 displays various screens to the user, for example, and accepts various operations from the user.

[0031] The internal clock 25 is connected to an NTP (Network Time Protocol) server (not shown) via the network N, adjusts its own time information, and supplies the time information to each part of the weather information processing device 20, the stereo camera 30, the visible light camera 31, the ceilometer 32, etc. Instead of using an NTP server, it is also possible to calculate, adjust, and supply the time information using a satellite positioning system such as a GPS (Global Positioning System).

[0032] The stereo camera 30, the visible light camera 31, and the ceilometer 32 are arranged at a measurement point such as an airport. The stereo camera 30 is composed of two cameras arranged with a predetermined baseline length. The stereo camera 30 has a lens with a wide angle of view such as a fisheye lens. The stereo camera 30 may be a visible light camera or a near-infrared camera. When a near-infrared camera is used, it is advantageous for improving the accuracy of parallax extraction by sharpening the shading of clouds, and it is also possible to capture clouds even under low light conditions such as at night. The stereo camera 30 adds the shooting time as attribute information to the stereo images captured of the sky, and transmits them to the weather information processing device 20 via the network N. The stereo images are used to calculate the distance to the subject based on the parallax, that is, to calculate the cloud height. Note that the stereo camera 30 may be configured by distributing three or more cameras and appropriately combining two of them. In this case, multiple stereo images with different baseline lengths can be obtained, and areas on the baseline with small parallax can be supplemented.

[0033] The visible light camera 31 captures the same imaging range as the stereo camera 30, adds the shooting time to the obtained visible light image as attribute information, and transmits it to the weather information processing device 20 via the network N. Note that if a visible light camera is used for the stereo camera 30, the visible light camera 31 may be omitted and one of the stereo images may be regarded as the visible light image.

[0034] The ceilometer 32 measures the altitude (cloud base altitude) of clouds directly above, adds the measurement time as attribute information, and transmits the information to the weather information processing device 20 via the network N. When there are multiple layers of clouds directly above, the ceilometer 32 can measure the cloud base altitude of each layer. In this specification, the ceilometer 32 may be considered to include ancillary equipment such as a communication module.

[0035] The external device 40 includes, for example, an anemometer, a visibility measuring device, a meteorological satellite, etc., and outputs wind direction and wind speed information 222, visibility information 223, and cloud top height information 224. Note that the term "weather satellite" in this specification does not refer to the satellite itself flying in outer space, but refers to the entire system including the communication system and ground equipment (computers, etc.) that support the satellite, but for the sake of simplicity, the term "system" is omitted. Other devices also refer to systems including ancillary equipment other than sensors, but for the sake of simplicity, the term "system" is omitted. Note that in the following and previous explanations, the calculation of the "cloud top height based on the satellite image" is performed by ancillary equipment of the meteorological satellite, and the meteorological information processing device 20 has been described as receiving (obtaining) information calculated by ancillary equipment of the meteorological satellite, but other realization methods are also possible.

[0036] <Weather information processing by the weather information processing device 20> Next, Fig. 3 is a flowchart for explaining an example of weather information processing by the weather information processing device 20. Note that only a part of the flowchart shown below may be excerpted. Also, although it depends on how it is excerpted, from one viewpoint, the weather information processing can be regarded as cloud height measurement processing since it calculates (measures) cloud height from stereo images. Also, from another viewpoint, the weather information processing can be regarded as weather information generation processing since it generates a weather report form.

[0037] As a premise for the weather information processing, it is assumed that the information acquisition unit 214 periodically acquires wind direction and speed information 222, visibility information 223, cloud top altitude information 224, and cloud base altitude information 225 from the ceilometer 32 and the external device 40, and stores them in the memory unit 22. The weather information processing is started, for example, in response to a predetermined start operation from the user.

[0038] First, the information acquisition unit 214 controls the stereo camera 30 and the visible light camera 31 via the communication unit 23 and the network N, supplies time information generated by the internal clock 25 as an initial setting, and synchronizes the shooting time added to the image with the time information of the internal clock 25. Also, a calibration is performed between the two cameras constituting the stereo camera 30 (step S1).

[0039] Next, the information acquisition unit 214 controls the stereo camera 30 and the visible light camera 31 to capture images of the entire sky, acquires the resulting stereo images and visible light images, and stores them in the memory unit 22 as image information 221 (step S2).

[0040] Next, the cloud height calculation unit 211 refers to the visible light image included in the image information 221 and detects clouds in the visible light image on a pixel-by-pixel basis (step S3). Cloud detection can be achieved, for example, by image recognition processing using a learning model that has been machine-learned in advance. Next, the cloud height calculation unit 211 determines whether or not clouds have been detected in step S3 (step S4).

[0041] Here, if it is determined that a cloud has been detected (YES in step S4), the cloud height calculation unit 211 then divides the stereo image into a plurality of regions for each detected cloud (step S5). Next, for each divided region, the cloud height calculation unit 211 regards non-cloud parts (for example, clear sky parts, the sun, the moon, stars, artificial satellites, rays of light, disturbance light such as lighting, ghosts and flares due to reflection within the lens, raindrops, snowfall, airplanes, birds, ground structures, etc.) as noise and masks them, excluding them from the processing target in step S7 and thereafter (step S6). This reduces the amount of processing in step S7 and thereafter and erroneous detection of parallax. More specifically, steps S3 to S6 are performed for each of the sub-stereo images included in the target stereo image.

[0042] Next, the cloud height calculation unit 211 detects corresponding points in the unmasked parts of the stereo images and extracts the parallax of the detected corresponding points (step S7). Next, the cloud height calculation unit 211 determines whether or not the parallax was extracted from the stereo images in step S7 (step S8). More specifically, the detection of corresponding points is performed for each of the sub-stereo images included in the target stereo image.

[0043] If it is determined that the parallax has been extracted from the stereo images (YES in step S8), then the cloud height calculation unit 211 calculates the distance to the corresponding point, that is, the cloud height, based on the extracted parallax (step S9).

[0044] Next, the cloud height correction unit 212 corrects the cloud heights of the corresponding points calculated by the cloud height calculation unit 211 based on the cloud top altitude information 224 and the cloud base altitude information 225 (step S10).

[0045] 4 to 8 are diagrams for explaining specific examples of the process in step S10. Note that the specific examples explained in Fig. 4 to Fig. 8 may be used in combination.

[0046] For example, as shown in FIG. 4, if the time difference between the imaging time of the stereo image on which the cloud height h11 of cloud C1 calculated based on the stereo image in step S9 is based and the imaging time of the satellite image on which the cloud top altitude information 224 is based is within a predetermined threshold (e.g., 30 minutes), and the cloud height h11 calculated for cloud C1 is higher than the altitude h31 represented by the cloud top altitude information 224, then the cloud height correction unit 212 will consider the calculated cloud height h11 to be an erroneous measurement and will exclude it from being recorded in the weather report formula (will not adopt it in the weather report formula).

[0047] Also, for example, as shown in Fig. 5, when the difference between the cloud height h11 of the cloud C1 calculated based on the stereo images in step S9 and the cloud base height h21 measured at the same time by the ceilometer 32 is Δh, the cloud height correction unit 212 corrects the cloud height h11 of the cloud C1 by the difference Δh to the cloud base height h21. In the case of Fig. 5, it can be said that the cloud base height measured by the ceilometer 32 is adopted, instead of the cloud base height based on the stereo images.

[0048] Note that even if time has passed since the state shown in Fig. 5, and the cloud C1 has been blown by the wind and moved from directly above the ceilometer 32 as shown in Fig. 6 (i.e., the cloud C1 has moved out of the measurement range of the ceilometer 32), if the elapsed time is within a predetermined threshold (e.g., about 10 minutes), it can be assumed that the cloud base altitude of the cloud C1 will not change (or the amount of change will be small) from the time point of Fig. 5. Therefore, the cloud height correction unit 212 also corrects the cloud height h11 of the cloud C1 calculated based on the stereo image at the time point of Fig. 6 by the difference Δh calculated at the time point of Fig. 5 to the cloud base altitude h21'. This is an example to explain that in this embodiment, a cloud that has moved from directly above the ceilometer 32 is a cloud that has moved out of the measurement range of the ceilometer.

[0049] Also, for example, as shown in Fig. 7, when the cloud heights h11 and h12 of the clouds C1 and C2 are calculated based on the stereo image in step S9, and the cloud base height h21 of the cloud C1 located directly above the ceilometer 32 is measured at the same time, the cloud height correction unit 212 corrects the cloud height h11 of the cloud C1 to the cloud base height h21, as in the case of Fig. 5. Also, the cloud height correction unit 212 corrects the cloud height h12 of the cloud C2 at the same ratio as the correction of the cloud height h11 of the cloud C1. For example, the cloud height h12 of the cloud C2 is multiplied by the correction ratio of the cloud height h11 of the cloud C1 (for example, h21 / h11) to correct it to the cloud height h12'. However, for the cloud C2 whose cloud base height is not measured by the ceilometer 32, the cloud height h12 based on the stereo image may be used as it is.

[0050] Furthermore, for example, as shown in Fig. 8, when clouds C1, C2, and C3 are detected from the visible light image in step S3, but the cloud heights h11 and h12 are calculated for clouds C1 and C2 based on the stereo image in step S9, and the cloud height for cloud C3 is not calculated, and at the same time, the cloud base heights h21 and h23 of clouds C1 and C3 are measured by the ceilometer 32, the cloud height correction unit 212 corrects the cloud height h11 of cloud C1 to the cloud base height h21, as in the case of Fig. 5. Also, the cloud height correction unit 212 corrects the cloud height h12 of cloud C2 to the cloud height h12' at the same ratio as the correction of the cloud height h11 of cloud C1, as in the case of Fig. 7. Furthermore, the cloud height correction unit 212 adopts the cloud base height h23 of cloud C3 measured by the ceilometer 32 as the overall cloud height of cloud C3.

[0051] Returning to Fig. 3, next, the weather report formula generating unit 213 generates a weather report formula 226 such as METAR based on the cloud height corrected in step S10, the wind direction and speed information 222, the visibility information 223, etc., and stores it in the memory unit 22 (step S11). The weather report formula 226 stored in the memory unit 22 is supplied by the weather report formula output unit 215 to an airport or the like as appropriate.

[0052] 9 shows an example of a METAR sentence, which is a type of weather report format 226 generated in step S11. The part marked "ABCDEFG" in the figure is the airport name and airport code. As shown in the figure, when the information on each item of weather is converted into the weather report format 226, the amount of text can be significantly reduced without reducing the amount of information.

[0053] Returning to Fig. 3, after step S11, the process returns to step S2, and the processes from step S2 onwards are repeated at a predetermined cycle (for example, every 10 minutes).

[0054] If it is determined in step S4 that no clouds are detected (NO in step S4), steps S5 to S10 are omitted and the process proceeds to step S11, where a weather report formula 226 indicating that no clouds are present is generated.

[0055] Furthermore, in step S8, if the cloud height calculation unit 211 determines that it has not been able to extract disparity from the stereo image (NO in step S8), the reason why disparity cannot be extracted despite the presence of clouds may be due to a misalignment in the imaging directions of the two cameras that make up the stereo camera 30, etc., so the UI unit 24 notifies the user of the occurrence of an abnormality (step S12), and the weather information processing device 20 terminates the weather information processing.

[0056] According to the weather information processing described above, the cloud height for the entire sky calculated based on the stereo image is corrected based on the cloud top altitude information 224 based on the satellite image and the cloud base altitude information 225 measured by the ceilometer 32, so that the cloud height for the entire sky can be measured with high accuracy.

[0057] <Variations and Summary> The present invention is not limited to the above-described embodiment, and various modifications are possible. For example, the above-described embodiment has been described in detail to clearly explain the present invention, and is not necessarily limited to those having all of the configurations described. In addition, it is possible to replace a part of the configuration of one embodiment with a configuration of another embodiment, or to add a part of the configuration of another embodiment.

[0058] In addition, each of the above configurations, functions, processing units, processing means, etc. may be realized in hardware, for example, by designing a part or all of them in an integrated circuit. In addition, each of the above configurations, functions, etc. may be realized in software by a processor interpreting and executing a program that realizes each function. Information such as a program, table, file, etc. that realizes each function can be placed in a memory, a recording device such as a hard disk or SSD, or a recording medium such as an IC card, an SD card, or a DVD. In addition, the control lines and information lines are shown as those that are considered necessary for explanation, and not all control lines and information lines are necessarily shown in the product. In reality, it may be considered that almost all configurations are connected to each other.

[0059] Furthermore, the calculation of the "cloud top height based on a satellite image" may be performed by the weather information processing device 20, instead of a meteorological satellite, which is an example of the external device 40. In this case, the weather information processing device acquires a satellite image from the meteorological satellite instead of the cloud top height. Also, the stereo camera may include more than two cameras. The installation angle of the ceilometer may be changed to set the measurement target point to something other than directly above. In addition, the measurement area of ​​the ceilometer may be conical, with the laser emission part at the apex, instead of linear. However, the apex angle of the cone is typically narrower than the viewing angle of the stereo camera. Also, the cloud height calculation unit 211 (corresponding to steps S2 to S9 in FIG. 4 as an example) that calculates the cloud base height based on the stereo images and the other functional blocks may be implemented by separate programs or computers. For example, if the auxiliary equipment of the stereo camera includes a computer, the cloud height calculation unit 211 may be moved to the computer of the auxiliary equipment.

[0060] In the above explanation, the stereo image is used as the starting point, and the cloud base height is calculated and then "corrected" using other data (typically the cloud base height of a ceilometer). On the other hand, if the cloud base height output by the ceilometer is used as the starting point, it can also be said that the cloud base height of the ceilometer is corrected using the stereo image. Therefore, broadly speaking, the above-described process can also be said to generate the cloud base height to be included in the weather report formula based on the cloud base height obtained based on the stereo image and the ceilometer (a ceilometer using a laser).

[0061] In the explanation so far, it has been described that the cloud base height based on the stereo images obtained for clouds outside the measurement range of the ceilometer is used to correct the cloud base height based on the stereo images obtained for clouds outside the measurement range of the ceilometer. Figures 6 and 7 show examples, but the correction method may be a method other than those described in these figures. For example, difference addition is used in Figure 6 and ratio multiplication is used in Figure 7, but if there is a function or data structure that outputs the correction amount or the corrected cloud base height using the cloud base height based on the stereo images of the cloud to be corrected and the cloud base height of the ceilometer as arguments, these may be used to correct the cloud base height.

[0062] From the perspective of the weather report formula being generated, it can be said that by carrying out the above explanation, a weather report formula having the following characteristics may be generated: A weather report contains cloud base heights for at least two of the following clouds: a cloud base height obtained based on the stereo image of the first cloud; a ceilometer-based cloud base height for the second cloud; and Cloud base height for the third cloud, generated based on stereo imagery and ceilometer. For example, when the case of cloud C3 in Figure 5 or Figure 8 is applied to the second cloud, and the cases of cloud C1 and cloud C2 in Figure 6, Figure 7 or Figure 8 are applied to the third cloud, a weather report formula having such characteristics is generated.

[0063] In summary, the above describes a system having the following characteristics: A stereo camera and A computer having one or more processors and one or more memory resources; A system having The processor: Acquire, from the stereo camera, a stereo image of the sky where clouds may exist; extracting a disparity between corresponding points of the stereo images, and obtaining a first cloud height representing a distance to the cloud based on the disparity; Determine the fourth cloud height based on: The first cloud height, and At least one of a second cloud height representing a cloud base height and a third cloud height representing a cloud top height; Generate a weather report including the fourth cloud height system.

[0064] It has been explained that the system may be: The second cloud height is a value measured by a laser-based ceilometer, The third cloud height is a value obtained based on a satellite image.

[0065] It has been explained that the system may be: To extract the disparity, the processor: Detecting the cloud from the stereo image; Dividing the stereo image into cloud regions; Masking portions other than the clouds for each region of the stereo image; The disparity at the corresponding points in the unmasked portion is extracted as the disparity.

[0066] It has been explained that the system may be: The processor excludes the first cloud height without determining the fourth cloud height if the following conditions are met: The difference between the imaging time of the stereo image on which the first cloud height was based and the imaging time of the satellite image on which the third cloud height was based is within a predetermined threshold value, and The first cloud height is higher than the third cloud height.

[0067] It has been explained that the system may be: 2. The system of claim 1, As the fourth cloud height determination, the processor: When the first cloud height and the second cloud height are acquired for the same cloud, and the acquired first cloud height and the acquired second cloud height are different, the second cloud height for the same cloud is determined to be the fourth cloud height.

[0068] It has been explained that the system may be: In determining the fourth cloud height, the processor determines the fourth cloud height for a second cloud based on a ratio of the first cloud height and the second cloud height for a first cloud if the following condition is met: The first cloud height and the second cloud height are obtained for the first cloud; and The first cloud height is not measured for the second cloud to obtain the second cloud height.

[0069] It has been explained that the system may be: As the fourth cloud height determination, the processor: When the first cloud height is not acquired for the same cloud but the second cloud height is acquired, the second cloud height for the same cloud is determined to be the fourth cloud height.

[0070] It has been explained that the system may be: The stereo camera is a visible light camera or a near-infrared camera.

[0071] It has been explained that the system may be: The system further comprises the laser-based ceilometer; The determination of the fourth cloud height is based on at least the second cloud height.

[0072] It has been explained that the system may be: The processor: The generated weather report formula is output. [Explanation of symbols]

[0073] 10 cloud height measurement system, 20 weather information processing device, 21 processing unit, 211 cloud height calculation unit, 212 cloud height correction unit, 213 weather report formula generation unit, 214 information acquisition unit, 215 weather report formula output unit, 22 memory unit, 221 image information, 222 wind direction and speed information, 223 visibility information, 224 cloud top altitude information, 225 cloud base altitude Information, 226...weather reporting system, 23...communication section, 24...UI section, 25...internal clock, 30...stereo camera, 31...visible light camera, 32...ceilometer, 40...external device, 100...computer, 101...processor, 102...memory, 103...storage, 104...input device, 105...output device, 106...communication module

Claims

1. A stereo camera and A computer having one or more processors and one or more memory resources; A system having The processor: Acquire, from the stereo camera, a stereo image of the sky where clouds may exist; extracting a disparity between corresponding points of the stereo images, and obtaining a first cloud height representing a distance to the cloud based on the disparity; 2. Determine a fourth cloud height based on: The first cloud height, and At least one of a second cloud height representing a cloud base height and a third cloud height representing a cloud top height; generating a weather report formula including the fourth cloud height; system.

2. 2. The system of claim 1, The second cloud height is a value measured by a laser-based ceilometer, The third cloud height is a value obtained based on a satellite image. system.

3. 2. The system of claim 1, To extract the disparity, the processor: Detecting the cloud from the stereo image; Dividing the stereo image into cloud regions; Masking portions other than the clouds for each region of the stereo image; extracting the disparity at the corresponding points in the unmasked portion as the disparity; system.

4. 2. The system of claim 1, The processor excludes the first cloud height without determining the fourth cloud height if the following condition is met: A difference between an imaging time of the stereo image on which the first cloud height is based and an imaging time of the satellite image on which the third cloud height is based is within a predetermined threshold value, and The first cloud height is higher than the third cloud height; system.

5. 2. The system of claim 1, As the fourth cloud height determination, the processor: When the first cloud height and the second cloud height are acquired for the same cloud, and the acquired first cloud height and the acquired second cloud height are different, the second cloud height for the same cloud is determined to be the fourth cloud height; system.

6. 6. The system of claim 5, In determining the fourth cloud height, the processor determines the fourth cloud height for a second cloud based on a ratio of the first cloud height and the second cloud height for a first cloud if the following condition is met: The first cloud height and the second cloud height are obtained for the first cloud; and The first cloud height is not measured for the second cloud to obtain the second cloud height; system.

7. 2. The system of claim 1, As the fourth cloud height determination, the processor: When the first cloud height is not acquired for the same cloud but the second cloud height is acquired, the second cloud height for the same cloud is determined as the fourth cloud height. system.

8. 2. The system of claim 1, The stereo camera is a visible light camera or a near-infrared camera. system.

9. The system of claim 2 further comprises the laser-based ceilometer, The determination of the fourth cloud height is based on at least the second cloud height. system.

10. 2. The system of claim 1, The processor: Outputting the generated weather report formula. system.

11. one or more processors; one or more memory resources; A computer comprising: The processor: Obtain stereo images of the sky where clouds may exist, extracting a disparity between corresponding points of the stereo images, and obtaining a first cloud height representing a distance to the cloud based on the disparity; 2. Determine a fourth cloud height based on: The first cloud height, and At least one of a second cloud height representing a cloud base height and a third cloud height representing a cloud top height; generating a weather report formula including the fourth cloud height; computer.

12. 12. The computer of claim 11, The second cloud height is a value measured by a laser-based ceilometer, The third cloud height is a value obtained based on a satellite image. computer.

13. 12. The computer of claim 11, To extract the disparity, the processor: Detecting the cloud from the stereo image; Dividing the stereo image into cloud regions; Masking portions other than the clouds for each region of the stereo image; extracting the disparity at the corresponding points in the unmasked portion as the disparity; computer.

14. 12. The computer of claim 11, The processor excludes the first cloud height without determining the fourth cloud height if the following condition is met: A difference between an imaging time of the stereo image on which the first cloud height is based and an imaging time of the satellite image on which the third cloud height is based is within a predetermined threshold value, and The first cloud height is higher than the third cloud height; computer.

15. 12. The computer of claim 11, As the fourth cloud height determination, the processor: When the first cloud height and the second cloud height are acquired for the same cloud, and the acquired first cloud height and the acquired second cloud height are different, the second cloud height for the same cloud is determined to be the fourth cloud height; computer.

16. 16. The computer of claim 15, In determining the fourth cloud height, the processor determines the fourth cloud height for a second cloud based on a ratio of the first cloud height and the second cloud height for a first cloud if the following condition is met: The first cloud height and the second cloud height are obtained for the first cloud; and The first cloud height is not measured for the second cloud to obtain the second cloud height; computer.

17. 12. The computer of claim 11, As the fourth cloud height determination, the processor: When the first cloud height is not acquired for the same cloud but the second cloud height is acquired, the second cloud height for the same cloud is determined as the fourth cloud height. computer.

18. 13. The computer of claim 12, The determination of the fourth cloud height is based on at least the second cloud height. computer.

19. 12. The computer of claim 11, The processor: Outputting the generated weather report formula. computer.

20. A program executable by one or more processors included in a computer, By executing the program, the processor: A stereo image of the sky where clouds may exist is acquired from a stereo camera. extracting a disparity between corresponding points of the stereo images, and obtaining a first cloud height representing a distance to the cloud based on the disparity; 2. Determine a fourth cloud height based on: The first cloud height, and At least one of a second cloud height representing a cloud base height and a third cloud height representing a cloud top height; generating a weather report formula including the fourth cloud height; program.

Citation Information

Patent Citations

  • Cloud height measuring device, measuring point determination method, and cloud type determination method

    JP2022013152A

Cited By

  • Packed-bed bioreactor systems and methods of using the same

    US12460168B2