Sea surface measurement system, sea surface measurement method, storage medium, and computer program product

Abstract

The invention provides a sea surface measurement system, a sea surface measurement method, a storage medium and a computer program product, which are used for measuring sea surface anomalies such as tsunami by applying an image measurement technology. Comprises: an imaging unit that simultaneously captures an image including the same sea surface region through a plurality of cameras provided at different locations; a wave extraction unit that extracts waves from respective images simultaneously captured by the plurality of cameras; a wave association means for finding out and associating the same wave from the images simultaneously captured by the plurality of cameras for each wave extracted by the wave extraction means; a three-dimensional information calculation unit that calculates the height of the sea surface and the height of the waves by performing three-dimensional analysis on each wave in the image associated by the wave association unit; and a sea surface abnormality determination means for determining whether or not a sea surface abnormality has occurred on the basis of the height of the sea surface and the height of the wave calculated by the three-dimensional information calculation means.

Description

[0001] This application is a divisional application of the Chinese invention patent application with the application date of January 10, 2020, application number 202010027415.3, and invention name “Sea surface measurement system, sea surface measurement method and storage medium” (application number 202210062109.2, invention name “Sea surface measurement system, sea surface measurement method and storage medium”). Technical Field

[0002] The present invention relates to a sea surface measurement system, a sea surface measurement method and a sea surface measurement program for measuring sea surface anomalies such as tsunamis, huge waves, high tides and rapid tides based on images captured by multiple cameras. Background Art

[0003] Conventional methods for measuring (observing) tsunamis can be roughly classified into two types: a method for observing a tsunami that has already reached land and a method for observing a tsunami before reaching land.

[0004] (1) Observation methods for tsunamis that have reached land

[0005] In observing the height of a tsunami that has reached land, in addition to using tide gauges set up by the Meteorological Bureau in various places, tsunami observation equipment such as tsunami observation instruments and giant tsunami observation instruments are also used. The Meteorological Bureau uses tide gauges and tsunami observation instruments to constantly observe the tide level, and publishes the observed data as tidal observation data (quick report values) and tidal observation data. However, these observation systems are all set up on the coast, so although they can observe tsunamis that have reached land, they cannot observe tsunamis that are far away (tens of kilometers (km) to more than 50 kilometers (km)) before arrival, and thus cannot estimate the arrival time.

[0006] (2) How to observe a tsunami before it reaches land

[0007] In order to observe a tsunami before it reaches land, an undersea earthquake and tsunami observation network, GPS wave meters, and the recently proposed tsunami radar are used.

[0008] The Seabed Earthquake and Tsunami Observation Network is a large-scale real-time earthquake and tsunami observation network promoted and deployed by the National Research Institute of Science and Technology for Disaster Prevention as part of the Japan Trench Seabed Earthquake and Tsunami Observation Network. In the Seabed Earthquake and Tsunami Observation Network, an observation device that integrates a seismograph and a tsunami meter is connected by a submarine optical cable and installed on the Pacific Ocean floor east of Japan to obtain observation data in real time and continuously for 24 hours. As of January 2018, observation devices have been installed at 150 locations along the coast of Japan, with a total cable length of approximately 5,700 km. The Seabed Earthquake and Tsunami Observation Network is expected to directly detect trench earthquakes and the tsunamis that follow, and contribute to disaster prevention measures such as disaster reduction and evacuation operations through rapid and high-precision information transmission.

[0009] However, the installation of submarine systems and land base stations requires huge engineering and the consent of autonomous organizations and fishery stakeholders is required. In addition, the maintenance and management of equipment and facilities also require huge costs. Therefore, it is difficult to popularize it in the sea areas of the whole country.

[0010] The GPS wave meter system is an oceanographic observation device that uses GPS satellites to measure the ups and downs of buoys (GPS wave meters) floating in coastal waters to directly observe sea surface changes such as waves and tides. As of January 2017, 18 GPS wave meters were installed in the coastal waters of Japan.

[0011] GPS wave meters are installed in harbor facilities to obtain the required offshore wave information, but since they can also observe the ups and downs of the sea surface caused by tsunamis when earthquakes occur, they are expected to be effectively used in tsunami prevention measures. GPS wave meters are usually installed around 20km offshore, so they can observe tsunamis tens of kilometers away and estimate the arrival time.

[0012] However, GPS wave meters are expensive equipment. When they were first installed in 2005, one unit cost more than 300 million yen, and it is still in the hundreds of millions today. In addition, GPS wave meters can only be installed at specific locations, making it difficult to conduct observations over a wide range.

[0013] Tsunami radar is a system that uses radar measurement technology to measure the sea surface in the distance to monitor the occurrence of tsunamis. In these methods, tsunami components are extracted from the flow speed of the sea surface observed by radar to estimate the wave height and arrival time of the tsunami.

[0014] However, it is difficult to extract tsunami components from the current velocity of the sea surface in these methods, and it is particularly difficult to measure long-period tsunamis in advance.

[0015] Prior art literature

[0016] Patent Literature

[0017] Patent Document 1: Japanese Patent Application Publication No. 2013-40898

[0018] Patent Document 2: Japanese Patent Application Publication No. 2018-4529

[0019] Patent Document 3: Japanese Patent Application Publication No. 2014-160044

[0020] Patent Document 4: Japanese Patent Application Publication No. 2004-191268

[0021] Patent Document 5: Japanese Patent Application Publication No. 2017-150917

[0022] Patent Document 6: Japanese Patent Application Publication No. 2016-85206 Summary of the invention

[0023] Problem that the invention aims to solve

[0024] In addition, image measurement technology has been studied since 30 years ago. With the advancement of hardware technologies such as digital camera technology and computer manufacturing technology, and the development of software technologies such as algorithms and artificial intelligence, the practical application of image measurement technology has been developing in recent years. Currently, image measurement technology is actually used in quality management fields such as printing quality management, product quality management, and jewelry and artwork identification. In addition, it is also widely used in the fields of face recognition and object recognition.

[0025] The object of the present invention is to provide a sea surface measurement system, a sea surface measurement method and a sea surface measurement program for measuring sea surface anomalies such as tsunamis by applying image measurement technology.

[0026] Solutions for solving problems

[0027] The sea surface measurement system of the present invention comprises: a photographing unit, which simultaneously captures images including the same sea surface area by using a plurality of cameras arranged at different locations; a wave extraction unit, which extracts waves from the images captured simultaneously by the plurality of cameras; a wave correspondence unit, which finds the same waves from the images captured simultaneously by the plurality of cameras and performs correspondence for the waves extracted by the wave extraction unit; a three-dimensional information calculation unit, which calculates the height of the sea surface and the height of the waves by performing a three-dimensional analysis on the waves in the images matched by the wave correspondence unit; and a sea surface abnormality determination unit, which determines the presence or absence of sea surface abnormalities based on the wave corresponding unit. Whether a sea surface abnormality occurs is determined based on the height of the sea surface and the height of the waves calculated by the three-dimensional information calculation unit, wherein the wave extraction unit divides the image captured by the multiple cameras into several adjacent rectangular or square small areas, and for the local area including the four corners of the upper left corner, the upper right corner, the lower left corner and the lower right corner of the small area, a threshold for binarizing the image is obtained, and the small area thresholds of the four corners of the upper left corner, the upper right corner, the lower left corner and the lower right corner of the small area where the focus point exists are multiplied by weight coefficients depending on the absolute distances to the four corners to obtain a threshold for binarizing the focus point.

[0028] In addition, the sea surface measurement system of the present invention includes: a photography unit, which simultaneously captures images including the same sea surface area by using a plurality of cameras arranged at different locations; a wave extraction unit, which extracts waves from each image captured simultaneously by the plurality of cameras; a wave correspondence unit, which finds and corresponds the same waves from each image captured simultaneously by the plurality of cameras for each wave extracted by the wave extraction unit; a three-dimensional information calculation unit, which calculates the height of the sea surface and the height of the waves by performing a three-dimensional analysis on each wave in the image captured simultaneously by the wave correspondence unit; and a sea surface abnormality determination unit, which determines whether a sea surface abnormality occurs based on the height of the sea surface and the height of the waves calculated by the three-dimensional information calculation unit, wherein the wave The corresponding unit performs the following processing: by analyzing the relationship between the polar line geometry and the horizontal and vertical coordinates of the two-dimensional image, a corresponding candidate area is obtained according to the multiple images captured by the multiple cameras; a two-dimensional feature vector is generated with the centroid coordinates, the number of pixels, the perimeter, the circularity and the length in each direction of the image of each wave extracted in the corresponding candidate area of ​​each image as the feature; for all waves in the same area of ​​each image, a candidate wave is found based on the polar line constraint, and the similarity of the two-dimensional feature vector of the candidate wave is obtained, and a candidate wave list of corresponding waves of each wave is prepared in descending order of similarity; and based on the image coordinates of the centroid of the wave of interest and the wave with high similarity in the candidate wave list, a three-dimensional world coordinate (X) of the wave of interest is obtained. 关注 , Y 关注 , Z 关注) and the three-dimensional world coordinates of the candidate wave (X 候选 , Y 候选 , Z 候选 ), by determining whether the difference between the three-dimensional world coordinates of the wave of interest and the three-dimensional world coordinates of the candidate wave is less than a predetermined threshold, it is determined whether the candidate wave of the corresponding wave is the corresponding wave.

[0029] In addition, the sea surface measurement method of the present invention includes the following steps: simultaneously shooting images including the same sea surface area by multiple cameras set at different locations; extracting waves from each image simultaneously shot by the multiple cameras; for each extracted wave, finding the same wave from each image simultaneously shot by the multiple cameras and matching them; calculating the height of the sea surface and the height of the wave by performing three-dimensional analysis on each wave in the matched image; and determining whether a sea surface abnormality occurs based on the calculated height of the sea surface and the height of the wave, wherein in the step of extracting waves, the image shot by the multiple cameras is divided into several adjacent rectangular or square small areas, and a threshold for binarizing the image is obtained for a local area including the four corners of the upper left corner, the upper right corner, the lower left corner, and the lower right corner of the small area, and a threshold for binarizing the focus point is obtained based on the small area threshold of the four corners of the upper left corner, the upper right corner, the lower left corner, and the lower right corner of the small area where the focus point exists, multiplied by a weight coefficient depending on the absolute distance to the four corners.

[0030] In addition, the sea surface measurement method of the present invention includes the following steps: using multiple cameras set at different locations to simultaneously capture images including the same sea surface area; extracting waves from each image captured simultaneously by multiple cameras; for each extracted wave, finding the same wave from each image captured simultaneously by multiple cameras and matching them; calculating the height of the sea surface and the height of the wave by performing three-dimensional analysis on each wave in the matched image; and determining whether a sea surface abnormality occurs based on the calculated height of the sea surface and the height of the wave, wherein the following processing is performed in the matching step: using the relationship between the polar geometry and the horizontal and vertical coordinates of the two-dimensional image The method comprises the steps of: analyzing the image of each wave captured by the plurality of cameras, obtaining a corresponding candidate area based on the plurality of images captured by the plurality of cameras; generating a two-dimensional feature vector characterized by the coordinates of the center of gravity of the image of each wave extracted from the corresponding candidate area of ​​each image, the number of pixels, the perimeter, the circularity, and the length in each direction; finding candidate waves for all waves in the same area of ​​each image based on the epipolar constraint, obtaining the similarity of the two-dimensional feature vectors of the candidate waves, and making a candidate wave list of corresponding waves of each wave in descending order of similarity; and obtaining the three-dimensional world coordinates (X-axis coordinates) of the wave of interest based on the image coordinates of the center of gravity of the wave of interest and the wave with high similarity in the candidate wave list.关注 , Y 关注 , Z 关注 ) and the three-dimensional world coordinates of the candidate wave (X 候选 , Y 候选 , Z 候选 ), by determining whether the difference between the three-dimensional world coordinates of the wave of interest and the three-dimensional world coordinates of the candidate wave is less than a predetermined threshold, it is determined whether the candidate wave of the corresponding wave is the corresponding wave.

[0031] In addition, the storage medium of the present invention stores a sea surface measurement program, and the sea surface measurement program causes a computer to execute the above-mentioned sea surface measurement method.

[0032] According to these inventions, multiple cameras set at different locations simultaneously capture images including the same sea surface area, waves are extracted from the respective images simultaneously captured by the multiple cameras, and for each extracted wave, the same wave is found and matched from the respective images simultaneously captured by the multiple cameras, and the height of the sea surface and the height of the waves are calculated by performing three-dimensional analysis on each wave in the matched images, and whether a sea surface abnormality has occurred is determined based on the calculated height of the sea surface and the height of the waves.

[0033] In addition, the sea surface measurement program of the present invention enables the computer to function as the following units: a photographing unit that simultaneously captures images including the same sea surface area by using a plurality of cameras installed at different locations; a wave extraction unit that extracts waves from each image captured simultaneously by the plurality of cameras; a wave correspondence unit that finds and corresponds the same wave from each image captured simultaneously by the plurality of cameras for each wave extracted by the wave extraction unit; a three-dimensional information calculation unit that calculates the height of the sea surface and the height of the wave by performing a three-dimensional analysis on each wave in the image matched by the wave correspondence unit; and a sea surface anomaly presence or absence determination unit that determines whether a sea surface anomaly occurs based on the height of the sea surface and the height of the wave calculated by the three-dimensional information calculation unit. The computer that executes the sea surface measurement program of the present invention functions in the same manner as the sea surface measurement device of the present invention described above.

[0034] In addition, the storage medium of the present invention stores a sea surface measurement program, which enables the computer to function as the following units: a photography unit, which simultaneously captures images including the same sea surface area by using multiple cameras set at different locations; a wave extraction unit, which extracts waves from each image captured simultaneously by multiple cameras; a wave corresponding unit, which finds and corresponds the same waves from each image captured simultaneously by multiple cameras for each wave extracted by the wave extraction unit; a three-dimensional information calculation unit, which calculates the height of the sea surface and the height of the waves by performing three-dimensional analysis on each wave in the image corresponding to the wave corresponding unit; and a sea surface abnormality determination unit, which determines whether a sea surface abnormality has occurred based on the height of the sea surface and the height of the waves calculated by the three-dimensional information calculation unit.

[0035] Effects of the Invention

[0036] According to the present invention, it is possible to determine in real time whether a tsunami, huge waves, high tide, rapid tide or other sea surface abnormality has occurred based on images captured by multiple cameras. In the event that a tsunami has occurred as a sea surface abnormality, it is possible to calculate in real time the scale and arrival time of the tsunami, or automatically send warnings about sea surface abnormalities such as tsunamis to necessary places and necessary people, which can help with evacuation and disaster reduction in response to sea surface abnormalities. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 It is a hardware configuration diagram of the sea level measurement system in the embodiment of the present invention.

[0038] Figure 2 It is shown Figure 1 A diagram illustrating the installation status of a camera group of a sea surface measurement system.

[0039] Figure 3 It is an explanatory diagram showing the coordinate relationship of three-dimensional image measurement based on stereoscopic observation.

[0040] Figure 4 It is shown Figure 1 Block diagram of the structure of the sea surface measurement system.

[0041] Figure 5 It is shown by Figure 1 Flow chart of the process of sea surface measurement performed by the sea surface measurement system.

[0042] Figure 6 This is an image of the atmospheric model.

[0043] Figure 7 is a flow chart of the processing performed by the rain effect mitigation unit.

[0044] Figure 8It is an image map extracted by waves.

[0045] Fig. 9 : is a flowchart showing the flow of wave matching processing using two images.

[0046] Fig.10 It is the image of the eigenvector of the wave.

[0047] Fig.11 This is an image map that calculates the height of the sea surface using images from two cameras, the left and the right.

[0048] Fig.12 This is a flowchart showing the flow of calculating the height of the sea surface using the images captured by the left and right cameras.

[0049] Fig.13 This is a diagram showing an example of a cylindrical target object.

[0050] Fig.14 is a flow chart showing the process of calibration performed by the long-distance camera system calibration unit.

[0051] Fig.15 This is a flowchart showing the flow of determining whether a tsunami has occurred.

[0052] Fig.16 This is an image diagram obtained by measuring the three-dimensional image of the sea surface height at each time.

[0053] Fig.17 It is an image diagram of the mechanical control of the camera's line of sight.

[0054] Fig.18 This is a precisely controlled image map using image measurement control using two cameras.

[0055] Fig.19 This is a flowchart showing the flow of precise control by image measurement control in a case where two cameras are provided.

[0056] Description of Reference Numerals

[0057] 1: server; 2A, 2B: client computers; 3A, 3B: camera group; 4A, 4B: camera; 5A, 5B: camera fixing and adjusting part; 10: photographing unit; 11: wave extraction unit; 12: wave corresponding unit; 13: three-dimensional information calculation unit; 14: sea surface abnormality determination unit; 15: long-distance camera system calibration unit; 16: tsunami travel speed calculation unit; 17: tsunami scale estimation unit; 18: tsunami arrival time estimation unit; 19: tsunami information sending unit; 20: storage unit; 21: fog effect reduction unit; 22: rain effect reduction unit; 23: two-stage control unit. DETAILED DESCRIPTION

[0058] Next, a sea level measurement system according to an embodiment of the present invention will be described with reference to the drawings. Figure 1 is a hardware structure diagram of a sea level measurement system in an embodiment of the present invention, Figure 2 It is shown Figure 1 An illustration of the installation status of the camera group of the sea surface measurement system, Figure 3 is an explanatory diagram showing the coordinate relationship of three-dimensional image measurement based on stereoscopic observation, Figure 4 It is shown Figure 1 Block diagram of the structure of the sea surface measurement system.

[0059] like Figure 1 As shown, the sea level measurement system in this embodiment includes a server 1 as a computer, client computers 2A and 2B respectively connected to the server 1, and a first camera group 3A and a second camera group 3B respectively connected to the client computers 2A and 2B. The first camera group 3A and the second camera group 3B each include a camera 4A, 4B and a camera fixing adjustment unit 5A, 5B such as a rotating table, and the camera 4A, 4B includes a camera body and a lens, and the camera fixing adjustment unit 5A, 5B fixes the camera 4A, 4B in a manner that allows the camera 4A, 4B to be adjusted in posture.

[0060] like Figure 2 As shown, the first camera group 3A and the second camera group 3B are set at different locations C1 and C2 in positions of a base length B and a setting height H on land near the coast of the ocean. In addition, strictly speaking, Figure 3 As shown in FIG. 1 , the distance between the centers of the camera lenses of the first camera group 3A and the second camera group 3B respectively set at locations C1 and C2 is set as the baseline length B. In the following description, the direction in which the baseline length B extends is set as the X direction, the height direction is set as the Y direction, and the direction orthogonal to the X direction and the Y direction is set as the Z direction.

[0061] like Figure 4As shown, the server 1 includes: a photographing unit 10, which simultaneously captures images including the same sea surface area by using a plurality of cameras 4A, 4B arranged at different locations; a wave extraction unit 11, which extracts waves from each image captured simultaneously by the plurality of cameras 4A, 4B; a wave correspondence unit 12, which finds and corresponds the same wave from each image captured simultaneously by the plurality of cameras 4A, 4B for each wave extracted by the wave extraction unit 11; a three-dimensional information calculation unit 13, which calculates the height of the sea surface and the height of the wave by performing a three-dimensional analysis on each wave in the image corresponding to the wave correspondence unit 12; and a sea surface abnormality determination unit 14, which determines whether a sea surface abnormality such as a tsunami occurs based on the sea surface height and the wave height calculated by the three-dimensional information calculation unit 13.

[0062] In addition, the sea surface measurement system in the present embodiment further includes: a long-distance camera system calibration unit 15 for improving the accuracy of calculating the three-dimensional information of the sea surface; a tsunami travel speed calculation unit 16 for calculating the travel speed of the tsunami when a tsunami is measured as a sea surface abnormality; a tsunami scale estimation unit 17 for estimating the scale of the tsunami; a tsunami arrival time estimation unit 18 for estimating the arrival time of the tsunami; a tsunami information transmission unit 19; and a storage unit 20. In addition, although the details will be described later, the sea surface measurement system in the present embodiment further includes a fog influence reduction unit 21, a rain influence reduction unit 22, and a two-stage control unit 23. The server 1 functions as each of the above units 10 to 23 by executing the sea surface measurement program.

[0063] Figure 5 It is shown by Figure 1 Flow chart of the process of tsunami measurement performed by the sea surface measurement system.

[0064] In order to realize image measurement of a tsunami, which is an abnormality on the sea surface, first, two or more cameras 4A and 4B are installed at different locations ( S101 ).

[0065] Next, images including the same sea surface area are simultaneously captured by multiple cameras 4A and 4B (S102). In addition, infrared cameras, fog effect mitigation unit 21 and rain effect mitigation unit 22 are used for photography in adverse environments such as night, rain, fog, etc., and the details are described later (S201).

[0066] Thereafter, the wave extraction unit 11 is used to extract waves required for tsunami measurement from the images captured by the cameras 4A and 4B ( S103 ).

[0067] After the waves are extracted from each image, the waves are associated using the wave association unit 12 to associate the waves extracted from the images of the cameras 4A and 4B, and a group of identical waves is obtained ( S104 ).

[0068] For each group of waves obtained by correspondence, the three-dimensional information of the sea surface is calculated using the three-dimensional information calculation unit 13. First, the three-dimensional world coordinates (X, Y, Z) of each detected wave are calculated, and the three-dimensional information of the sea surface, that is, the height of the sea surface and the height of the wave are calculated based on the three-dimensional world coordinates (X, Y, Z) (S105).

[0069] Furthermore, in order to ensure the accuracy of three-dimensional information calculation, the camera system is calibrated using the long-distance camera system calibration unit 15 ( S202 ).

[0070] Based on the calculated sea surface height and wave height, whether a tsunami has occurred is determined using a tsunami occurrence determination unit ( S106 , S107 ).

[0071] When it is determined that a tsunami has occurred, the tsunami advance speed calculation unit 16 , the tsunami scale estimation unit 17 , and the tsunami arrival time estimation unit 18 are used to estimate the scale of the tsunami ( S108 ), and estimate the arrival time ( S109 ).

[0072] Finally, the measured tsunami information is transmitted to necessary places and necessary persons through the tsunami information transmitting unit 19 ( S110 ).

[0073] Next, each unit will be described in detail.

[0074] [Photography Unit 10]

[0075] The photographing unit 10 simultaneously captures images of the same sea surface area by using a plurality of cameras 4A and 4B disposed at different locations. As described above, the first camera group 3A including the camera 4A and the second camera group 3B including the camera 4B are configured as follows. Figure 2 As shown in the figure, different locations C1 and C2 are set on the land near the coast of the ocean to obtain images of the distant sea surface within a radius of about 5km to 20km. In order to realize the three-dimensional image measurement of the distant sea surface with a radius of 5km to 20km, in this embodiment, a three-dimensional image measurement technology based on stereoscopic observation is used. The distance between each camera 4A and 4B, that is, the baseline length B between the cameras 4A and 4B is set to more than 20 meters, and the altitude of the installation location of each camera 4A and 4B, that is, the installation height H of the cameras 4A and 4B is set to more than 20 meters.

[0076] In order to minimize the damage to the first camera group 3A and the second camera group 3B caused by a tsunami, etc., to suppress the interference during photography as much as possible, and to achieve photography at a place as far as possible, it is desirable to set the first camera group 3A and the second camera group 3B at the coast, and the closer the first camera group 3A and the second camera group 3B are to the ocean, the better. In addition, in the present embodiment, two groups of the first camera group 3A and the second camera group 3B are used, but three or more groups may also be used.

[0077] It is desirable that the cameras 4A and 4B use high-sensitivity cameras with a sensitivity of 8 bits or more to improve the measurement accuracy and to be able to capture images required for measurement even in harsh environments. In addition, it is desirable to use a large-diameter telephoto lens or a telephoto zoom lens to capture distant sea surfaces.

[0078] The client computers 2A and 2B have functions of adjusting parameters of the cameras 4A and 4B to control photography, and of acquiring images captured by the cameras 4A and 4B. The server 1 has a communication function with the client computers 2A and 2B, and has the following functions: controlling the client computers 2A and 2B, sending commands required for photographing to the client computers 2A and 2B, and acquiring photographic images from the client computers 2A and 2B.

[0079] In addition, in the sea surface measurement system of the present embodiment, in order to shoot different sea surfaces, it is necessary to adjust the sight lines of each camera 4A, 4B. Therefore, the camera fixing adjustment parts 5A, 5B can be rotated in the pan direction (horizontal direction) and the pitch direction (vertical direction). By adjusting the angles of the camera fixing adjustment parts 5A, 5B in the pan direction and the pitch direction, the shooting direction angles of the cameras 4A, 4B provided in the camera fixing adjustment parts 5A, 5B are adjusted up and down and left and right, so that multiple cameras 4A, 4B can shoot the same sea surface.

[0080] The rotation angles in the panning direction and the pitching direction are controlled according to the command of the client computers 2A and 2B. The specific values ​​of the rotation angles in the panning direction and the pitching direction vary depending on the position of the measurement sea area, and the values ​​are sent from the server 1 to the client computers 2A and 2B that control each camera group 3A and 3B. In addition, by making multiple cameras 4A and 4B shoot at the same time according to an electronic external synchronization signal or a software synchronization command from the server 1, the synchronous photography of the cameras 4A and 4B is realized. The client computers 2A and 2B control the parameters such as the exposure time of the camera body of the cameras 4A and 4B, and control the zoom and focus of the lenses. First, the client computers 2A and 2B connected to each camera 4A and 4B obtain the images shot by each camera 4A and 4B, and then send the images to the server 1 according to the command of the server 1, and the image processing is performed in the server 1.

[0081] Furthermore, when measuring sea surface abnormalities such as tsunamis, it is necessary to photograph the ocean for 24 hours. Therefore, in the sea surface measurement system of this embodiment, as cameras 4A and 4B, visible light cameras are mainly used during the day, and infrared cameras such as far infrared cameras are mainly used at night.

[0082] [Fog effect reduction unit 21]

[0083] In addition, the server 1 is provided with a fog effect reduction unit 21 for reducing the effect of fog etc. The fog effect reduction unit 21 reduces the adverse effect of fog etc. on the images captured by the cameras 4A and 4B based on the intensity component of sunlight and the longitudinal length of the image of the ocean captured by the cameras 4A and 4B, that is, the transmittance distribution of the atmosphere of the sea surface linearly approximated to the y-coordinate of the image, based on the atmospheric model.

[0084] exist Figure 6 The image diagram of the atmospheric model is shown in FIG. There is atmospheric glow in the atmospheric environment, and its intensity is represented by A. The amount of light entering the camera includes two components: the atmospheric glow component that enters directly and the reflected light component from the object. The atmospheric model can be represented by equation (1).

[0085] [Formula 1]

[0086] I=J·t+A·(1-t) (1)

[0087] Among them, I is the amount of light entering the camera, that is, the image affected by the environment such as fog, A is the intensity component of atmospheric glow, J is the reflection component of the object, and t is the atmospheric transmittance from the object to the camera. J is the image without the environmental influence of fog, etc., and is the target component required when the fog effect reduction unit 21 performs the removal process of fog, etc.

[0088] The transmittance t can be expressed by Equation (2).

[0089] [Equation 2]

[0090] t = e -βd (2)

[0091] Among them, β is the atmospheric diffusion coefficient, and d is the distance from the object to the camera.

[0092] According to Equation (1), Equation (3) can be obtained.

[0093] [Equation 3]

[0094]

[0095] That is, as long as the intensity component A of the atmospheric light and the transmittance t of the atmosphere can be estimated, the influence of fog can be removed by the fog influence reduction unit 21 to obtain an image having only the simple reflection component J.

[0096] In Reference 1 (Robby T. Tan, Visibility in bad weather from a single image, IEEE Conference on Computer Vision and Pattern Recognition, pp. 1 - 8, 2008), Mr. Tan proposed a method based on maximizing the local region contrast. The proposed method means that, on the premise that t is constant and J < A in the local region, t of this region can be estimated based on maximizing the local region contrast. Since this method only uses contrast, it sometimes becomes oversaturated.

[0097] In Reference 2 (R. Fattal. Single Image Dehazing [J]. ACM SIGGRAPH'08, pp. 1 - 9, 2008), a fog removal method based on independent component analysis was proposed. In the local region, since the pattern of the reflection component J of the object and the atmospheric transmittance are statistically independent, the reflectance can be estimated by the independent component analysis method. The setting of the independence of this method is based on the setting of the color information of the image, so it has low adaptability for photos of the ocean with poor color information changes.

[0098] In this embodiment, since the estimation of the atmospheric light intensity component A uses a conventional method, detailed description is omitted. Regarding the atmospheric transmittance t, the following calculation method is proposed: Without performing complicated calculations, the atmospheric transmittance t is approximated in a linear relationship to the longitudinal length of the captured ocean image, that is, the y coordinate of the image, as shown in Equation (4) below.

[0099] [Equation 4]

[0100] t=t0+ky (4)

[0101] Here, t0 is the initial value of the atmospheric transmittance, k is the proportionality coefficient, and y is the y coordinate of the image of the ocean captured by the camera. This can significantly shorten the calculation time and quickly calculate tsunami information.

[0102] [Rain Impact Mitigation Unit 22]

[0103] In addition, the server 1 includes a rain influence mitigation unit 22 for mitigating the influence of rain. Figure 7 is a flowchart of the processing performed by the rain effect mitigation unit 22. Figure 7 As shown, the rain effect reduction unit 22 uses a plurality of images captured continuously to obtain a maximum value image and a minimum value image of pixel color intensity, and processes the maximum value image using a guide filter using the minimum value image as a guide.

[0104] First, five images are continuously captured at fixed time intervals (S301).

[0105] Next, for the five input images I n For each pixel (n=1, 2, 3, 4, 5), the maximum value of the color intensity of the pixel is calculated by equation (5), and the minimum value of the color intensity of the pixel is calculated by equation (6). The minimum value image I is generated using the minimum value of each pixel. min , use the maximum value of each pixel to generate the maximum value image I max (S302).

[0106] [Formula 5]

[0107] I max (i,j)=max(11(i,j),..,I n (i,j)) (5)

[0108] [Formula 6]

[0109] I min (i, j) = min(I1(i, j), ..., I n (i,j)) (6)

[0110] Wherein, (i, j) is the pixel coordinate, and n=5.

[0111] Next, for the pixel of interest k(i, j), in the small area ω centered at k(i, j) k In the minimum image I min Let the guide image be the intermediate image I. f (S303).

[0112] [Formula 7]

[0113] I f (i,j)=a k I min (i, j)+b k , (i, j)∈ω k (7)

[0114] [Formula 8]

[0115]

[0116] [Formula 9]

[0117]

[0118] Among them, μ k and σ k They are small areas ω k The minimum image I max The average and variance of the color intensity of the pixels, |ω| is the small area ω k The number of pixels within For a small area ω k The average of the color intensities of the input image.

[0119] Finally, based on the input image I n and intermediate image I f , a guided filter is applied to process the maximum value image as described below, thereby removing the influence of rain and obtaining an output image in which the wave pattern is enhanced (S304).

[0120] [Formula 10]

[0121] I out =(I n -I f )A+I f (10)

[0122] Among them, A is the adjustment coefficient.

[0123] [Wave extraction unit 11]

[0124] The wave extraction unit 11 extracts waves from each image captured simultaneously by the plurality of cameras 4A and 4B. It is difficult to extract waves from photographic images of the sea area, especially from long-distance photographic images of 5 km to 20 km, which are the target of the sea surface measurement system in this embodiment, by applying a general pattern extraction method. The reasons for this include uncertainty in the wave shape, differences in the characteristics of the waves in the plurality of images captured by the plurality of cameras 4A and 4B, and changes in the photographic environment associated with all-weather photography. In addition, high-speed detection is required.

[0125] Figure 8 is the image extracted by the wave. Figure 8 As shown, first, the captured image is divided into several adjacent rectangular or square small areas. For the four corners of each small area, namely the upper left corner, the upper right corner, the lower left corner, and the lower right corner, a local area with the same area as the small area is selected around the corner as the center, and the threshold value (small area threshold value) T for binarizing the image in the local area is calculated.

[0126] As shown in formula (11), the small area threshold T based on the upper left corner, upper right corner, lower left corner and lower right corner of the small area where the focus point P (i, j) exists is multiplied by the weight coefficients k1~k4 depending on the absolute distance to the four corners to calculate the threshold T (i, j) used for the binarization of the image coordinates (i, j), i.e., the focus point P (i, j).

[0127] [Formula 11]

[0128] T(i,j)=k1T(LUi,LUj)+k2T(RUi,RUj)+k3T(LDi,LDj)+k4T(RDi,RDj) (11)

[0129] [Formula 12]

[0130]

[0131] [Formula 13]

[0132]

[0133] [Formula 14]

[0134]

[0135] [Formula 15]

[0136]

[0137] [Formula 16]

[0138]

[0139] That is, for each focus point, different weight coefficients are used for binarization according to the distance between the focus point and the four corners. Thus, the binarization results of each pixel in the focus small area can be smoothly connected with the binarization results of the pixels in the adjacent small areas above, below, left and right.

[0140] As described above, the components extracted by the binarization method (dynamic threshold calculation algorithm) based on the dynamic threshold value in which the threshold value for binarizing the attention point changes dynamically are regarded as waves in the image.

[0141] [Wave Correspondence Unit 12]

[0142] The wave matching unit 12 finds out the same wave from each image simultaneously captured by the plurality of cameras 4A and 4B for each wave extracted by the wave extraction unit 11 and matches it. Fig. 9 2 is a flowchart showing the flow of wave handling processing using two images. A similar method can be used even when there are three or more images.

[0143] First, by analyzing the relationship between the polar geometry and the horizontal and vertical coordinates of the two-dimensional image, the area where the corresponding waves may exist, that is, the corresponding candidate area, is obtained from multiple images taken by multiple cameras set at different locations (S401).

[0144] Next, a wave extraction unit is applied to the corresponding candidate regions of each image to extract m waves from the left image and n waves from the right image (S402). Fig.10 As shown, a two-dimensional feature vector is generated with parameters such as the centroid coordinates, number of pixels, perimeter, circularity, length in each direction, etc. of each wave image extracted as the characteristics of the wave. As an example, the two-dimensional feature vector V can be defined as follows.

[0145] [Formula 17]

[0146] V={G,S,L,C,T} (17)

[0147] Where G is the coordinate of the center of gravity of the wave, represented by G(x0, y0). S is a parameter representing the size of the wave, which is the number of pixels included in the extracted wave. L is the circumference of the wave, and C is the circularity of the wave. T is a parameter representing the longitudinal and lateral lengths of the wave, represented by T(h1, h2, w1, w2).

[0148] [Formula 18]

[0149] h1=y0-y1 (18)

[0150] [Formula 19]

[0151] h2=y2-y0 (19)

[0152] [Formula 20]

[0153] w1=x0-x1 (20)

[0154] [Formula 21]

[0155] w2=x2-x0 (21)

[0156] In addition, P1 (x1, y1) is the pixel coordinate of the upper left corner of the rectangle surrounding the wave, and P2 (x2, y2) is the pixel coordinate of the lower right corner.

[0157] Next, a feature vector VL of the wave of the left image and a feature vector VR of the wave of the right image are created ( S403 ).

[0158] For the m waves extracted from the left image, the characteristic parameter VL of the i-th (i=1, 2, ..., m) wave is calculated according to the following formula: i , and generate the wave vector VL of the left image.

[0159] [Formula 22]

[0160] V L T = {VL1, VL2, ..., VL m-1 , VL m} (twenty two)

[0161] For the n waves extracted from the right image, the characteristic parameter VR of the jth (j=1, 2, ..., n) wave is calculated according to the following formula: j , and generate the vector VR of the wave of the right image.

[0162] [Formula 23]

[0163] VR T = {VR1, VR2, ..., VR n-1 , VR n} (twenty three)

[0164] Next, for all waves in the same area of ​​each image, candidate waves are found based on the epipolar constraint, and the similarity of their two-dimensional feature vectors is calculated, and a candidate wave list of corresponding waves of each wave is prepared in descending order of similarity. Specifically, for the i-th wave (i=1, 2, . . . , m) extracted from the left image, k waves near the epipolar line are extracted from the right image, and the k waves are set as k candidate waves of the corresponding wave in the right image corresponding to the i-th wave in the left image (S404, S405).

[0165] For the k candidate waves, the similarity of the two-dimensional feature vectors is calculated, and a candidate wave list of corresponding waves corresponding to the i-th wave extracted from the left image is prepared in descending order of similarity (S406). The ranking of the candidate waves is determined by setting the wave with the highest similarity as the first candidate wave and the wave with the second highest similarity as the second candidate wave.

[0166] Based on the image coordinates of the center of gravity of the wave of interest and the high-ranking waves in the candidate wave list, that is, the waves with high similarity, the three-dimensional world coordinates (X 关注 , Y 关注 , Z 关注 ) and the three-dimensional world coordinates (X 候选 , Y 候选 , Z 候选 )(S407), determine the three-dimensional world coordinates (X 关注 , Y 关注 , Z 关注 ) and the three-dimensional world coordinates (X 候选 , Y 候选 , Z 候选 ) is less than a predetermined threshold (S408).

[0167] If we focus on the three-dimensional world coordinates of the wave (X 关注 , Y 关注 , Z 关注 ) and the three-dimensional world coordinates (X 候选 , Y 候选 , Z 候选 ) is less than a predetermined threshold, then the wave is set as the corresponding wave of the focus wave (S409), otherwise the next wave is verified (S410, S411).

[0168] [Three-dimensional information calculation unit 13]

[0169] The three-dimensional information calculation unit 13 calculates the height of the sea surface and the height of the waves by performing three-dimensional analysis on each wave of the image matched by the wave matching unit 12 .

[0170] If there are waves on the sea surface, the shape and size of the waves change from time to time. Therefore, when calculating the height of the sea surface based on the three-dimensional measurement results of the waves on the sea surface, it is difficult to decide which part of the wave should be used as the height of the sea surface.

[0171] On the other hand, in tsunami measurement, there is no need to find the height of a specific point on the sea surface. It is sufficient to calculate the height of the sea surface in a certain area to determine whether a tsunami has occurred.

[0172] Therefore, in the sea level measurement system of this embodiment, the height of the sea level in a certain small area is calculated based on the photographic image. Fig.11 This is an image that uses the photographic images from the left and right cameras to calculate the height of the sea surface. Fig.12 This is a flowchart showing the flow of calculating the height of the sea surface using the images captured by the left and right cameras.

[0173] like Fig.12 As shown, first, the left and right cameras 4A and 4B continuously shoot the same area to obtain multiple continuous time series images (S501). For example, the shooting speed is 1 / 30 second per image for 5 seconds, and 150 images are taken by each left and right camera to obtain a total of 300 images.

[0174] Next, waves are extracted from each image and matched ( S502 ).

[0175] Afterwards, for all the corresponding waves, the three-dimensional world coordinates (X, Y, Z) of the feature points such as the center of gravity, center, top, bottom, left end, and right end are obtained (S503). For the camera, X represents the horizontal direction, Y represents the vertical direction, and Z represents the depth direction.

[0176] Based on the calculated three-dimensional world coordinates (X, Y, Z) of each wave, the waves are classified into several small areas (S504). Fig.11 As shown, the waves are classified into three small areas.

[0177] For a specific small area, as in formula (24), for all the corresponding waves in all time series images, the average Y coordinate value of the vertical direction of the three-dimensional world coordinates of the wave center and other feature points, i.e., the direction perpendicular to the sea surface, is calculated: A , the mean value Y A Multiply by the coefficient to calculate the height H of the sea surface in this small area s (S505).

[0178] [Equation 24]

[0179] H s =k s Y A +H s0 (twenty four)

[0180] Among them, k s is the proportionality factor used for calculation, H s0 A factor used to adjust the 3D world coordinates when calculating the height of the sea surface.

[0181] Finally, for a specific small area, as in formula (25), the variance value V(Y) of the Y coordinate values ​​of all the corresponding waves in all time series images is calculated, and the variance value V(Y) is multiplied by the coefficient to calculate the height H of the wave in the small area. V (S506).

[0182] [Formula 25]

[0183] H v =k v V(Y)+H v0 (25)

[0184] Among them, k V is the proportionality factor used for calculation, H V0 A factor used to adjust the world coordinates when calculating the wave height.

[0185] [Long-range camera system calibration unit 15]

[0186] Regarding the calibration of the camera system, internal parameters such as the pixel arrangement of the image sensors of the cameras 4A and 4B and the focal length of the lenses, as well as external parameters such as the distance between the two cameras 4A and 4B, i.e., the baseline length, and the positions and postures of the cameras 4A and 4B, i.e., the rotation angle, are obtained. There are many methods for calibrating the camera system, and Zhang's method (Zhang.Z, A Flexible New Technique for Camera Calibration, IEEE Transactions on Pattern Analysis and Machine Intelligence. Vol.22, No.11, 2000, pp.1330-1334.) is particularly evaluated and actually implemented in OpenCV.

[0187] However, it is necessary to take photos at a long distance to measure tsunamis, and there are still problems in the calibration of long-distance three-dimensional image measurement systems, such as the need for large-sized calibration objects and the need to photograph the objects from different angles.

[0188] Therefore, the long-distance camera system calibration unit 15 in this embodiment is configured as a two-stage calibration using different targets, which combines an internal parameter calibration method using a plate-shaped target and an external parameter calibration method using a cylindrical target.

[0189] In the long-distance camera system calibration unit 15 of the present embodiment, two or more cylindrical targets or other elongated targets having a fixed width are used as targets for extrinsic parameter calibration. Fig.13 This is an example of a cylindrical object.

[0190] The reason for choosing a cylindrical object as the calibration object is that Fig.13 As shown in (A), the distance from the measurement point used for image measurement to the center line of the cylindrical target object does not change even when viewed from different viewpoints. For example, when viewing the target object from line of sight 1, measurement point 1 is observed on the image, and the distance from measurement point 1 to the center line of the cylinder is R. Even when observing from line of sight 2, the distance from observation point 2 to the center line of the cylinder is the same as when observing from line of sight 1, R. That is, in the case of a cylindrical target, no calibration error will occur due to changes in the observation point.

[0191] In addition, considering long-distance calibration, the length of the elongated target such as a cylinder is set to be more than 1 meter. In addition, in order to easily identify the target, the surface of the target is set to have two or more colors of white and red or other colors that are easily distinguished.

[0192] Will Fig.13 A point on the boundary line of the different colors such as the feature point candidate 1 and the feature point candidate 3 shown in (B), or a point in the center of the color such as the feature point candidate 2, or a center of gravity is set as a feature point for calibration.

[0193] In addition, Fig.13 The sizes of the different color regions of the above-mentioned target, such as L1, L2, and L3 shown in (C), are used as known parameters during calibration.

[0194] Fig.14 1 is a flowchart showing the flow of calibration performed by the long-distance camera system calibration unit 15 .

[0195] First, a picture of a calibration target is taken by a plurality of cameras 4A and 4B ( S601 ).

[0196] Next, a plurality of feature points required for calibration are extracted from each image ( S602 ).

[0197] Thereafter, the extracted feature points are matched ( S603 ).

[0198] Finally, the corresponding feature points and the known parameters such as L1, L2, and L3 are used to calculate the parameters of the camera (S604).

[0199] [Sea surface abnormality determination unit 14]

[0200] The sea surface abnormality presence / absence determination unit 14 determines whether a tsunami has occurred based on the height of the sea surface and the height of the waves calculated by the three-dimensional information calculation unit 13 . Fig.15 This is a flowchart showing the flow of determining whether a tsunami has occurred.

[0201] like Fig.15 As shown, first, in order to determine whether a tsunami has occurred, the height of the sea surface in several specific areas calculated based on the three-dimensional image measurement technology is compared with the tide level in normal times when no tsunami has occurred (S701). According to the comparison result, these specific areas are classified into three areas: candidate areas for tsunami occurrence, areas where tsunami may occur, or areas where no tsunami has occurred.

[0202] In the case where the measured differences between the sea level in multiple specific areas and the tidal level in normal times when no tsunamis occur are all greater than a specified threshold, these specific areas are classified as candidate areas where tsunamis occur. In the case where there are mixed areas where the difference between the sea level in the above-mentioned multiple specific areas and the tidal level in normal times when no tsunamis occur is greater than a specified threshold and areas where the difference is less than a specified threshold, these specific areas are classified as areas where tsunamis may occur. In the case where the differences between the sea level in the above-mentioned multiple specific areas and the tidal level in normal times when no tsunamis occur are all less than a specified threshold, these specific areas are classified as areas where no tsunamis occur.

[0203] For a candidate area where a tsunami occurs or an area where a tsunami may occur, multiple small areas are further added around the area as measurement areas (S702), and the measurement results of the sea surface heights of these measurement areas are compared with the normal tide levels of these measurement areas (S703). Based on the comparison results of the sea surface heights of these measurement areas with the normal tide levels, the fuzzy inference theory (Japanese: ファージ inference theory) is used to determine whether a tsunami has occurred (S704), and the specific area is determined as an area where a tsunami has occurred or an area where a tsunami has not occurred. In addition, sea surface anomalies other than tsunamis, such as huge waves, high tides, and rapid tides, can also be determined through the same process.

[0204] [Tsunami travel speed calculation unit 16, tsunami scale estimation unit 17, tsunami arrival time estimation unit 18]

[0205] The tsunami advancing speed calculation unit 16 uses the advancing speed of the tsunami calculated based on the conventional tsunami advancing speed calculation method to correct the advancing speed of the tsunami calculated based on the three-dimensional image measurement, thereby calculating a more accurate advancing speed of the tsunami.

[0206] Fig.16 This is an image diagram obtained by measuring the three-dimensional image of the sea surface height at each time.

[0207] When a tsunami is detected as a sea surface abnormality, the height of the sea surface in the tsunami occurrence area and its surrounding areas is measured at any time, and the area with the highest sea surface height is detected at any time. In addition, the area with the highest sea surface height closest to the cameras 4A and 4B, i.e., the tsunami front, is tracked by controlling the posture of the cameras 4A and 4B, and the distance to the tsunami front, i.e., the Z value of the three-dimensional measurement result, is obtained.

[0208] like Fig.16 As shown, by measuring the three-dimensional image of the sea surface height at time 1, time 2, etc., the four-dimensional information (X, Y, Z, t (measurement time)) of the sea surface height change is obtained to calculate the height change of the sea surface height of the tsunami front, the moving direction, and the moving speed, that is, the tsunami moving speed V calculated based on the image measurement. m .

[0209] In addition, the tsunami propagation speed V is calculated based on the conventional calculation method using formula (26): C .

[0210] [Equation 26]

[0211]

[0212] Where h is the depth of the ocean in the measurement area and g is the acceleration due to gravity.

[0213] The tsunami propagation speed V calculated using the conventional calculation method C , the tsunami travel speed V calculated by image measurement m Correction is performed to calculate a more accurate tsunami travel speed V.

[0214] [Formula 27]

[0215] V=k m V m +(1-k m )V c (27)

[0216] Among them, k m is the coefficient used for tsunami velocity correction.

[0217] The tsunami scale estimation unit 17 estimates the scale of the tsunami based on the height of the tsunami. For example, the tsunami scale estimation unit 17 evaluates the scale into six levels: -1, 0, 1, 2, 3, and 4, depending on the height of the tsunami.

[0218] The tsunami arrival time estimation unit 18 calculates the time when the tsunami reaches the coast based on the position of the tsunami front, the height of the tsunami, and the tsunami travel speed measured by three-dimensional image measurement.

[0219] [Storage unit 20]

[0220] First, the observer makes a measurement plan of which sea area to measure at which time. The server 1 determines the posture of each camera according to the measurement plan, calculates the rotation angle of each camera and sends it to each client computer 2A, 2B. The storage unit 20 stores the photographic images of each camera 4A, 4B obtained from each client computer 2A, 2B. The server 1 processes these images stored in the storage unit 20 to calculate and output whether a sea surface abnormality such as a tsunami has occurred.

[0221] [Two-stage control unit 23]

[0222] Furthermore, in the sea level measurement system of the present embodiment, the imaging unit 10 has a two-stage control unit 23 utilizing mechanics and images, which combines mechanical control based on angle feedback and image measurement control based on image photography and image processing.

[0223] In the two-stage control unit 23, first, a low-resolution angle sensor is used to perform mechanical control based on angle feedback to roughly control the sight lines of each camera 4A, 4B so that the plurality of cameras 4A, 4B face approximately the same sea area. As a target of the rough control, the sight lines of each camera 4A, 4B are roughly adjusted so that each camera 4A, 4B can capture a portion of the waves at the measurement target sea surface.

[0224] After the rough control, the cameras 4A and 4B take pictures, and the images taken by the cameras 4A and 4B are fed back to perform precise control of the sight lines of the cameras 4A and 4B through image measurement control. The goal of the precise control is to maximize the capture of waves at the target sea surface by the cameras 4A and 4B even if there are errors in mechanical control, vibrations of the cameras, etc.

[0225] Fig.17 This is an image diagram of the mechanical control of the line of sight of the camera. Mechanical control is the first level of rough control, which is to ensure that the lines of sight of the two cameras 4A and 4B are facing the same sea area. For rough control, the accuracy of line of sight control is not required, as long as it can ensure that two or more cameras 4A and 4B are facing the same sea area, that is, as long as several identical waves can be detected in the images captured by the two cameras 4A and 4B. Therefore, a low-resolution angle sensor of about 0.1° can be used in rough control.

[0226] Fig.18This is a precision controlled image diagram by the image measurement control of two cameras. The same control method can be applied when there are three or more cameras. Image measurement control is the second level of precision control. The angle and image of cameras 4A and 4B are fine-tuned through image feedback to ensure that the two cameras 4A and 4B capture the same target.

[0227] In order to shoot the same target with the left and right cameras 4A and 4B, it is necessary to send the respective target sight angles to the left and right cameras 4A and 4B. One of the plurality of cameras is set as the main camera and the other is set as the sub-camera. In this embodiment, the left camera 4A is set as the main camera and the right camera 4B is set as the sub-camera. The sight angle of the right camera 4B, i.e., the sub-camera, is automatically calculated based on the sight angle of the left camera 4A, i.e., the main camera.

[0228] The key point of image measurement control is to feed back the images captured by the left and right cameras 4A and 4B, compare them with the target photographic image, and further adjust the pan and tilt angles of the camera fixing adjustment parts 5A and 5B based on the difference between them. In this way, errors caused by vibration of the cameras 4A and 4B, such as Fig.18 Errors such as interference shown.

[0229] Fig.19 1 is a flowchart showing the flow of precision control by image measurement control in the case of two cameras. First, the sight angle for adjusting the posture of the cameras 4A and 4B, that is, the rotation angle in the pan and pitch directions, is calculated in the server 1 according to the photographic target area. Then, the sight angle of the right camera 4B, that is, the rotation angle in the pan and pitch directions, is calculated using the theory of triangulation according to the spatial relationship between the cameras 4A and 4B and the photographic target area, and is sent to the client computers 2A and 2B that control the respective cameras 4A and 4B (S801).

[0230] The client computer 2A connected to the left camera 4A sets the pan and tilt rotation angles of the left camera 4A received from the server 1 as the target value of the sight line angle, compares the actual sight line angle of the left camera 4A obtained from the angle sensor with the target value of the sight line angle, and adjusts the pan and tilt angles of the camera fixing adjustment unit 5A so that the difference between them is minimized to perform mechanical control (S802). The right camera 4B is also mechanically controlled in the same manner (S803).

[0231] After that, the left camera 4A and the right camera 4B respectively take pictures, and the waves are respectively extracted from the left and right photographic images (S804, S805).

[0232] Next, based on the waves extracted from the left and right images, the area existing in both the left and right images, i.e. the common area, is found, a wave located in the center of the common area is extracted, and this wave is set as the target wave for adjusting the camera posture (S806).

[0233] It is expected that the target wave is located at the center of the image in both the left and right images. The target wave is the target of the posture control of the left and right cameras 4A and 4B, that is, the shooting target. For the left camera 4A, it is determined whether the target wave enters the fixed range at the center periphery of the image (S807). If it enters the range, the posture adjustment of the left camera 4A is terminated. If it does not enter the range, the above-mentioned mechanical control is performed to adjust the pan and tilt angles (S809).

[0234] Next, after mechanical control, a photo is taken, the target wave is extracted from the photographic image, and it is investigated whether image adjustment is required (S811, S813). If the target wave enters a fixed area in the center of the image, image adjustment is not required, otherwise the image is translated within a fixed range in the horizontal and vertical directions to perform image adjustment (S815).

[0235] Through the above-mentioned image adjustment, it is checked whether the target wave is adjusted to the center of the left image. If it is located in the central area of ​​the image, the adjustment is terminated. Otherwise, it returns to the mechanical control of S809 and repeats the control (S809-S815). That is, no matter how the image is translated, the situation where the target wave is not near the center of the image means that the range that can be controlled by the image measurement control is narrow and the control target cannot be achieved. This situation, that is, the part that cannot be covered by the image measurement control, is realized by mechanical control. For the right camera 4B, the posture of the camera 4B is also adjusted in the same way.

[0236] Finally, check whether the adjustment of the left and right cameras 4A and 4B has been completed. If so, end the control of the cameras; otherwise, wait until it is completed (S819, S820).

[0237] That is, in the above-mentioned image measurement control, there is angle feedback performed by an angle sensor for obtaining the pan and tilt angles of the camera fixing adjustment part 5A, 5B that fixes the cameras 4A, 4B in a manner that allows the posture to be adjusted, and image feedback for determining the posture of the cameras 4A, 4B by extracting waves from the photos taken by the cameras 4A, 4B. The angle feedback is set as internal feedback, and the image feedback is set as external feedback. The image feedback has a dual feedback structure around the angle feedback. Mechanical control is first performed, and the part that is insufficient in the mechanical control, that is, the error of the mechanical control is supplemented by the image measurement control, so that the part that cannot be covered by the image measurement control is realized by the mechanical control.

[0238] Industrial Applicability

[0239] The sea level measurement system, sea level measurement method and sea level measurement program of the present invention can also be applied to the measurement of whether a tsunami occurs, the measurement of the height of the tsunami when a tsunami occurs, the measurement of the traveling speed of the tsunami, the estimation of the arrival time of the tsunami, etc. In addition, it can be applied not only to tsunamis but also to the measurement of the height, position and arrival time of waves generated by typhoons as sea surface anomalies. In addition, it can also be applied to the measurement of sea surface anomalies such as huge waves, high tides, and rapid tides, and the monitoring of the coast.

Claims

1. A sea surface measurement system, comprising: A photography unit, which simultaneously captures images including the same sea surface area through a plurality of cameras arranged at different locations; a wave extraction unit for extracting waves from each of the images simultaneously captured by the plurality of cameras; a wave matching unit, which finds the same wave from each image simultaneously captured by the plurality of cameras and matches it with each wave extracted by the wave extraction unit; a three-dimensional information calculation unit, which calculates the height of the sea surface and the height of the waves by performing three-dimensional analysis on each wave in the image matched by the wave matching unit; as well as a sea surface abnormality determination unit for determining whether a sea surface abnormality occurs based on the sea surface height and the wave height calculated by the three-dimensional information calculation unit; Wherein, the wave corresponding unit performs the following processing: By analyzing the relationship between the epipolar geometry and the horizontal and vertical coordinates of the two-dimensional image, the corresponding candidate area is obtained according to the multiple images captured by the multiple cameras; generating a two-dimensional feature vector characterized by the coordinates of the center of gravity, the number of pixels, the perimeter, the circularity, and the length in each direction of the image of each wave extracted in the corresponding candidate area of ​​each image; For all waves in the same area of ​​each image, candidate waves are found based on the epipolar constraint, and the similarity of the two-dimensional feature vectors of the candidate waves is calculated, and a candidate wave list of the corresponding waves of each wave is prepared in descending order of similarity; and Based on the image coordinates of the center of gravity of the wave of interest and waves with high similarity in the candidate wave list, the three-dimensional world coordinates of the wave of interest and the three-dimensional world coordinates of the candidate waves are calculated, and by determining whether the difference between the three-dimensional world coordinates of the wave of interest and the three-dimensional world coordinates of the candidate waves is less than a predetermined threshold, it is determined whether the candidate wave of the corresponding wave is the corresponding wave.

2. The sea level measurement system according to claim 1, characterized in that: The three-dimensional information calculation unit calculates the three-dimensional coordinates of the feature points of each wave in the image matched by the wave matching unit, and calculates the height of the sea surface based on the average value of the calculated three-dimensional coordinates of the feature points in the height direction.

3. The sea level measurement system according to claim 2, characterized in that: The three-dimensional information calculation unit calculates the height of the wave based on the variance of the three-dimensional coordinates of the feature point in the height direction.

4. The sea level measurement system according to any one of claims 1 to 3, characterized in that: The sea surface abnormality presence / absence determination unit determines whether a sea surface abnormality has occurred by comparing a normal tide level when no sea surface abnormality has occurred with a sea surface height obtained by taking into account a wave height calculated by the three-dimensional information calculation unit.

5. The sea level measurement system according to any one of claims 1 to 3, characterized in that: The plurality of cameras are two or more camera groups including a camera body, a lens, and a camera fixing and adjusting unit, wherein the camera fixing and adjusting unit fixes the plurality of cameras in a manner that allows the cameras to be adjusted in posture. The sea surface measurement system comprises: a client computer having a function of controlling photography by adjusting parameters of the camera body and the lens, and a function of acquiring images captured by the plurality of cameras; and The server has a communication function for communicating with the client computer and has the following functions: controlling the client computer, sending commands required for photo shooting to the client computer, and acquiring the photographic image of the camera from the client computer.

6. The sea level measurement system according to any one of claims 1 to 3, characterized in that: The camera is a far infrared camera, comprising a fog effect reduction unit and a rain effect reduction unit. The fog effect mitigation unit mitigates the effect of fog on the image captured by the camera based on an atmospheric model, according to the intensity component of sunlight and the transmittance distribution of the atmosphere on the sea surface that is linearly approximated to the longitudinal length of the image of the ocean captured by the camera. The rain effect alleviating unit uses a plurality of images continuously captured by the camera to obtain a maximum value image and a minimum value image of pixel color intensity, and processes the maximum value image using a guided filter using the minimum value image as a guide.

7. A method for measuring sea surface, comprising the following steps: A plurality of cameras arranged at different locations simultaneously capture images including the same sea surface area; extracting waves from each of the images simultaneously captured by the plurality of cameras; For each extracted wave, find the same wave from each image simultaneously captured by the multiple cameras and make a correspondence; Calculating the height of the sea surface and the height of the waves by performing three-dimensional analysis on each wave in the corresponding images; and Based on the calculated sea surface height and wave height, it is determined whether there is a sea surface abnormality. Among them, the following processing is performed in the corresponding steps of performing the wave: By analyzing the relationship between the epipolar geometry and the horizontal and vertical coordinates of the two-dimensional image, the corresponding candidate area is obtained according to the multiple images captured by the multiple cameras; generating a two-dimensional feature vector characterized by the coordinates of the center of gravity, the number of pixels, the perimeter, the circularity, and the length in each direction of the image of each wave extracted in the corresponding candidate area of ​​each image; For all waves in the same area of ​​each image, candidate waves are found based on the epipolar constraint, and the similarity of the two-dimensional feature vectors of the candidate waves is calculated, and a candidate wave list of the corresponding waves of each wave is prepared in descending order of similarity; and Based on the image coordinates of the center of gravity of the wave of interest and waves with high similarity in the candidate wave list, the three-dimensional world coordinates of the wave of interest and the three-dimensional world coordinates of the candidate waves are calculated, and by determining whether the difference between the three-dimensional world coordinates of the wave of interest and the three-dimensional world coordinates of the candidate waves is less than a predetermined threshold, it is determined whether the candidate wave of the corresponding wave is the corresponding wave.

8. A storage medium storing a sea surface measurement program, wherein the sea surface measurement program enables a computer to execute the sea surface measurement method according to claim 7.

9. A computer program product, comprising a sea surface measurement program, wherein the sea surface measurement program enables a computer to execute the sea surface measurement method according to claim 7.

Images