Earthquake visual early warning method combining monocular depth estimation and optical flow method

Abstract

The invention provides an earthquake visual early warning method combining monocular depth estimation and an optical flow method. According to the method, through fusion of monocular depth estimation and a robust optical flow algorithm suitable for a traffic scene, inversion of ground acceleration at a street lamp type equipment base is realized, a distributed earthquake monitoring network can be constructed based on inversion data, the industrial pain points of high deployment cost and insufficient coverage density of a traditional seismograph are solved, and the method is suitable for popularization and application. And a solution with high cost performance is provided for urban earthquake early warning.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of earthquake early warning, in particular to an earthquake visual early warning method combining monocular depth estimation and optical flow method. BACKGROUND

[0002] Earthquake is the embodiment of the sudden release of huge energy in the earth's interior, usually caused by the movement or rupture of the crustal plate. This intense energy release propagates in the form of elastic waves to all directions, and when it acts on the ground surface, it will cause great damage. Strong earthquakes can directly destroy buildings, bridges and other infrastructure, and trigger landslides, liquefaction, tsunamis, fires and other serious secondary disasters, posing a great threat to human life and property. Earthquakes mainly propagate through seismic waves, and seismic waves contain P waves (fast speed, mainly causing up and down vibration), S waves (slow speed, mainly causing horizontal shaking, strong destructive power) and surface waves (slow speed but large amplitude, the main factor causing serious structural swing damage).

[0003] With the breakthrough progress of seismic wave propagation theory, P wave first arrival detection algorithm (P wave with less destructive power propagates faster but has less destructive power, while S wave and surface wave with more destructive power propagate slower but have more destructive power, by detecting P wave and quickly estimating earthquake parameters, early warning can be given before S wave arrives) and real-time data transmission technology, the earthquake early warning system has the ability to respond within seconds, and its technical framework is becoming mature. However, the system efficiency is limited by the coverage density and sustainable operation and maintenance capability of the sensor network, and the current mainstream deployment mode faces significant bottlenecks: traditional wideband seismographs have high sensitivity characteristics, but their single purchase cost is as high as tens of thousands of yuan, and they require professional anti-seismic machine rooms and continuous power supply, resulting in high construction and operation and maintenance costs. Although micro-electromechanical system (MEMS) sensors have gradually been applied to earthquake monitoring networks due to their cost advantage (single price reduced to thousands of yuan), the actual layout distance still maintains at 15 kilometers range due to the limitation of signal noise reduction algorithm accuracy and station power supply / communication supporting requirements. In addition, the low-frequency performance of MEMS sensors is limited by electronic noise and mechanical thermal drift, which requires high-precision signal amplification and filtering technology for processing, and there are usually problems in low-frequency signal monitoring (such as long-period ground motion, vibration of high-rise buildings, bridges and dams).

[0004] Traditional seismic observation technology is limited by high cost, low density and low environmental adaptability, and the large-scale application of MEMS sensors still faces the bottleneck of insufficient power consumption and intelligence. In addition, due to the influence of urbanization construction and other production activities, the observation environment of a considerable part of the seismic observation station is affected to varying degrees, and the observation quality has decreased significantly. With the development of new generation of information technology such as artificial intelligence, it is possible to create a "low power consumption, miniaturization and intelligence" seismic observation technology and equipment based on computer vision. At present, the camera coverage density of key cities exceeds 150 units / km² (the coverage density of some areas such as Shanghai and Shenzhen even exceeds 500 units / km²), and its near-surface dense layout characteristics and complete power supply / communication infrastructure provide a new solution for low-cost, high-resolution seismic monitoring. Relying on the city camera network, a "deprofessionalized" monitoring mode is expected to greatly shorten the distance between seismic monitoring stations. SUMMARY

[0005] The purpose of the present application is to solve the problems in the prior art, and a seismic visual early warning method combining monocular depth estimation and optical flow method is proposed.

[0006] The present application is realized by the following technical solutions, and the present application proposes a seismic visual early warning method combining monocular depth estimation and optical flow method, which comprises the following steps: Step one, constructing a single degree of freedom motion model of a streetlight type visual perception device: taking an outdoor streetlight type monitoring device as an object, a rigid connection simplified model between the device and the streetlight is established to eliminate the relative displacement interference; combined with the calibrated camera internal parameter, the spatial reference of visual perception is established to provide physical constraints for seismic solution; Step two, device seismic identification by fusing monocular depth estimation and optical flow method: based on the monitoring video stream, the scene depth distribution is calculated by using monocular depth estimation, and the acceleration and displacement vector of the camera in the three-dimensional space is calculated by combining the adaptive robust optical flow algorithm designed for the traffic scene, so as to realize accurate perception of the carrier vibration; Step three, obtaining the real ground seismic acceleration based on the visual seismic perception: the non-seismic interference signal is filtered out through the seismic identification system, and the device seismic frequency domain characteristics are extracted; based on the kinematic relationship of the single degree of freedom motion model, the time history curve of the ground input acceleration is inverted to form a reliable representation of the seismic parameter of single point inversion; Step four, constructing a multi-device cooperative seismic early warning system: based on the single point inversion ground seismic acceleration data, a distributed visual perception device network is deployed, and the P wave first arrival time and acceleration peak value of each site are synchronously collected; the seismic phase automatic identification is realized by combining the STA / LTA first arrival picking algorithm, the focal position is inverted by using the inter-station P wave arrival time difference, and the magnitude is calculated by fusing the acceleration amplitude characteristics of multiple nodes; finally, according to the difference between the S wave and P wave propagation velocity, the regional seismic intensity prediction and warning time window are generated to drive the second-level automatic warning response.

[0007] Further, in step one, first of all, it is clear that the monitoring camera body is fixedly installed on the top of the street lamp pole; secondly, the connection between the camera and the lamp pole is simplified as a rigid connection, ignoring its actual possible flexibility and vibration; then the camera intrinsic parameter is accurately calibrated with the help of the calibration board, ensuring the geometric accuracy of the image and physical space mapping; when performing ground vibration inversion analysis, the whole system including the camera and the lamp pole support structure is equivalent to a single degree of freedom structure.

[0008] Further, in step two, the adaptive robust optical flow algorithm is used to stably extract the pixel-level background key point displacement representing device vibration from the traffic scene video stream, which includes two stages: robust feature point extraction and displacement data robust screening.

[0009] Further, the robust feature point extraction specifically includes: first, the edge detection technology is used to identify the object contour in the picture, then the high-precision line segment detection method is used to extract the significant line segment in the contour, and the line segment end points are accurately selected as the key feature points.

[0010] Further, the displacement data robust screening specifically includes: for a large number of feature points extracted, the pixel displacement of the feature points in the continuous video frames is calculated, the K-means clustering algorithm is used to statistically analyze the physical displacement calculated by all feature points, effectively eliminating abnormal outliers, and the average value of the feature point displacement in the largest cluster is selected as the final effective structural displacement corresponding to the video frame.

[0011] Further, in step two, the spatial distance of the key static reference point is calculated in real time by using the pre-trained monocular depth network, the spatial distance is spatio-temporally fused with the optical flow displacement, and the acceleration and displacement vector pose change of the camera in the three-dimensional space is calculated based on the calibrated camera intrinsic parameter and imaging geometry, that is, the direct kinematic response of the structure to the ground vibration; this pose information is used as the core input to drive the single degree of freedom motion model to complete the ground acceleration inversion, and then the source parameters are inverted.

[0012] Further, in step three, the ground acceleration is inverted by using the simplified model of the street lamp type visual perception device and its vibration equation, which specifically includes: Model parameter identification: the frequency domain characteristics of the structural vibration response time domain signal generated by the device under the excitation of the environmental dynamic load in the normal operating state are extracted by fast Fourier transform FFT, and then the key parameters of the single degree of freedom motion model are identified, including the natural frequency and the damping ratio; Inversion calculation: the identified model parameters and the monitored structural vibration response are combined, and the ground acceleration time history at the installation point of the device base is calculated and output according to the single degree of freedom vibration equation ; (1).

[0013] Further, in step four, based on the P-wave first arrival time difference obtained by the multiple stations with spatial distribution, the spatial position of the earthquake source is accurately calculated and determined by using the inter-station P-wave arrival time difference inversion method; further, the maximum acceleration amplitude characteristics after the P-wave arrival measured by each station are combined, the multi-node amplitude data are fused for comprehensive analysis, and the magnitude representing the intensity of earthquake energy release is solved; on the basis of determining the source position and the magnitude, according to the inherent difference between the propagation velocities of S waves and P waves in the stratum, the peak intensity prediction distribution diagram of ground motion at different positions in the target region is calculated, and the S wave early warning time window corresponding to each position is generated; finally, the early warning system drives the automatic response mechanism according to the generated ground motion intensity prediction and early warning time window information, and publishes the early warning signal to the affected area within a time scale of seconds.

[0014] The application further provides an electronic device comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the method for earthquake visual early warning combining monocular depth estimation and optical flow method when executing the computer program.

[0015] The application further provides a computer readable storage medium for storing computer instructions, wherein the computer instructions implement the steps of the method for earthquake visual early warning combining monocular depth estimation and optical flow method when executed by a processor.

[0016] The application has the following beneficial effects: The application provides a method for earthquake visual early warning combining monocular depth estimation and optical flow method, and the core innovation is that: by fusing monocular depth estimation and robust optical flow algorithm suitable for traffic scenes, the inversion of ground acceleration at the base of the street lamp type device is realized, and a distributed earthquake monitoring network can be constructed based on the inversion data, thereby solving the industry pain points of high deployment cost and insufficient coverage density of traditional seismographs, and providing a high cost-effective solution for city-level earthquake early warning.

[0017] The application uses the widely deployed street lamp type visual perception device in the city as the earthquake monitoring node, without the need of additionally installing special seismic sensors, thereby solving the pain points of high deployment cost, low laying density and difficulty in realizing high coverage in the whole city of traditional seismographs (strong motion seismographs). Only software upgrading is needed to transform a large number of existing street lamps into monitoring points, thereby significantly improving the spatial resolution and coverage range of the earthquake monitoring network. BRIEF DESCRIPTION OF DRAWINGS

[0018] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed in the embodiments or prior art description. Obviously, the drawings in the following description only constitute a part of the embodiments of the present application, and for those skilled in the art, other drawings can also be obtained without creative labor on the basis of the provided drawings.

[0019] Figure 1 Fig. 1 is a schematic diagram of a street lamp pole model and a simplified single degree of freedom model according to the present application.

[0020] Figure 2 Fig. 2 is a schematic diagram of a high-robust optical flow method combined with line segment detection according to the present application.

[0021] Figure 3 Fig. 3 is a schematic diagram of a simulation device according to an embodiment of the present application.

[0022] Figure 4 Fig. 4 is a schematic diagram of calibration results according to an embodiment of the present application.

[0023] Figure 5 Fig. 5 is a schematic diagram of depth estimation results according to an embodiment of the present application.

[0024] Figure 6 Fig. 6 is a comparison diagram of pole top acceleration time history results according to an embodiment of the present application.

[0025] Figure 7 Fig. 7 is a comparison diagram of ground acceleration time history results according to an embodiment of the present application.

[0026] In the drawings, the reference signs are as follows: 1-camera, 2-street lamp pole scale model, 3-seismic simulation vibration table, 4-simulated traffic scene, 5-acceleration sensor. DETAILED DESCRIPTION

[0027] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments only constitute a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present application.

[0028] The present application proposes a seismic visual warning method combining monocular depth estimation and optical flow method. The method realizes outdoor vibration recognition by combining monocular depth estimation and optical flow method, and obtains seismic motion inversion results by a physical model of a street lamp type outdoor visual perception device, thereby establishing a visual ground acceleration monitoring network. By using the monitoring network and combining with existing P-wave first arrival detection algorithm, the epicenter can be quickly located and the magnitude can be estimated, so as to issue a seismic warning.

[0029] Specifically, the method comprises the following steps:Figures 1-7 The present application proposes a seismic visual warning method combining monocular depth estimation and optical flow method, and the method comprises: Step one, constructing a single degree of freedom motion model of the street lamp type visual perception device: taking the outdoor street lamp type monitoring device as the object, a rigid connection simplified model between the device and the street lamp is established to eliminate the relative displacement interference; combined with the calibrated camera internal parameter, the spatial reference of visual perception is established to provide physical constraints for the vibration solution; The present application proposes a simplified physical modeling method for ground vibration inversion application for the widely deployed street lamp type monitoring device in urban space. In the present embodiment, the research object is selected as a typical street lamp type outdoor visual perception device. First, it is clear that the monitoring camera main body is fixedly installed on the top of the street lamp pole; secondly, the connection between the camera and the lamp pole is simplified as a rigid connection, ignoring the actual possible flexibility and vibration; then the camera internal parameter (focal length, principal point, distortion coefficient, etc.) is accurately calibrated by means of the calibration plate, to ensure the geometric accuracy of the image and physical space mapping; when performing ground vibration inversion analysis, the whole system including the camera and the lamp pole support structure is equivalent to a single degree of freedom (Single Degree of Freedom, SDOF) structure, as shown in Figure 1 The simplified model aims to efficiently depict the dominant dynamic behavior of the system under the ground vibration excitation, as the physical basis for subsequent vibration source parameter inversion using the visual sensing data of the device.

[0030] Step two, fusion of monocular depth estimation and optical flow method for device vibration identification: based on the monitoring video stream, the scene depth distribution is calculated by using monocular depth estimation, and the acceleration and displacement vector of the camera in three-dimensional space is calculated by combining the adaptive robust optical flow algorithm designed for traffic scene, to realize the accurate perception of the carrier vibration; The seismic warning system constructed by the present application is based on the simplified physical model (single degree of freedom model) of the aforementioned street lamp type visual perception device, and innovatively fuses monocular depth estimation and optical flow technology. A high-robustness optical flow algorithm combined with accurate line segment detection for traffic scene is developed for the complexity of traffic monitoring scene (such as motion object interference), as shown in Figure 2 The high-robustness optical flow algorithm combined with accurate line segment detection for traffic scene is used to stably extract the pixel-level background key point displacement representing the device vibration from the traffic scene video stream, which specifically includes two stages: robust feature point extraction and displacement data robust screening.

[0031] The robust feature point extraction specifically refers to: first, the object contour in the picture is identified by using edge detection technology, then the significant straight line segment in the contour is extracted by using high-precision line segment detection method, and the line segment end points are accurately selected as key feature points. Compared with the traditional corner point or pixel point feature, this method has stronger anti-motion interference and light change ability in traffic scene.

[0032] The displacement data robust screening is specifically: for the extracted large number of feature points, the pixel displacement of the feature points in the continuous video frames is calculated, the physical displacement of all feature points calculated by the K-means clustering algorithm is statistically analyzed, the abnormal outliers (usually generated by the interference of in-scene moving objects) are effectively eliminated, and the average value of the displacement of the feature points in the largest cluster is selected as the final effective structural displacement corresponding to the video frame. This process significantly improves the accuracy and stability of displacement measurement in complex dynamic traffic environment.

[0033] In step two, the spatial distance of the key static reference points is calculated in real time by using the pre-trained monocular depth network, which is spatio-temporally fused with the optical flow displacement, and the acceleration and displacement vector pose change of the camera in three-dimensional space is calculated based on the calibrated camera intrinsic parameters and imaging geometry, that is, the direct kinematic response of the structure to the ground vibration; this pose information is used as the core input to drive the single degree of freedom motion model to complete the inversion of the ground acceleration, and then the source parameters are inverted.

[0034] Step three, inversion of real ground vibration acceleration based on visual vibration perception: filtering out non-seismic interference signals (such as vehicle passing) through the vibration recognition system, extracting the frequency domain features of the device vibration; based on the kinematic relationship of the single degree of freedom motion model, the time history curve of the ground input acceleration is inverted, forming a reliable representation of the single-point inverted seismic ground motion parameters; In step three, the simplified model (single degree of freedom motion model) of the street lamp type visual perception device and its vibration equation (formula (1)) are used to invert the ground acceleration, which specifically includes: Model parameter identification: using the structural vibration response time domain signal generated by the device under normal operating conditions excited by environmental dynamic load (such as wind, vehicle passing, etc. Natural or operating state excitation), the frequency domain characteristics are extracted by fast Fourier transform FFT, and then the key parameters of the single degree of freedom motion model are identified, including natural frequency and damping ratio; Inversion calculation: combining the identified model parameters (natural frequency, damping ratio) with the monitored structural vibration response (as known input), according to the single degree of freedom vibration equation (formula (1)), the ground acceleration time history ; (1).

[0035] Step four, constructing a multi-device cooperative earthquake early warning system: deploying a distributed visual perception device network based on single-point inversion of ground vibration acceleration data, synchronously collecting the P-wave first arrival time and acceleration peak value of each site; combining the STA / LTA first arrival picking algorithm to realize automatic identification of seismic phase, using the inter-station P-wave arrival time difference inversion method to invert the source location, and combining the acceleration amplitude characteristics of multiple nodes to solve the magnitude; finally, according to the difference between the propagation speeds of S and P waves, generating regional seismic intensity prediction and warning time window, driving the second-level automatic warning response.

[0036] The present application constructs an earthquake early warning system in cooperation with multiple devices based on single-device ground acceleration inversion method, which includes a distributed visual perception device network deployed at each monitoring site, which is used to synchronously collect and real-time transmit the ground vibration acceleration data recorded by each site, and extract the key waveform features including the P-wave first arrival time and acceleration peak value. The system processes the collected data in real time through the short-time average / long-time average (STA / LTA) first arrival picking algorithm, realizes the automatic identification and accurate picking of seismic phase, especially P-wave arrival.

[0037] Specifically, in step four, based on the P-wave first arrival time difference obtained by multiple spatially distributed sites, the spatial position of the earthquake source is accurately calculated and determined by using the inter-station P-wave arrival time difference inversion method; further combining the maximum acceleration amplitude characteristics measured by each site after P-wave arrival, the multi-node amplitude data is integrated for comprehensive analysis, and the magnitude representing the intensity of seismic energy release is solved; based on the determination of the source location and the magnitude, according to the inherent difference between the propagation speeds of S and P waves in the stratum, the peak intensity prediction distribution map of seismic motion at different positions in the target area is calculated, and the S-wave warning time window corresponding to each position is generated; finally, according to the generated seismic intensity prediction and warning time window information, the warning system drives the automatic response mechanism to issue warning signals to the affected area within the second-level time scale, thereby significantly improving the timeliness of earthquake warning.

[0038] Embodiment The combination of monocular depth estimation and optical flow method for earthquake visual early warning method proposed by the present application will be described in detail below in combination with the drawings. This embodiment illustrates the implementation process of the present application for inverting ground acceleration with the help of streetlight-type visual perception devices through a seismic vibration table, and further illustrates the implementation process of the earthquake early warning method in combination with the flow chart. Among them, the simulation device is as shown in Figure 3 (1 is a camera, 2 is a streetlight pole scale model, 3 is a seismic simulation vibration table, 4 is a simulated traffic scene, and 5 is an acceleration sensor).

[0039] The present application proposes a combination of monocular depth estimation and optical flow method for earthquake visual early warning method, which comprises: Step one, constructing a single degree of freedom motion model of the street lamp type visual perception device: taking an outdoor street lamp type monitoring device as an object, a rigid connection simplified model between the device and the street lamp is established to eliminate the relative displacement interference; in combination with the calibrated camera internal parameter, the spatial reference of visual perception is established to provide physical constraints for the vibration solution; Step two, fusing the device vibration identification of monocular depth estimation and optical flow method: based on the monitoring video stream, the monocular depth estimation is applied to solve the scene depth distribution, and in combination with the adaptive robust optical flow algorithm designed for the traffic scene, the acceleration and displacement vector of the camera in the three-dimensional space is solved to realize the accurate perception of the carrier vibration; Step three, obtaining the real ground vibration acceleration based on the vibration perception inversion: through the vibration identification system, the non-seismic interference signal is filtered out, and the device vibration frequency domain characteristics are extracted; based on the kinematic relationship of the single degree of freedom motion model, the time history curve of the ground input acceleration is inverted to form a reliable representation of the single point inversion of the seismic ground motion parameters; Step four, constructing a multi-device cooperative earthquake early warning system: based on the single point inversion of the ground vibration acceleration data, a distributed visual perception device network is deployed, the P wave first arrival time and the acceleration peak value of each site are synchronously collected; in combination with the STA / LTA first arrival picking algorithm, the seismic phase automatic identification is realized, the focal position is inverted by using the inter-station P wave arrival time difference, and the magnitude is solved by fusing the acceleration amplitude characteristics of multiple nodes; finally, according to the difference between the S wave and the P wave propagation velocity, the regional ground motion intensity prediction and the warning time window are generated to drive the second-level automatic warning response.

[0040] In the inversion process, the camera internal parameter is obtained by using the calibration board, the scene scale is perceived by depth estimation, specifically: the calibration board selects a 9x12 chessboard calibration board, the calibration result is as shown in Figure 4 The camera internal parameter and the distortion coefficient matrix are as shown in formula 2.

[0041] (2) The camera picture depth result is obtained by perceiving the scene scale through the pre-trained depth estimation model, as shown in Figure 5 .

[0042] In the inversion process, the response of the structure under excitation is obtained, and the damping ratio and natural frequency of the measured structure are obtained by applying the method of the application; specifically: a broadband random white noise excitation signal with a preset intensity and frequency range is input to the seismic simulation vibration table through the vibration control system, the top acceleration time history of the rod is obtained according to the back transmission of the acceleration sensor, the signal is transformed into the frequency domain through fast Fourier transform, and the natural frequency of the structure is 31.36.

[0043] In the inversion process, the simulated seismic motion arrives, and the ground acceleration is inverted, specifically: the amplitude-modulated seismic motion is input to the seismic simulation shaking table through the vibration control system, the top-of-pole acceleration response is identified by the method of the application, and the comparison result is as shown in Figure 6 Further, the top-of-pole structure response is used to solve the time history of the table surface acceleration of the seismic simulation shaking table according to a simplified single-degree-of-freedom model, and the comparison result is as shown in Figure 7 It can be known from the comparison result that the method of the application can correctly identify the top-of-pole acceleration response, and can correctly reflect the ground acceleration through a simplified physical model.

[0044] During the implementation of the whole method, first of all, a distributed network needs to be built in the earthquake risk area. Specifically, the network is composed of ground monitoring sites formed by multiple streetlight visual perception devices with known camera parameters. These sites are reasonably distributed in space and ensure μs-level time synchronization through high-precision timing technologies such as GPS or Beidou, laying the foundation for subsequent collaborative analysis. Each site continuously and synchronously collects ground vibration acceleration data at a high frequency and calculates it in real time on the edge device, and then transmits it to the central processing system. Secondly, the central processing system automatically processes the real-time acceleration waveform data received by each site to estimate the source parameters. Specifically, the core is to use the short-term average / long-term average (STA / LTA) first arrival picking algorithm to scan and analyze the input waveform. When the ratio of the average amplitude of a short time window (such as 0.2 seconds) to that of a longer time window (such as 4 seconds) significantly exceeds the preset threshold (such as 3.5), the system automatically identifies and accurately marks the key time point marking the P-wave first arrival, and extracts the initial acceleration value at that time and the subsequent maximum acceleration peak value (PGA). When the system confirms that at least 3 or more spatially distributed monitoring sites have successfully identified and reported the P-wave first arrival time, it enters the key source location stage. Using the precise differences (time differences) of the P-wave first arrival times recorded by these sites and the known geographic coordinates of each site, the system applies the inter-station P-wave arrival time difference inversion method to establish an equation set and solve it. This equation set is based on the propagation velocity model of P-waves in the stratum to calculate the spatial source location (latitude, longitude, and depth coordinates) of the earthquake occurring in the target area, as well as the absolute time of earthquake occurrence. Based on the obtained source location information, the system integrates the maximum acceleration amplitude (PGA) characteristics recorded after the arrival of P-waves from each monitoring site. By fusing and analyzing these PGA values distributed at different spatial locations, and according to the pre-calibrated empirical relationship model between regional seismic intensity and PGA, the system calculates the magnitude of the earthquake representing the overall energy release intensity. Then, the central processing system estimates the spatial distribution of seismic intensity and the warning time window based on the determined source location and magnitude. Specifically, based on the determined source location and magnitude, the system uses the inherent difference in propagation speed between S-waves and P-waves in the stratum (usually S-waves are slower) to perform two key prediction calculations: one is to predict and draw the spatial distribution of seismic intensity that each specific geographic location will encounter; the other is to calculate and generate the remaining time (S-wave warning time window) for each specific location to the arrival of the more destructive S-wave. Finally, the system initiates an automatic emergency response mechanism based on the generated seismic intensity prediction distribution map and the S-wave warning time window information for each location.In the post-earthquake second-level time scale, the system will differentially and hierarchically release early warning information to the designated terminals (such as emergency broadcast system, high-speed train control system, urban power and gas network control system, etc.) in the possible affected area. This mechanism aims to provide critical pre-acting time for life and property safety protection.

[0045] The application further provides an electronic device comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the method for earthquake visual early warning by combining monocular depth estimation and optical flow method when executing the computer program.

[0046] The application further provides a computer readable storage medium for storing computer instructions, wherein the computer instructions implement the steps of the method for earthquake visual early warning by combining monocular depth estimation and optical flow method when executed by a processor.

[0047] The memory in the embodiments of the present application can be a volatile memory or a non-volatile memory, or can include both volatile and non-volatile memories. The non-volatile memory can be a read only memory (ROM), a programmable read only memory (PROM), an erasable programmable read only memory (EPROM), an electrically EPROM (EEPROM), or a flash memory. The volatile memory can be a random access memory (RAM) used as an external cache. By way of example, and not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchlink DRAM (SLDRAM), and direct rambus RAM (DR RAM). It is to be noted that the memory of the method described in the present application is intended to include, but not be limited to, these and any other suitable types of memory.

[0048] In the above embodiments, all or part of the methods can be implemented by software, hardware, firmware, or any combination thereof. When implemented by software, all or part of the methods can be implemented in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of the present application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium or transferred from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions can be transferred from one website, computer, server, or data center to another website, computer, server, or data center through a wired (such as coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (such as infrared, wireless, microwave, etc.) manner. The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server, data center, etc. that includes one or more available media sets. The available media can be a magnetic medium (such as a floppy disk, a hard disk, a magnetic tape), an optical medium (such as a high-density digital video disc (DVD)), or a semiconductor medium (such as a solid state disc (SSD)), etc.

[0049] In the implementation process, each step of the above method can be completed by the integrated logic circuit of hardware in the processor or the instruction in the form of software. The steps of the method disclosed in the embodiments of the present application can be directly embodied as hardware processor execution completion, or executed by a combination of hardware and software modules in the processor. The software module can be located in a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable memory, a register, or other mature storage media in the art. The storage medium is located in the memory, and the processor reads the information in the memory and combines the hardware to complete the steps of the above method. To avoid repetition, it will not be described in detail here.

[0050] It should be noted that the processor in the embodiments of the present application can be an integrated circuit chip with a signal processing capability. In the implementation process, each step of the method embodiments can be completed by the integrated logic circuit of hardware or the instruction in the form of software in the processor. The processor mentioned above can be a general processor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. Each method, step and logic block diagram disclosed in the embodiments of the present application can be implemented or executed. The general processor can be a microprocessor or the processor can also be any conventional processor. The steps of the method disclosed in combination with the embodiments of the present application can be directly embodied as a hardware code processor for execution, or a combination of hardware and software modules in the code processor for execution. The software module can be located in a random access memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register or other mature storage medium in the art. The storage medium is located in the memory, and the processor reads the information in the memory and combines the hardware to complete the steps of the above method.

[0051] The above describes in detail the method of the present application, which combines monocular depth estimation and optical flow method for seismic visual early warning. The principle and implementation of the present application are described by using specific examples. The above description of the embodiments is only used to help understand the method of the present application and its core idea. Meanwhile, for those skilled in the art, according to the idea of the present application, the specific implementation and application range will be changed. In summary, the content of the specification should not be understood as a limitation of the present application.

Claims

1. A method for earthquake visual early warning combining monocular depth estimation and optical flow method, characterized in that: The method comprises: Step 1: Construct a single-degree-of-freedom motion model for streetlight-mounted visual perception equipment: Using outdoor streetlight-mounted monitoring equipment as the target, a simplified rigid connection model is established between the equipment and the streetlight to eliminate relative displacement interference. Combined with calibrated camera intrinsic parameters, a spatial reference for visual perception is established to provide physical constraints for vibration resolution. Step 2: Equipment vibration recognition by integrating monocular depth estimation and optical flow: Based on the surveillance video stream, monocular depth estimation is applied to calculate the scene depth distribution. Combined with an adaptive robust optical flow algorithm designed for traffic scenarios, the camera's acceleration and displacement vector in three-dimensional space are calculated to achieve accurate perception of vehicle vibration. Step 3: Invert the ground's true vibration acceleration based on visually acquired vibration perception: Use the vibration recognition system to filter out non-seismic interference signals and extract the equipment's vibration frequency domain characteristics. Based on the kinematic relationship of the single-degree-of-freedom motion model, invert the time history curve of the ground input acceleration to form a reliable representation of the seismic motion parameters for single-point inversion. Step 4. Build a multi-device collaborative earthquake early warning system: Deploy a distributed visual perception device network based on single-point inversion of ground vibration acceleration data, and synchronously collect the first arrival time and acceleration peak of each station; combine the STA / LTA first arrival picking algorithm to realize automatic identification of seismic phases, use the time difference of P wave arrival between stations to invert the source location, and integrate the acceleration amplitude characteristics of multiple nodes to solve the magnitude; finally, based on the difference in propagation speed of S waves and P waves, generate regional seismic intensity predictions and warning time windows, and drive automatic warning responses in seconds.

2. The method according to claim 1, characterized in that In step one, it is first determined that the main body of the surveillance camera is fixedly installed on the top of the street light pole; secondly, the connection between the camera and the lamp pole is simplified to a rigid connection, ignoring the actual flexibility and vibration; then, the camera internal parameters are accurately calibrated with the help of a calibration plate to ensure the geometric accuracy of the mapping between the image and the physical space; when performing ground vibration inversion analysis, the overall system including the camera and the lamp pole support structure is equivalent to a single-degree-of-freedom structure.

3. The method according to claim 1, characterized in that In step 2, an adaptive robust optical flow algorithm is used to stably extract pixel-level background key point displacements that characterize device vibrations from traffic scene video streams. This method includes two stages: robust feature point extraction and robust screening of displacement data.

4. The method according to claim 3, characterized in that The robust feature point extraction is specifically as follows: first, edge detection technology is used to identify the outline of the object in the picture, and then a high-precision line segment detection method is used to extract the significant straight line segments in the outline, and the endpoints of the line segments are accurately selected as key feature points.

5. The method according to claim 4, characterized in that The robust screening of displacement data is specifically as follows: for a large number of extracted feature points, their pixel displacements in continuous video frames are calculated, and the physical displacements calculated for all feature points are statistically analyzed using the K-means clustering algorithm to effectively eliminate abnormal outliers, and the average value of the feature point displacements within the largest cluster is selected as the final effective structural displacement corresponding to the video frame.

6. The method according to claim 5, characterized in that In step 2, a pre-trained monocular deep network is used to infer the spatial distance of key static reference points in real time, which is then fused with the optical flow displacement in space and time. Based on the calibrated camera intrinsic parameters and imaging geometry principles, the acceleration and displacement vector pose changes of the camera in three-dimensional space are calculated, that is, the direct kinematic response of the structure to ground vibration. This pose information serves as the core input to drive the single-degree-of-freedom motion model to complete the ground acceleration inversion and then invert the source parameters.

7. The method according to claim 1, characterized in that In step 3, the simplified model of the streetlight visual perception device and its vibration equation are used to invert the ground acceleration, including: Model parameter identification: Utilizing the time-domain signal of the structural vibration response generated by environmental dynamic loads during normal operation, the Fast Fourier Transform (FFT) is used to extract its frequency-domain characteristics, thereby identifying the key parameters of the single-degree-of-freedom motion model, including the natural frequency and damping ratio. Inverse calculation: Combine the identified model parameters with the monitored structural vibration response, and calculate and output the ground acceleration time history at the equipment base, i.e. the installation point, based on the single degree of freedom vibration equation. ; （1）。 8. The method according to claim 1, characterized in that In step 4, based on the P-wave first arrival time difference obtained from multiple spatially distributed stations, the inter-station P-wave arrival time difference inversion method is used to accurately calculate and determine the spatial location of the earthquake source. Further, combined with the maximum acceleration amplitude characteristics after the P-wave arrival measured at each station, the multi-node amplitude data are integrated for comprehensive analysis to solve the magnitude of the earthquake energy release intensity. On the basis of determining the location and magnitude of the earthquake source, according to the inherent difference in the propagation speed of S-wave and P-wave in the stratum, the predicted distribution map of the peak intensity of the earthquake motion at different locations in the target area is calculated, and the S-wave warning time window corresponding to each location is generated at the same time. Ultimately, the early warning system drives the automated response mechanism based on the generated earthquake intensity forecast and warning time window information, and issues early warning signals to the affected areas within seconds.

9. An electronic device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 8 are implemented.

10. A computer-readable storage medium for storing computer instructions, characterized in that: When the computer instructions are executed by a processor, the steps of the method according to any one of claims 1 to 8 are implemented.

Images