Multi-system linkage method and device for dam safety monitoring under emergency conditions

Abstract

The present invention discloses a multi-system linkage method and device for dam safety monitoring under emergency conditions, which is particularly suitable for improving the safety monitoring capability of dams in emergency situations such as earthquakes through real-time monitoring and response mechanisms. First, through the linkage of multiple sensors, data collection of the dam status is realized and stored in the database; secondly, the Flink CDC streaming data synchronization tool is used to realize real-time monitoring and fusion of strong earthquake data stored in the SQL Server database; then, the data identified as abnormal is published to the Kafka message queue in real time to ensure rapid information transmission; finally, the monitoring system determines the degree of hazard of abnormal data, automatically executes the call task, uses pyautogui to simulate mouse click operations, and simulates the execution process of the dam safety monitoring system. The present invention combines modern data synchronization technology, message queue and image recognition technology to effectively improve the efficiency and response speed of dam safety monitoring.

Description

Technical Field

[0001] The invention relates to a multi-system linkage method and device for dam safety monitoring under emergency conditions, belonging to the dam emergency linkage technology in the field of water conservancy projects. Background Art

[0002] In the field of water conservancy engineering, dam safety is an extremely important research direction. It is directly related to the safety of people's lives and property, and also poses a severe challenge to the effective management and utilization of water resources. Earthquakes are unpredictable natural disasters, and their impact on dam structures may occur in an instant. Managers need to collect data from dam monitoring instruments in a timely manner and use them for scientific analysis and research, quickly analyze the potential risks of earthquakes to dams, take emergency measures for dam anomalies, and minimize disaster losses.

[0003] At present, dam safety monitoring technology plays a vital role in the field of water conservancy projects. However, in actual application scenarios, the data collection of the dam safety monitoring system is a scheduled task, and it is impossible to complete the real-time data collection task of special working conditions; at the same time, the strong earthquake monitoring system and the dam safety monitoring system are two independent systems and run on different servers. This isolation between systems limits the real-time data collection when an earthquake occurs; in earthquake conditions, if you want to obtain real-time monitoring data of the dam, manual intervention is required, and there is a lack of intelligence and adaptability in earthquake conditions. Nowadays, system linkage solutions are widely used in various fields. In earthquake conditions, Flink CDC is used to collect data from the strong earthquake monitoring system, and the data that meets the trigger conditions is pushed to the message queue. The message queue pushes the strong earthquake data to the dam safety monitoring system. According to the magnitude of the peak acceleration of the earthquake, the data collection task is automatically triggered in stages, which can achieve automatic triggering between multiple systems without manual intervention. Therefore, based on the field of hydraulic engineering, by analyzing the data in the strong earthquake system, the real-time linkage of the dam safety monitoring system to collect data has important research value for event risk control, danger prevention, and potential risk resistance.

[0004] In the traditional monitoring system, the lack of close connection between the strong earthquake monitoring system and the dam safety monitoring system means that even with the real-time data of the strong earthquake monitoring system, this information cannot be transmitted to the dam safety monitoring system in a timely and effective manner. This disconnection may lead to poor information flow between the monitoring systems, missing critical response opportunities, and thus posing serious hidden dangers to the safety of the dam. Especially in the event of a large earthquake, human intervention may be hindered by environmental factors, making it impossible to quickly and safely complete the data collection of the dam monitoring equipment in an emergency. This bottleneck limits our ability to understand the dam's condition in a timely manner when an earthquake occurs, and also restricts our ability to take quick and effective measures to ensure the safety of the dam. Summary of the invention

[0005] Purpose of the invention: In view of the problems and deficiencies in the prior art, the present invention provides a multi-system linkage method and device for dam safety monitoring under earthquake conditions. Specifically, when the dam strong earthquake system monitors the arrival of an earthquake, a FlinkCDC streaming data synchronization tool is used to process a large number of strong earthquake-related data that need to be calculated in real time, and analyze whether the peak acceleration of the earthquake reaches the trigger condition for multi-system linkage. If the trigger condition is met, the feature of Flink that can directly interact with the Kafka message queue will be utilized, and the disk reading and writing will be avoided through Kafka's zero-copy technology. At the same time, Kafka can support accurate consumption semantics at a low cost, and push the strong earthquake data to the Kafka message queue. As a consumer, the dam automation monitoring system will take out the strong earthquake data from the message queue, match the peak acceleration with the call level, and clarify the collection tasks that the data collection system needs to perform in the next step. Using standard correlation matching based on Open CV, the corresponding test task and calculation task images in the call test level are matched with the real-time screenshots, the pixel positions of the test task and calculation task are located, and the mouse click event is simulated to complete the real-time data collection of the concerned measuring points by the dam safety monitoring system, thereby providing data support for the dam emergency response under earthquake conditions, quickly locating the location of the problematic dam section, and supporting relevant personnel to quickly analyze the dam operation safety status under emergency conditions. The application of this technology can improve the real-time and accuracy of the dam's emergency response under earthquake conditions, and can be applied to multi-system linkage in other fields, which can realize the decoupling of the complicated manual operation steps of multi-system linkage in dangerous emergency conditions and realize the automation of the whole process.

[0006] The present invention proposes a multi-system linkage scheme for dam safety monitoring based on Flink CDC streaming data synchronization tool under dam earthquake emergency conditions. This scheme forms an efficient data synchronization mechanism through real-time data monitoring and processing, and transforms data monitoring and response tasks into real-time data processing tasks, solving the problems of large data processing delays and untimely responses in traditional monitoring systems. At the same time, it can solve the problem of insufficient data synchronization and response capabilities caused by limited processing capabilities in existing methods when processing large-scale data streams and considering complex correlations between data. It provides a solution to consider personnel safety under dangerous emergency conditions and realize automation of the monitoring to collection process.

[0007] Technical solution: A multi-system linkage method for dam safety monitoring under emergency conditions, using seismic sensor equipment to obtain earthquake characteristic parameters in real time, and storing the monitored earthquake characteristic parameters in real time; then, through the Flink CDC streaming data synchronization tool, these earthquake characteristic parameters are processed and analyzed in real time, including calculating the earthquake peak acceleration and judging whether the earthquake peak acceleration reaches the preset multi-system linkage trigger condition. When the system determines that the earthquake peak acceleration exceeds the threshold, it will match the name of the dam automation monitoring and collection task within the corresponding threshold range, and use the direct interaction ability of Flink and Kafka to push the key earthquake data and the name of the dam automation monitoring and collection task to be executed to the message queue through Kafka's zero copy technology, avoiding the delay caused by disk reading and writing, and ensuring the efficiency and real-time performance of data transmission. At this time, the dam safety monitoring system, as a consumer, will obtain the latest earthquake data and the name of the task to be executed from the Kafka message queue. Finally, run the script based on pyautogui and Open CV, use Open CV to compare the current screen capture with the task name image, accurately locate the screen position where the task needs to be executed, use pyautogui to simulate the mouse click operation at the specified position, and complete the automatic collection process of dam safety data by the dam monitoring system.

[0008] The methods include the following:

[0009] (1) Linked data collection

[0010] Multiple seismic sensor devices are used and distributed at key structural locations of the dam to automatically and in real time collect dam vibration data in order to capture potential danger signals. The collected data will be stored in the event folder in the .evt file format. In addition, the system uses an automated script based on the pywin32 library to monitor new event files in the event folder in real time. The script is responsible for extracting the acceleration data of the east-west, north-south and vertical components recorded in the file, and storing the data extracted by the script as earthquake characteristic parameters in the database for subsequent analysis.

[0011] The key structural positions of the dam (such as those distributed at the key supporting parts or connection points of the dam) are locations that can reflect the strength of the earthquake suffered by the core parts of the dam; earthquake sensor equipment can be installed at the bottom of the dam to monitor earthquake fluctuations at the earliest and provide strong earthquake data reference; the key supporting parts or connection points of the dam are the core of the dam structure, and monitoring these can help evaluate the stability of the overall structure of the dam.

[0012] (2) Real-time monitoring of linkage data

[0013] The Flink CDC streaming data synchronization tool is used to process and analyze earthquake characteristic parameters in real time. This tool can monitor and capture the newly added earthquake characteristic parameters in the database in real time. By setting the time window, the peak acceleration calculated at each measuring point within this time period is verified. The peak acceleration of more than half of the measuring points reaches the minimum trigger threshold. The specific task name that needs to be performed by the dam automation monitoring system is queried and determined based on the recorded maximum peak acceleration. The peak acceleration analysis result of the earthquake characteristic parameters is encapsulated as a strong earthquake message and pushed to the Kafka message queue.

[0014] After calculation of earthquake characteristic parameters, peak acceleration can be obtained, and the peak acceleration can be used to measure the intensity of the earthquake. According to the calculated peak acceleration value, in order to prevent errors, a time window will be used to ensure that multiple measuring points detect earthquakes within the same window period. Only then will the maximum value of the peak acceleration of these detected earthquake measuring points be matched with the corresponding call task (different call tasks require different numbers of dam measuring points to be collected, which are set according to different earthquake intensities).

[0015] (3) Abnormal data release

[0016] The dam safety monitoring system is configured as a consumer of the Kafka message queue, receiving strong earthquake messages from the Kafka queue in real time. Whenever the system successfully consumes a strong earthquake message, it will parse the message to obtain the name of the task to be executed. The system then matches this task name with the corresponding task folder in the local Earthquake folder, which contains all the icons for executing the call task. These icons are used to provide positioning information during the automated execution of the call task.

[0017] (4) Automatic execution of the dam safety monitoring system call task

[0018] The pyautogui library is used to take a high-precision screenshot of the current operating system interface, and then Open CV is used for image processing. The real-time captured screen image is compared with the task name icon pre-stored in the "Earthquake" folder through a standard correlation matching algorithm to see if it matches. If it matches, the exact position of the task icon on the screen will be determined, and the position information will be converted into the screen pixel position. pyautogui will perform automatic click operations based on the pixel position, thereby triggering the dam safety monitoring system to call for measurement tasks, automatically collect real-time data from various instruments, and facilitate subsequent dam safety status analysis and comprehensive evaluation tasks.

[0019] The multi-system linkage method for dam safety monitoring under earthquake conditions is embedded in the dam emergency response system and runs on computer equipment. First, different trigger tasks can be configured in the dam emergency response system according to different peak acceleration ranges. At the same time, each task name corresponds to a call task name in the dam safety monitoring system, and its task icon is stored in the "Earthquake" folder; when an earthquake occurs, the strong earthquake information collection script will extract the acceleration data of the east-west, north-south and vertical components from the latest event file and store them in the database; use the Flink-based The CDC streaming data synchronization tool obtains the newly added strong earthquake data from the database in real time. In a sliding window of fixed time, it determines whether the peak acceleration calculated by more than half of the measuring points exceeds the linkage trigger threshold. If so, it takes out the linkage task in the peak acceleration maximum value matching rule, and writes the strong earthquake data and the name of the call task to be executed into the Kafka message queue; as a consumer, the dam safety monitoring system takes out the latest strong earthquake data from the Kafka message queue, parses out the name of the call task to be executed, uses image recognition technology to calculate the pixel position of the call task icon on the current screen, and uses pyautogui to realize the automated simulation click operation of the entire call task process, automatically collects real-time data from each instrument, and provides data support for relevant departments to timely analyze the real-time safety status of the dam.

[0020] Furthermore, the step (1) is linked to data collection, and the specific steps are as follows:

[0021] (1.1) Source of earthquake characteristic parameters: Multiple earthquake sensor devices are deployed at key structural parts of the dam to comprehensively capture vibration data including acceleration, vibration frequency, duration and other data of the east-west, north-south and vertical components. Whenever an earthquake activity is detected, the sensor automatically records the data and transmits it to the strong earthquake server. The data is packaged into an event file in .evt format and stored in the event folder to facilitate the subsequent processing and analysis of earthquake data. The file naming format is "year-month-day-hour-minute-second + measurement point number".

[0022] (1.2) Extraction of earthquake characteristic parameter data: By implementing an automated script based on the pywin32 library, the event folder is monitored in real time and the earthquake characteristic parameters in the latest file are extracted. The event folder is scanned using the glob method and sorted according to the timestamp in the file name. The current year, month and day are obtained and matched with the year, month and day in the file name. Only the event file of the day is executed, and the processed files are marked. Only the latest event file of the day is processed each time. Using the pywin32 clipboard function, the acceleration, vibration frequency, duration and other data of the east-west, north-south and vertical components in the event file are stored in the earthquake characteristic parameter database table.

[0023] Furthermore, in step (2), the linkage data is monitored in real time, and the Flink CDC streaming data synchronization tool is used to process and analyze the earthquake characteristic parameters in the linkage data collection in real time. This tool can monitor and capture the newly added earthquake characteristic parameters in the database in real time, and verify the peak acceleration calculated at each measuring point within this time period by setting a time window. If the peak acceleration of more than half of the measuring points reaches the minimum trigger threshold, the system will query and determine the specific task name that needs to be executed by the dam automation monitoring system based on the recorded maximum peak acceleration, and push the strong earthquake message to the Kafka message queue.

[0024] (2.1) Data capture and synchronization: For the newly added earthquake characteristic parameters in the database table, use the Flink CDC tool to monitor the changes in the database table in real time. Apache Flink is an event-driven distributed stream processing engine that can be deployed in a cluster environment to perform fast memory-based calculations on data of various sizes. It is applied to distributed, high-performance, and highly available real-time data stream applications. After it is enabled, it will capture changes in the data stream by reading the database log file. Enable the SQL Server Agent in the SQL Server "Object Resource Management". Then, for the strong earthquake database that needs to be monitored, enable the database-level CDC function. Finally, enable the table-level CDC for the table that stores earthquake characteristic parameters. At this time, a data change table about earthquake characteristic parameters is added to the system table to record the real-time information of the addition, deletion, and modification of the earthquake characteristic parameter data table. At this time, when a new earthquake characteristic parameter record is inserted into the database in step (1), the system table will capture this new data. The background can use CDC as the data source to read the new data of the SQL Server earthquake characteristic parameter data table, and use the SourceFunction object to obtain the new streaming data, indicating that new earthquake information has arrived at this time, and it is necessary to calculate and determine whether to trigger linkage.

[0025] (2.2) Setting the time window: During earthquake monitoring, the earthquake characteristic parameters of each measuring point are collected in real time into the Flink data synchronization tool. In order to capture the instantaneous data when an earthquake occurs and prevent the false alarm of a single measuring point from causing the wrong triggering of the call task, an appropriate time window is set to calculate and fuse the earthquake characteristic parameters collected at the measuring point within the appropriate time, ensuring that the data is collected and calculated within the same time window, thereby increasing the probability of successful triggering of the call task and facilitating timely response and analysis of earthquake data.

[0026] (2.2.1) Setting the size of the time window: Through simulation experiments, the time when all measurement points are inserted into the database and monitored by Flink when an earthquake occurs is simulated, and this time is set as the size of the time window, which is set through the window() method of the DataStream object.

[0027] (2.2.2) Fusion of earthquake characteristic parameters at multiple measuring points: Set the earthquake occurrence timestamp as the window grouping basis, so that the strong earthquake data collected at different measuring points at the same time will enter the same time window. For the east-west, north-south and vertical component accelerations contained in one measuring point, use the following formula to calculate the peak acceleration d i (t), and record the maximum value D(t) of the peak acceleration at all measuring points:

[0028]

[0029] D(t)=Max(d i (t),d i-1 (t))

[0030] Among them, d i (t) represents the peak acceleration of the i-th measuring point, is the east-west component acceleration, is the north-south component acceleration, is the vertical component acceleration.

[0031] At the same time, the calculated peak acceleration is compared with the minimum trigger peak acceleration for the call test task. If the comparison result exceeds the minimum trigger peak acceleration, the counter will count the value of the measuring point. When the peak acceleration statistics of all measuring points in the sliding window are completed, the counter will count the number of measuring points that need to execute the call test task.

[0032] (2.2.3) Data transmission trigger condition setting: When the number of measurement points counted by the counter exceeds half of the total number of measurement points, an earthquake is very likely to occur. The maximum value of the peak acceleration is matched with the trigger condition rule to determine the name of the corresponding call task for the peak acceleration. By using the feature that Flink can directly interact with the Kafka message queue, the task name, peak acceleration, timestamp and other information are encapsulated as a strong earthquake message and passed into the Kafka message queue.

[0033] Furthermore, in the step (3) of abnormal data release, the dam safety monitoring system is configured as a consumer of the Kafka message queue, and receives strong earthquake messages from the Kafka queue in real time. Whenever the system successfully consumes a strong earthquake message, the system will parse the message to obtain the name of the task to be executed. The system then matches this task name with the corresponding task folder in the local Earthquake folder, which contains all the icons for executing the call task. These icons are used to provide positioning information during the automated execution process.

[0034] (3.1) Set the producer: Configure Flink as the producer, use the fused data of Flink as the data produced by the producer, and store the data in the strong earthquake topic specified by the message queue.

[0035] (3.2) Set up consumers: Establish a connection with the dam safety monitoring system server through the Kafka client API and subscribe to the strong earthquake message topic. At this time, the dam safety monitoring system monitors and receives the strong earthquake messages published in the source Kafka queue in real time.

[0036] (3.3) Message parsing and task matching: As a consumer, the dam safety monitoring system listens to new strong earthquake data stored in Kafka, obtains the new strong earthquake message and parses the content of the JSON data to extract the name of the task to be executed. This task name will be matched with the task folder name under the local Earthquake folder. After matching, the icon displayed in the system for the executed task will be obtained, providing positioning information for step (4).

[0037] Furthermore, in step (4), the dam safety monitoring system automatically executes the task call, uses pyautogui to intercept the current operating system interface, and verifies through Open CV whether the current page contains a pre-stored task icon. If the match is successful, the system will automatically locate and click the icon, triggering the dam monitoring task and automatically collecting relevant data for subsequent analysis.

[0038] (4.1) Screen capture and image processing: Take a screenshot of the current system interface every 1 second as the target image based on the OpenCV template matching method. There are two different situations. The first situation captures the main interface (excluding the task bar) as the target image, which is mainly used when the dam safety monitoring system is not started and the call task is executed; the second situation captures the task bar part, which is mainly used when the dam safety monitoring system has been started and is reduced to the task bar and needs to be moved to the top of the desktop for operation.

[0039] (4.1.1) System main interface capture: Use the screenshot() method of the pyautogui library and pass in the screenshot area parameter. The parameter is calculated as follows:

[0040] desktop_box=(0,0,screen_width,screen_height-toolbar_height)

[0041] Among them, desktop_box represents the area of ​​the system interface. The size() method of the pyautogui library is used to obtain the screen width as screen_width, the height as screen_height, and the taskbar height as toolbar_height at the current resolution; the first and second parameters are both 0, indicating that the starting point of the screenshot is located in the upper left corner of the screen; the third parameter screen_width represents the width of the screenshot area, which is the width of the entire screen; the fourth parameter screen_height-toolbar_height represents the height of the screenshot area, which is the current screen height minus the taskbar height, so that a screenshot of the system main interface is obtained.

[0042] (4.1.2) System taskbar capture: Use the screenshot() method of the pyautogui library and pass in the screenshot area parameter. The parameter is calculated as follows:

[0043] toolbar_box=(0,screen_height-toolbar_height,screen_width,toolbar_height)

[0044] Among them, toolbar_box represents the area of ​​the system interface taskbar. The size() method of the pyautogui library is used to obtain the screen width as screen_width, the height as screen_height, and the taskbar height as toolbar_height at the current resolution; the first parameter is 0, and the second parameter is screen_height-toolbar_height, which means that the starting point of the taskbar on the left side of the screen is used as the starting point of the screenshot; the third parameter represents the width of the screenshot area, which is the width of the entire screen; the fourth parameter toolbar_height represents the height of the screenshot area, so that a screenshot of the system taskbar is obtained.

[0045] (4.2) Standard correlation matching based on Open CV: Standard correlation matching is a statistical method that evaluates similarity by calculating the correlation coefficient between the template image and the target image. In step (4.1), a real-time screenshot of the screen has been obtained and used as the target image. In step (3.3), the task icon in the task folder is found by matching the task name and used as the template image. Standard correlation matching slides the template image on the target image. For each position where the template image slides on the target image, the sum of the products of the template image and the pixel points of the area it covers is calculated and normalized, as shown in the following formula:

[0046]

[0047] For each position of the template T on the target image I, remove the overlapping image blocks, calculate the similarity measurement results, and store them in the result matrix R; each (x, y) coordinate in the target image contains an entry in the result matrix R, unless the template is out of bounds; (x', y') represents the coordinates in the template image T. When the template image completely overlaps with the original image, the position of the target image where the upper left corner of the template image is located stores the maximum similarity between the template and the target image, indicating that the task icon appears in the current interface.

[0048] Use the minMaxLoc() method in the Open CV library to find the result matrix R width×height The minimum and maximum values ​​and their positions. Since we use the standard correlation matching method, we only focus on the position of the maximum value. If the maximum value is lower than 0.7, the template image is not found in the target image, that is, the test task icon does not appear in the current interface; if the maximum value is greater than 0.7, it is considered that the template image is found in the target image, that is, the test task icon appears in the current interface, the test task can be triggered, and the center coordinate position of the target image in the template image is returned.

[0049] (4.3) Automatic triggering of the call-up task: Based on the exact position of the task icon on the screen obtained in step (4.2), this position is converted into the position of screen pixels, and then the doubleClick() method in the pyautogui library is used to execute the call-up task. After the task is completed, the data of the relevant measuring points of the dam are collected back to the database in real time. Relevant personnel can use this information to conduct real-time analysis of the safety status of the dam and take corresponding emergency measures based on the analysis results.

[0050] Furthermore, by using the above method in the dam emergency response system, it is possible to quickly collect the operating status and abnormal conditions of various points on the dam during dangerous emergency response conditions. The results can provide data support for the dam emergency response measures, assist relevant personnel who cannot enter the dam during dangerous emergency conditions, and can also obtain the dam operating status in real time, and conduct scientific analysis and research, so as to quickly understand the abnormal parts of the dam during dangerous conditions, and facilitate managers to make scientific and reasonable emergency disposal measures in a timely manner to minimize disaster losses.

[0051] A multi-system linkage device for dam safety monitoring under earthquake conditions, comprising:

[0052] Linkage data acquisition module: Multiple seismic sensor devices are used to distribute them at key structural locations of the dam to automatically and in real time collect dam vibration data in order to capture potential danger signals. The collected data will be stored in the event folder in .evt file format. In addition, an automated script based on the pywin32 library is used to monitor new event files in the event folder in real time. The script is responsible for extracting the acceleration data of the east-west, north-south and vertical components recorded in the file, and storing these key data in the database for subsequent analysis.

[0053] Real-time monitoring module for linkage data: Use the Flink CDC streaming data synchronization tool to process and analyze the earthquake characteristic parameters in the linkage data acquisition module in real time. This tool can monitor and capture the newly added earthquake characteristic parameters in the database in real time. By setting the time window, the peak acceleration calculated at each measuring point within this time period is verified. If the peak acceleration of more than half of the measuring points reaches the minimum trigger threshold, the specific task name that needs to be performed by the dam automation monitoring system will be queried and determined based on the recorded maximum peak acceleration, and the message will be pushed to the Kafka message queue.

[0054] Abnormal data publishing module: The dam safety monitoring system is configured as a consumer of the Kafka message queue and receives earthquake messages from the Kafka queue in real time. Whenever the system successfully consumes an earthquake message, it will parse the message to obtain the name of the task to be executed. The system then matches this task name with the corresponding task folder located in the local Earthquake folder, which contains all the icons for executing the call task. These icons are used to provide positioning information during the automated execution process.

[0055] Automatic execution module for the dam safety monitoring system call task: Use the pyautogui library to take high-precision screenshots of the current operating system interface, then use Open CV for image processing, and use an algorithm to compare the real-time captured screen image with the task name icon pre-stored in the "Earthquake" folder to see if it matches. If it matches, the exact position of the task icon on the screen will be determined, and the position information will be converted into the screen pixel position. Pyautogui will perform automatic click operations based on the pixel position, thereby triggering the dam safety monitoring system call task and automatically collecting real-time data from each instrument to facilitate subsequent dam safety status analysis and other tasks.

[0056] The implementation process and method of the system are the same and will not be repeated here.

[0057] A computer device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the multi-system linkage scheme for dam safety monitoring under earthquake conditions as described above is implemented.

[0058] A computer-readable storage medium stores a computer program for executing the multi-system linkage scheme for dam safety monitoring under earthquake conditions as described above.

[0059] Beneficial effects: Compared with the prior art, the data processing delay of the existing monitoring system is large and the response is not timely. At the same time, multiple monitoring systems are relatively independent and cannot trigger each other to complete the call task in emergency conditions, providing real-time analysis for the dam operation status. The present invention utilizes the efficient data processing capability of Flink CDC to capture and synchronize data changes in real time, thereby ensuring that the monitoring system can respond quickly and accurately in emergency situations such as earthquakes, enhancing the real-time and accuracy of data processing; at the same time, Kafka is used to store abnormal data, and the first-in-first-out characteristics of the message queue are used to sequentially push abnormal data to the dam safety monitoring system; using the standard correlation matching based on Open CV, the call process of the dam safety monitoring system is configured as an automatic execution process, and the corresponding task icon is matched and automatically executed according to different degrees of abnormal data, so as to collect real-time data of dam operation, and assist relevant personnel in earthquakes and other highly destructive emergency conditions. , Can safely obtain the real-time operation status of the dam, and make effective judgments and adjust corresponding measures based on this status, and effectively ensure the safety of the dam and personnel. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] Figure 1 This is a Flink-based time window setting model architecture diagram for an embodiment of the present invention;

[0061] Figure 2 is a flow chart of a method according to an embodiment of the present invention;

[0062] Figure 3 It is a functional module diagram of an embodiment of the present invention. DETAILED DESCRIPTION

[0063] The present invention is further explained below in conjunction with specific embodiments. It should be understood that these embodiments are only used to illustrate the present invention and are not used to limit the scope of the present invention. After reading the present invention, various equivalent forms of modifications to the present invention by those skilled in the art all fall within the scope defined by the claims attached to this application.

[0064] like Figure 2 As shown in FIG. 1 , the multi-system linkage method for dam safety monitoring under earthquake conditions includes the following steps:

[0065] (1) Linked data collection

[0066] (1.1) Source of earthquake characteristic parameters: Multiple seismic sensors are deployed at key structural locations of the dam to measure acceleration in the east-west, north-south and vertical directions, as well as vibration frequency and duration. These sensors have the ability to automatically respond to seismic activity and can record data immediately when an earthquake occurs. The recorded data is packaged into an event file in .evt format, and the file naming follows the format of "year-month-day-hour-minute-second + measurement point number" to facilitate subsequent data processing and analysis. Figure 3 As shown, the working condition types include earthquakes but are not limited to earthquake conditions. This device can be applied to a variety of emergency conditions, such as sudden water level changes and flood conditions.

[0067] (1.2) Extraction of earthquake characteristic parameters: Use an automated script based on the pywin32 library to monitor the event folder in real time to extract the latest earthquake characteristic parameters. The script uses the glob method to scan the event folder and sort the files according to the timestamp in the file name to ensure that the latest event file of the day is processed each time. By comparing the year, month, and day in the file name with the current date, the system only processes the files of the day and marks the processed files to avoid repeated processing. During the file processing process, the automated script uses the clipboard function of pywin32 to automatically extract the earthquake characteristic parameters in the event file, such as the acceleration, vibration frequency, and duration of the east-west, north-south, and vertical components, and stores these data in the earthquake characteristic parameter database table for subsequent analysis and reference.

[0068] as follows Figure 3 As shown, the earthquake characteristic parameters will be stored from the sensor into the corresponding earthquake characteristic parameter database table for persistent storage.

[0069] (2) Real-time monitoring of linkage data: Use the Flink CDC streaming data synchronization tool to process and analyze the earthquake characteristic parameters in the linkage data acquisition module in real time, and push abnormal data to the Kafka message queue.

[0070] (2.1) Data capture and synchronization: Use the Flink CDC tool to monitor the changes in the strong earthquake database table in SQL Server in real time, and capture and synchronize the newly added earthquake characteristic parameters in the database. For example, if we put the strong earthquake data into the data table in the EarthquakeData database, we need to enable the database-level CDC function for the strong earthquake database EarthquakeData, and enable the table-level CDC function for the data table that stores the earthquake characteristic parameters. This operation will add a data change table dbo_data_CT to the system table under the EarthquakeData database, which records the addition, deletion, and modification activities of the strong earthquake data table in real time. When the earthquake characteristic parameter record is inserted into the strong earthquake database, the system table will capture this new data. The background can use CDC as the data source and read the new data stream of the SQL Server strong earthquake data table through the SourceFunction object, so as to obtain new earthquake information in a timely manner and calculate whether to trigger the corresponding linkage mechanism.

[0071] as follows Figure 3 As shown in the figure, Flink captures the changes of earthquake characteristic parameters in the database in real time and can obtain newly added earthquake characteristic parameters in time, indicating that a sensor has detected an earthquake at the current moment.

[0072] (2.2) Setting the time window: When an earthquake occurs, the earthquake characteristic parameters of each measuring point are collected in real time to the Flink data synchronization tool. An appropriate time window is set to collect the earthquake characteristic parameters of each measuring point where the earthquake occurs at the same time. These parameters are calculated and integrated to improve the accuracy of the call task triggering. The specific process is as follows: Figure 1 shown.

[0073] (2.2.1) Setting the size of the time window: Simulate an earthquake in the strong earthquake system and observe the time interval for all measurement points to be inserted into the database and monitored by Flink. For example, if there are 10 measurement points in total and it takes 5 seconds to simulate an earthquake, store the earthquake characteristic parameters in the database and monitor them by Flink, then pass TumblingEventTimeWindows.of(Time.seconds(5)) as a parameter to the window() method of the DataStream object for setting.

[0074] (2.2.2) Multi-point strong earthquake data fusion: Set the earthquake occurrence timestamp as the window grouping basis, so that the strong earthquake data collected by different measuring points at the same time will enter the same time window. For each measuring point in the window, the east-west, north-south and vertical component acceleration is calculated using the formula Calculate the peak acceleration value of each measuring point and use the formula D(t) = Max(d i(t),d i-1 (t)) to obtain the maximum value of the peak acceleration in the sliding window. Then compare the calculated peak acceleration with the minimum trigger peak acceleration for the call test task. If the comparison result exceeds the minimum trigger peak acceleration, the counter will count the value of the measuring point. When the peak acceleration statistics of all measuring points in the sliding window are completed, the counter will count the number of measuring points that need to execute the call test task.

[0075] (2.2.3) Data transmission trigger condition setting: When the number of measuring points counted by the counter exceeds half of the total number of measuring points, it means that multiple measuring points have detected vibrations, and an earthquake is very likely to occur. Since different peak accelerations determine different tasks performed by the dam safety monitoring system, users can set the peak acceleration trigger interval and the name of the task to be executed within the peak acceleration interval. At this time, the maximum value of the peak acceleration can be matched with the trigger condition rule to determine the name of the corresponding call task that should be executed for the peak acceleration. By taking advantage of the feature that Flink can directly interact with the Kafka message queue, the task name, peak acceleration, timestamp and other information are combined into strong earthquake data and passed into the Kafka message queue.

[0076] as follows Figure 1 As shown in the figure, for example, 10 earthquake monitoring sensor devices are deployed in the dam, namely, measuring points 1 to 10. We use the time of earthquake occurrence as the basis for window allocation, and can aggregate the data of the measuring points in the same time period. Here, in the earthquake condition, the data we aggregate include the acceleration of the east-west, north-south and vertical components. According to the window allocation, the data in the same window are fused according to the fusion function in the figure, the peak acceleration is calculated and the maximum peak acceleration of the 10 measuring points is recorded. When half of the measuring points in the same window exceed the minimum trigger threshold of linkage, that is, the peak acceleration of at least 5 measuring points in the window is greater than the threshold of linkage task triggering, it can be determined that an earthquake is very likely to occur. The maximum value of the peak acceleration in the window and the relevant information of the measuring point can be encapsulated as strong earthquake information and transmitted to the Kafka message queue.

[0077] (3) Abnormal data release

[0078] (3.1) Set up the producer: Configure Flink as a producer and create a topic named mutisystem_earthquake in Kafka. Store the fused earthquake data from Flink into the topic and push the earthquake data to the message queue.

[0079] (3.2) Set up consumers: Establish a connection with the dam safety monitoring system server through the Kafka client API and subscribe to the earthquake message topic. At this time, the dam safety monitoring system monitors and receives the strong earthquake data messages published in the source Kafka queue in real time.

[0080] (3.3) Message parsing and task matching: As a consumer, the dam safety monitoring system listens to new strong earthquake data stored in Kafka, obtains the new strong earthquake message and parses the content of the JSON data to extract the name of the task to be executed. This task name will be matched with the task folder name under the local Earthquake folder. After matching, the icon displayed in the system for the executed task will be obtained, providing positioning information for step (4).

[0081] as follows Figure 3 As shown in the figure, strong earthquake messages follow the first-in-first-out principle of the queue to execute linkage tasks in sequence.

[0082] (4) Automatic execution of the dam safety monitoring system call task

[0083] (4.1) Screen capture and image processing: The dam safety monitoring system interface is captured every 1 second as a template image based on the Open CV template matching method. There are two different situations. The first one considers that the dam safety monitoring system has been started and the call task can be executed directly; the second one considers that the dam safety monitoring system has not been started and needs to find the dam safety monitoring system icon in the current interface and run it before executing the corresponding call task.

[0084] (4.1.1) Capture of the system main interface: At this time, the dam safety monitoring system has not been started, and it is necessary to capture the part of the system interface except the task bar. Or at this time, the system has been started and is in an activated state, and it is necessary to execute the call task. Use the screenshot() method of the pyautogui library and pass in the parameter desktop_box, which means that only the system interface area is captured as the target image.

[0085] (4.1.2) System taskbar capture: At this time, the dam safety monitoring system has been started and can be activated from the taskbar. You only need to take a screenshot of the taskbar of the system interface. Use the screenshot() method of the pyautogui library and pass in the parameter toolbar_box, which indicates that the area of ​​the system interface taskbar is used as the target image.

[0086] (4.2) Standard correlation matching based on Open CV: In step (4.1), a real-time screenshot of the screen is obtained and used as the target image; in step (3.3), the task icon in the task folder is found by matching the task name and used as the template image.

[0087] When the dam safety monitoring system is not started, the dam safety monitoring system icon in the Earthquake folder is used as the template image, and the screenshot of the system main interface is used as the target image. The dam safety monitoring system icon is slid on the screenshot of the system main interface. For each sliding position of the dam safety monitoring system icon on the screenshot of the system main interface, the sum of the products of the dam safety monitoring system icon and its pixel points in the screenshot area of ​​the system interface is calculated, and normalized. The result is stored in the matrix R, as shown in the following formula:

[0088]

[0089] Use the minMaxLoc() method in the Open CV class library to find the maximum value in the result matrix. If the value is lower than 0.7, the dam safety monitoring system icon is not found from the system interface screenshot area, and it is necessary to return to the main interface. Use the hotkey('win','d') method of pyautogui to return to the main interface and continue searching; if the maximum value is greater than 0.7, the dam safety monitoring system icon is found from the system interface screenshot area and the center coordinate position of the target image in the template image is returned.

[0090] The same principle applies to the automatic execution of the call task, so I will not repeat it here. However, it should be noted that when the dam safety monitoring system is started, we need to activate it from the taskbar to the top of the page. At this time, the dam safety monitoring system icon will be matched with the taskbar screenshot. When calculating the center coordinates, the height needs to be added to the height of the system main interface based on the calculation result to get the actual position of the dam safety monitoring system icon in the entire interface.

[0091] (4.3) Automatic triggering of the call-up task: Based on the exact position of the task icon on the screen obtained in step (4.2), this position is converted into the position of screen pixels, and then the doubleClick() method in the pyautogui library is used to execute the call-up task. After the task is completed, the data of the relevant measuring points of the dam are collected back to the database in real time. Relevant personnel can use this information to conduct real-time analysis and comprehensive evaluation of the safety status of the dam, and take corresponding emergency measures based on the analysis results.

[0092] Obviously, those skilled in the art should understand that the various steps of the multi-system automatic linkage scheme for dam safety monitoring under earthquake conditions or the multi-system automatic linkage system for dam safety monitoring under earthquake conditions of the above-mentioned embodiments of the present invention can be implemented by a general computing device, which can be concentrated on a single computing device or distributed on a network composed of multiple computing devices. Optionally, they can be implemented by program codes executable by computing devices, so that they can be stored in a storage device and executed by the computing device, and in some cases, the steps shown or described can be executed in a different order from this, or they can be made into individual integrated circuit templates, or multiple modules or steps in them can be made into a single integrated circuit module for implementation. In this way, the embodiments of the present invention are not limited to any specific combination of hardware and software.

Claims

1. A multi-system linkage method for dam safety monitoring under emergency conditions, characterized in that: It includes the following: (1) Linked data collection: collect data on dam earthquake characteristic parameters in real time, and dump them into event files and store them in event folders. Read newly stored event files in real time, parse the strong earthquake related data in the files, and realize instant warehousing and storage of data. (2) Real-time monitoring of linkage data: Configure the Flink CDC synchronization tool to continuously monitor and capture data updates in the database to ensure that all changes in strong earthquake data are identified and processed in real time, allowing the system to respond quickly to the latest strong earthquake data; (3) Abnormal data release: When strong earthquake data exceeding the monitoring threshold is identified, the data is automatically encapsulated and published to the Kafka message queue, so that the information can be quickly distributed to the dam safety monitoring system that needs to perform emergency response tasks; (4) Automatic execution of the dam safety monitoring system call task: Through automated control scripts and image recognition technology, the mouse click operation is simulated to realize the automatic execution of the dam safety monitoring system call task; In the (2) real-time monitoring of linkage data, the Flink CDC tool is used to process and analyze earthquake characteristic parameters in real time. The specific steps are as follows: (2.1) Data capture and synchronization: For the newly added earthquake characteristic parameters in the database table, use the Flink CDC tool to monitor the changes in the database table in real time; (2.2) Set a time window to collect the earthquake characteristic parameters of all measuring points within the same earthquake occurrence time for calculation and fusion; (2.2.1) Setting the size of the time window: Through simulation experiments, the time when all measurement points are inserted into the database and monitored by Flink when an earthquake occurs is simulated, and this time is set as the time window size; (2.2.2) Fusion of strong earthquake data from multiple measuring points: Set the earthquake occurrence timestamp as the window grouping basis, so that the strong earthquake data collected by different measuring points at the same time will enter the same time window, calculate the peak acceleration based on the earthquake characteristic parameters, and record the maximum value of the peak acceleration and the number of measuring points that exceed the threshold for triggering the call for measurement task; (2.2.3) Data transmission trigger condition setting: When the number of measurement points whose statistics are greater than the trigger threshold exceeds half of the total number of measurement points, the information of the measurement point corresponding to the maximum value of the peak acceleration is transmitted to the Kafka message queue.

2. The multi-system linkage method for dam safety monitoring under emergency conditions according to claim 1 is characterized in that: In the linkage data collection, earthquake sensing equipment is used to automatically and in real time collect dam vibration data, and the collected data is stored in an event folder in the .evt file format; An automated script based on the pywin32 library is used to monitor new event files in the event folder in real time. The script is responsible for extracting the acceleration data of the east-west, north-south and vertical components recorded in the file, and storing the acceleration data in the earthquake characteristic parameter database for subsequent analysis.

3. The multi-system linkage method for dam safety monitoring under emergency conditions according to claim 1 is characterized in that: In the aforementioned (3) abnormal data release, the dam safety monitoring system is set as a consumer of the Kafka message queue and receives strong earthquake messages from the Kafka queue in real time. After consuming the message, the dam safety monitoring system parses the task name in the strong earthquake message and matches it with the corresponding task folder in the local Earthquake folder. This task folder contains all icons for executing the call task. The icons provide positioning information during the automated execution of the call task. The specific steps are as follows: (3.1) Set the producer: Configure the Flink end as the producer, use the data after Flink fusion as the data produced by the producer, and store the data in the strong earthquake topic specified by the message queue; (3.2) Set up consumers: Establish a connection with the dam safety monitoring system server through the Kafka client API and subscribe to the earthquake message topic. At this time, the dam safety monitoring system monitors and receives strong earthquake messages published in the Kafka message queue in real time; (3.3) Message parsing and task matching: As a consumer, the dam safety monitoring system receives new strong earthquake data from Kafka, parses the message content in JSON format, and extracts the task name. This task name will be matched with the task folder name under the local Earthquake folder to obtain the icon that needs to execute the task call to provide positioning information.

4. The multi-system linkage method for dam safety monitoring under emergency conditions according to claim 1 is characterized in that: (4) During the automatic execution of the dam safety monitoring system call task, pyautogui is used to intercept the current operating system interface, and Open CV is used to verify whether the current page contains a pre-stored call task icon; If the match is successful, the system will automatically locate and click the icon containing the pre-stored call task, trigger the dam monitoring task and automatically collect relevant data for subsequent analysis; the specific steps are as follows: (4.1) Screen capture and image processing: The system captures the current interface at set intervals as the target image based on the Open CV template matching method. There are two situations: one is to capture the main interface without the taskbar, which is used as the target image when the dam safety monitoring system is not started or performing a call task; the other is to capture the taskbar part, which is suitable for when the dam safety monitoring system is started and minimized to the taskbar as the target image; (4.1.1) System main interface capture: Use the screenshot() method of the pyautogui library to obtain a screenshot of the system main interface as the target image, which is suitable for identification operations when the dam safety monitoring system is not started or performing a call task; (4.1.2) System taskbar capture: Use the screenshot() method of the pyautogui library to obtain a screenshot of the system taskbar as the target image, which is suitable for identification operations when the dam safety monitoring system has been started and minimized to the taskbar; (4.2) Standard correlation matching based on Open CV: Use standard correlation matching based on Open CV to find the position of the template image in the target image, and calculate the position of the task icons in the main interface in the order of the task calls; (4.3) Automatic triggering of the call task: Using the position of the task icon on the screen obtained in step (4.2), this position is converted into the position of screen pixels, and the doubleClick() method in the pyautogui library is used to automatically execute the call task in sequence, so that relevant personnel can conduct real-time analysis and comprehensive evaluation of the dam safety status.

5. The multi-system linkage method for dam safety monitoring under emergency conditions according to claim 1 is characterized in that: The method is applicable to multi-system linkage solutions in various emergency conditions.

6. A multi-system linkage device for dam safety monitoring under earthquake conditions, characterized in that: include: Linkage data acquisition module: Utilize multiple seismic sensors to automatically collect vibration data in real time, and store the collected data in the event folder in .evt format; Use pywin32-based automated scripts to monitor newly added event files in the folder, extract and store key acceleration data in the database for analysis; Linkage data real-time monitoring module: Use the Flink CDC tool to process and analyze the seismic characteristic parameters of the linkage data acquisition module in real time; Abnormal data publishing module: The dam safety monitoring system is configured as a consumer of the Kafka message queue to receive earthquake messages in real time; the system parses each message to obtain the task name and matches it with the corresponding task folder in the local Earthquake folder; the folder contains the icon of the call task, which provides positioning information for the automated execution of the call task; Automatic execution module for the call task of the dam safety monitoring system: Use the pyautogui library to take a screenshot of the operating system interface, then use Open CV to process the image, and use an algorithm to compare the real-time screenshot with the task icon in the Earthquake folder to see if it matches. After a successful match, determine the screen position of the icon and convert it into a pixel position; pyautogui automatically clicks based on this position to trigger the call task of the dam safety monitoring system and automatically collects data for safety analysis; In the linkage data real-time monitoring module, the Flink CDC tool is used to process and analyze earthquake characteristic parameters in real time. The specific steps are as follows: (2.1) Data capture and synchronization: For the newly added earthquake characteristic parameters in the database table, use the Flink CDC tool to monitor the changes in the database table in real time; (2.2) Set a time window to collect the earthquake characteristic parameters of all measuring points within the same earthquake occurrence time for calculation and fusion; (2.2.1) Setting the size of the time window: Through simulation experiments, the time when all measurement points are inserted into the database and monitored by Flink when an earthquake occurs is simulated, and this time is set as the time window size; (2.2.2) Fusion of strong earthquake data from multiple measuring points: Set the earthquake occurrence timestamp as the window grouping basis, so that the strong earthquake data collected by different measuring points at the same time will enter the same time window, calculate the peak acceleration based on the earthquake characteristic parameters, and record the maximum value of the peak acceleration and the number of measuring points that exceed the threshold for triggering the call for measurement task; (2.2.3) Data transmission trigger condition setting: When the number of measurement points whose statistics are greater than the trigger threshold exceeds half of the total number of measurement points, the information of the measurement point corresponding to the maximum value of the peak acceleration is transmitted to the Kafka message queue.

7. A computer device, characterized in that: The computer device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, a multi-system linkage method for dam safety monitoring under emergency conditions as described in any one of claims 1 to 4 is implemented.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program for executing the multi-system linkage method for dam safety monitoring under emergency conditions as described in any one of claims 1-4.

Images