Method for acquiring rainwater condition monitoring station data in area based on message queue
By creating a rainwater monitoring site information table and traffic information table in the ORACLE database, and putting the data into the Kafka message queue for prediction, the problem of real-time display and prediction of rainwater monitoring data in the existing technology is solved, and the data is accurate and Timely access to support more effective flood control and water resource management.
Patent Information
- Application Number
- CN202411049458.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-01
- Publication Date
- 2024-11-22
AI Technical Summary
The existing technology is difficult to realize real-time display and prediction of rainwater monitoring data, resulting in the inability to effectively support flood control, drought resistance and water resource management.
By creating a rainwater monitoring site information table and traffic information table in the ORACLE database, collecting rainwater situation data and putting it into the Kafka message queue, using time series analysis model or statistical regression model to predict the water level change trend in each site area, and performing Visual display.
Real-time acquisition and accurate prediction of rainwater situation data is realized, and decision-making basis is provided, which avoids the problem of high concurrent data writing, ensuring the accuracy and timeliness of data.
Smart Images

Figure CN119003634A_ABST
Abstract
Description
Technical Field
[0001] The invention discloses a method, which relates to the technical field of data collection, and in particular to a method for acquiring data of rainwater monitoring sites in a region based on a message queue. Background Art
[0002] For flood prevention, drought resistance, water resources management and ecological environment protection, it is necessary to monitor and report rainfall conditions to obtain rainfall conditions in the region. However, the current method of collecting rainfall conditions by manually aggregating data from various sites is usually used, which is not conducive to real-time display and prediction of rainfall conditions.
[0003] Kafka is an open source stream processing platform developed by the Apache Software Foundation and written in Scala and Java. Kafka is a high-throughput distributed publish-subscribe messaging system that can handle all action stream data of consumers on the website. Kafka can unify online and offline message processing through Hadoop's parallel loading mechanism. Summary of the invention
[0004] In view of the problems of the prior art, the present invention provides a method for obtaining data of rain and water condition monitoring stations in a region based on a message queue, and automatically provides and predicts data of rain and water condition monitoring stations in the region.
[0005] The specific scheme proposed by the present invention is:
[0006] The present invention provides a method for obtaining data of rainwater monitoring sites in a region based on a message queue, comprising:
[0007] Step 1: Create a rain monitoring site information table in the ORACLE database, and record the rain monitoring site number, longitude, latitude, detailed address, and river or reservoir of the rain monitoring site through the rain monitoring site information table.
[0008] And create a flow information table in the ORACLE database, and record the reporting time, rain monitoring point number, station water level and gradient rainfall through the flow information table.
[0009] Step 2: Collect the rainfall data obtained by the rainfall monitoring site according to the rainfall monitoring site information table and flow information table, and put the rainfall data into the Kafka message queue.
[0010] Save the rainfall data in the Kafka message queue to the ORACLE database in batches.
[0011] Step 3: Call the rainfall data in the ORACLE database for visualization, and use the time series analysis model or statistical regression model to predict the water level change trend of the river or reservoir in each station area based on the rainfall data.
[0012] Furthermore, in the method for obtaining rain monitoring site data in a region based on a message queue, after initializing the rain monitoring site information table and the flow information table in step 1, the data of the rain monitoring site information table and the flow information table are regularly maintained, and the rain monitoring site information table and the flow information table are adjusted according to the increase or decrease of rain monitoring sites.
[0013] Furthermore, in the method for obtaining regional rainfall monitoring site data based on a message queue, a timer is set in step 2 to regularly check the rainfall data in the Kafka message queue, and save the rainfall data in the Kafka message queue in batches to the ORACLE database according to a preset time.
[0014] Furthermore, in the method for obtaining data of rain monitoring sites in a region based on a message queue, step 3 calls the rain data in an ORACLE database for visual display, including: forming a rainfall and water level change curve and a bar chart of the rain data according to the time period.
[0015] Furthermore, in the method of obtaining regional rainfall monitoring station data based on a message queue, when predicting the water level change trend of the river or reservoir in each station area in step 3, the water level change trend is predicted according to the dimensions of minutes, 10 minutes, 30 minutes, and hours.
[0016] The present invention provides a device for acquiring data of rainwater monitoring sites in a region based on a message queue, comprising a database table management module, a data management module and a data display and prediction module.
[0017] The database table management module creates a rain monitoring site information table in the ORACLE database, and records the rain monitoring site number, longitude, latitude, detailed address, and river or reservoir of the rain monitoring site through the rain monitoring site information table.
[0018] And create a flow information table in the ORACLE database, and record the reporting time, rain monitoring point number, station water level and gradient rainfall through the flow information table.
[0019] The data management module collects the rainfall data obtained by the rainfall monitoring site according to the rainfall monitoring site information table and flow information table, and puts the rainfall data into the Kafka message queue.
[0020] Save the rainfall data in the Kafka message queue to the ORACLE database in batches.
[0021] The data display and prediction module calls the rainfall data in the ORACLE database for visual display, and uses the time series analysis model or statistical regression model to predict the water level change trend of the river or reservoir in each station area based on the rainfall data.
[0022] Furthermore, after the database table management module of the device for obtaining the data of rain monitoring sites in the area based on the message queue initializes the rain monitoring site information table and the flow information table, it regularly maintains the data of the rain monitoring site information table and the flow information table, and adjusts the rain monitoring site information table and the flow information table according to the increase or decrease of rain monitoring sites.
[0023] Furthermore, the data management module of the device for obtaining rainfall monitoring site data in a region based on a message queue sets a timer, regularly checks the rainfall data in the Kafka message queue, and saves the rainfall data in the Kafka message queue in batches to the ORACLE database according to a preset time.
[0024] Furthermore, the data display and prediction module of the device for obtaining data from rain monitoring sites in a region based on a message queue calls the rain data in an ORACLE database for visual display, including: forming a rainfall and water level change curve and a bar chart based on the time period.
[0025] Furthermore, when the data display and prediction module of the device that obtains the data of rainfall monitoring stations in the area based on the message queue predicts the water level change trend of the river or reservoir in each station area, the water level change trend is predicted according to the dimensions of minute, 10 minute, 30 minute and hour.
[0026] The benefits of the present invention are:
[0027] 1. Obtain rainfall data for each time period for review by relevant departments and provide a basis for decision-making;
[0028] 2. Use message queues to store rainfall data, avoiding high-concurrency data writing into the ORACLE database;
[0029] 3. Ensure the accuracy and timeliness of data. BRIEF DESCRIPTION OF THE DRAWINGS
[0030] Figure 1 It is a schematic diagram of the application architecture of the method of the present invention. DETAILED DESCRIPTION
[0031] The present invention is further described below in conjunction with the accompanying drawings and specific embodiments so that those skilled in the art can better understand the present invention and implement it, but the embodiments are not intended to limit the present invention.
[0032] Example 1
[0033] The present invention provides a method for obtaining data of rainwater monitoring sites in a region based on a message queue, comprising:
[0034] Step 1: Create a rain monitoring site information table in the ORACLE database, and record the rain monitoring site number, longitude, latitude, detailed address, and river or reservoir of the rain monitoring site through the rain monitoring site information table.
[0035] A flow information table is created in the ORACLE database to record the reporting time, rainfall monitoring point number, site water level and gradient rainfall.
[0036] For the rain monitoring station information table, please refer to Table 1, and for the flow information table, please refer to Table 2.
[0037] Table 1
[0038]
[0039] Table 2
[0040]
[0041] Step 2: Collect the rainfall data obtained by the rainfall monitoring site according to the rainfall monitoring site information table and flow information table. The back-end platform puts the rainfall data into the Kafka message queue. The back-end platform can set a timer to check the rainfall data in the Kafka message queue at regular intervals, and save the rainfall data in the Kafka message queue in batches to the ORACLE database according to the preset time.
[0042] Step 3: Call the rainfall data in the ORACLE database for visualization, and use the time series analysis model or statistical regression model to predict the water level change trend of the river or reservoir in each station area based on the rainfall data.
[0043] The method of the present invention mainly obtains precipitation and water level data in real time through rain and water monitoring points, sends them through message queues, and finally forms water level change trend data of various rivers and reservoirs in the region. After data aggregation and sorting, multi-dimensional water level trend data can be provided, which can more accurately grasp the weather conditions in the future and possible rainstorms and floods, so as to make correct and effective defense measures. At the same time, it also provides some reference basis for decision-making. Use this information to formulate reasonable defense measures to reduce losses, casualties or property losses.
[0044] Example 2
[0045] Based on Example 1, after further initializing the rain monitoring site information table and the flow information table in step 1 of the method of the present invention, the data of the rain monitoring site information table and the flow information table can be regularly maintained, and the rain monitoring site information table and the flow information table can be adjusted according to the increase or decrease of rain monitoring sites.
[0046] In step 3 of the method of the present invention, the rainfall data in the ORACLE database can be further called for visual display, including: according to the time period, the rainfall data is formed into a rainfall and water level change curve and a bar chart, etc., and displayed on the prevention and control command screen,
[0047] When predicting the water level change trend of the river or reservoir in each station area in step 3, the water level change trend is predicted according to the minute, 10-minute, 30-minute and hourly dimensions.
[0048] Example 3
[0049] The present invention also provides a device for acquiring data of rainwater monitoring sites in a region based on a message queue, including a database table management module, a data management module and a data display and prediction module.
[0050] The database table management module creates a rain monitoring site information table in the ORACLE database, and records the rain monitoring site number, longitude, latitude, detailed address, and river or reservoir of the rain monitoring site through the rain monitoring site information table.
[0051] And create a flow information table in the ORACLE database, and record the reporting time, rain monitoring point number, station water level and gradient rainfall through the flow information table.
[0052] The data management module collects the rainfall data obtained by the rainfall monitoring site according to the rainfall monitoring site information table and flow information table, and puts the rainfall data into the Kafka message queue.
[0053] Save the rainfall data in the Kafka message queue to the ORACLE database in batches.
[0054] The data display and prediction module calls the rainfall data in the ORACLE database for visual display, and uses the time series analysis model or statistical regression model to predict the water level change trend of the river or reservoir in each station area based on the rainfall data.
[0055] As the information interaction, execution process and other contents between the modules in the above-mentioned device are based on the same concept as the embodiment of the method of the present invention, the specific contents can be found in the description of the embodiment of the method of the present invention and will not be repeated here.
[0056] Likewise, the benefits of the device of the present invention are:
[0057] 1. Obtain rainfall data for each time period for review by relevant departments and provide a basis for decision-making;
[0058] 2. Use message queues to store rainfall data, avoiding high-concurrency data writing into the ORACLE database;
[0059] 3. Ensure the accuracy and timeliness of data.
[0060] It should be noted that not all steps and modules in the above-mentioned processes and device structures are necessary, and some steps or modules can be ignored according to actual needs. The execution order of each step is not fixed and can be adjusted as needed. The system structure described in the above-mentioned embodiments can be a physical structure or a logical structure, that is, some modules may be implemented by the same physical entity, or some modules may be implemented by multiple physical entities, or some components in multiple independent devices may be implemented together.
[0061] The above-described embodiments are only preferred embodiments for fully illustrating the present invention, and the protection scope of the present invention is not limited thereto. Equivalent substitutions or changes made by those skilled in the art based on the present invention are within the protection scope of the present invention. The protection scope of the present invention shall be subject to the claims.
Claims
1. A method for obtaining data of rainwater monitoring stations in a region based on a message queue, characterized in that include: Step 1: Create a rain monitoring site information table in the ORACLE database, and record the rain monitoring site number, longitude, latitude, detailed address, and river or reservoir of the rain monitoring site through the rain monitoring site information table. And create a flow information table in the ORACLE database, and record the reporting time, rain monitoring point number, station water level and gradient rainfall through the flow information table. Step 2: Collect the rainfall data obtained by the rainfall monitoring site according to the rainfall monitoring site information table and flow information table, and put the rainfall data into the Kafka message queue. Save the rainfall data in the Kafka message queue to the ORACLE database in batches. Step 3: Call the rainfall data in the ORACLE database for visualization, and use the time series analysis model or statistical regression model to predict the water level change trend of the river or reservoir in each station area based on the rainfall data.
2. According to a method for obtaining regional rain and water condition monitoring site data based on a message queue as described in claim 1, it is characterized in that after initializing the rain and water condition monitoring site information table and the flow information table in step 1, the data of the rain and water condition monitoring site information table and the flow information table are regularly maintained, and the rain and water condition monitoring site information table and the flow information table are adjusted according to the increase or decrease of rain and water condition monitoring sites.
3. The method for obtaining rainwater monitoring station data in a region based on a message queue according to claim 1 is characterized in that In step 2, a timer is set to regularly check the rainfall data in the Kafka message queue, and the rainfall data in the Kafka message queue is saved in batches to the ORACLE database according to the preset time.
4. The method for obtaining rainwater monitoring station data in a region based on a message queue according to claim 1 is characterized in that In step 3, the rainfall data in the ORACLE database is called for visual display, including: forming a rainfall and water level change curve and a bar chart based on the time period.
5. A method for obtaining rainwater monitoring station data in a region based on a message queue according to claim 1 or 4, characterized in that When predicting the water level change trend of the river or reservoir in each site area in step 3, the water level change trend is predicted according to the dimensions of minute level, 10 minute level, 30 minute level and hour level.
6. A device for obtaining data of rain monitoring stations in a region based on a message queue, characterized in that Including database table management module, data management module and data display and prediction module, The database table management module creates a rain monitoring site information table in the ORACLE database, and records the rain monitoring site number, longitude, latitude, detailed address, and river or reservoir of the rain monitoring site through the rain monitoring site information table. And create a flow information table in the ORACLE database, and record the reporting time, rain monitoring point number, station water level and gradient rainfall through the flow information table. The data management module collects the rainfall data obtained by the rainfall monitoring site according to the rainfall monitoring site information table and flow information table, and puts the rainfall data into the Kafka message queue. Save the rainfall data in the Kafka message queue to the ORACLE database in batches. The data display and prediction module calls the rainfall data in the ORACLE database for visual display, and uses the time series analysis model or statistical regression model to predict the water level change trend of the river or reservoir in each station area based on the rainfall data.
7. The device for acquiring data of rainwater monitoring stations in a region based on a message queue according to claim 6 is characterized in that After the database table management module initializes the rain monitoring site information table and the flow information table, it regularly maintains the data of the rain monitoring site information table and the flow information table, and adjusts the rain monitoring site information table and the flow information table according to the increase or decrease of rain monitoring sites.
8. The device for acquiring data of rainwater monitoring stations in a region based on a message queue according to claim 6 is characterized in that The data management module sets a timer to check the rainfall data in the Kafka message queue at regular intervals, and saves the rainfall data in the Kafka message queue to the ORACLE database in batches according to the preset time.
9. The device for acquiring data of rainwater monitoring stations in a region based on a message queue according to claim 6 is characterized in that The data display and prediction module calls the rainfall data in the ORACLE database for visual display, including: forming rainfall and water level change curves and bar charts based on time periods.
10. The device for acquiring data of rainwater monitoring stations in a region based on a message queue according to claim 6 or 9, characterized in that The data display and prediction module predicts the water level change trend of the river or reservoir in each station area according to the minute, 10-minute, 30-minute and hourly dimensions.
Citation Information
Patent Citations
Flood prevention rainwater information automatic monitoring central system and monitoring method thereof
CN104103153A
Drought early warning method and system based on reservoir level and rainfall
CN115422252A
Water conservancy data acquisition method based on water conservancy Internet of Things
CN115550444A
Coal mine disaster data acquisition, analysis and visual display system
CN116624226A
