Reservoir area floating object monitoring method, device and equipment based on multi-source data fusion, storage medium and product

Abstract

The invention provides a multi-source data fusion-based reservoir floating object monitoring method, device and equipment, a storage medium and a product, relates to the technical field of environment monitoring, and aims to solve the problems of low efficiency, limited coverage range and insufficient data precision in related technologies of reservoir floating object monitoring. The method comprises the following steps: acquiring various data; the multiple kinds of data comprise multispectral satellite remote sensing data, unmanned aerial vehicle aerial photography data, ground sensor data and meteorological data; performing data fusion on the various data to obtain fused data; the data fusion comprises space fusion, time fusion and feature extraction and fusion; inputting the fusion data into a floating object identification model to identify the floating object, obtaining a monitoring result, and carrying out data analysis; the data analysis result comprises a floating object distribution diagram, statistical data and a change trend diagram; and judging whether the total number of the floating objects in the statistical data exceeds a preset threshold, and outputting early warning information if the total number of the floating objects in the statistical data exceeds the preset threshold.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of environmental monitoring technology, and in particular to a method, device, equipment, storage medium and product for monitoring floating objects in a reservoir area based on multi-source data fusion. Background Art

[0002] Monitoring floating objects in reservoir areas is an important part of environmental protection and water resources management. It plays an important role in maintaining the ecological environment of water bodies, ensuring water safety and effectively managing water resources. Traditional monitoring methods mainly rely on manual inspections and single sensor data, which have many problems such as low efficiency, limited coverage and insufficient data accuracy. Summary of the invention

[0003] The embodiments of the present application provide a method, device, equipment, storage medium and product for monitoring floating objects in a reservoir area based on multi-source data fusion, aiming to solve the problems of low efficiency, limited coverage and insufficient data accuracy in related technologies for monitoring floating objects in a reservoir area.

[0004] A first aspect of an embodiment of the present application provides a method for monitoring floating objects in a reservoir area based on multi-source data fusion, comprising:

[0005] Acquire multiple data in the target area and target time period; the multiple data include: multi-spectral satellite remote sensing data, drone aerial photography data, ground sensor data and meteorological data; the drone aerial photography data is obtained by photographing floating objects in the target area; the ground sensor data include: water quality parameters, the number and density of floating objects;

[0006] Performing data fusion on the multiple data to obtain fused data; the data fusion includes: spatial fusion, temporal fusion and feature extraction and fusion;

[0007] Input the fused data into a floating object recognition model to identify the floating objects and obtain monitoring results; the monitoring results include: the location and type of the floating objects at each time point within the target time period;

[0008] According to the monitoring results, data analysis is performed to obtain data analysis results; and the data analysis results are displayed; the data analysis results include: a floating object distribution map, statistical data and a change trend map; the statistical data include: the number of each type of floating objects existing in the target area and the total number of floating objects within the target time period;

[0009] Determine whether the total number of the floating objects exceeds a preset threshold; if the total number of the floating objects exceeds the preset threshold, output warning information.

[0010] In an optional implementation manner, the performing data fusion on the multiple data to obtain fused data includes:

[0011] Performing spatial fusion on the multiple data to obtain a first data set;

[0012] Performing time fusion on the first data set to obtain a second data set;

[0013] Feature extraction and fusion are performed on the second data set to obtain the fused data.

[0014] In an optional implementation, the spatially fusing the multiple data to obtain the first data set includes:

[0015] Using an image registration method to spatially align the multiple data to obtain spatially aligned data, the image registration method comprising: feature point matching and interpolation;

[0016] The aligned data are spatially fused using a data fusion method to obtain the first data set, wherein the data fusion method includes: weighted average method and minimum variance fusion.

[0017] In an optional implementation, performing time fusion on the first data set to obtain the second data set includes:

[0018] Using an interpolation method or a resampling method to time-align the data in the first data set to obtain time-aligned data;

[0019] Performing time series analysis on the time-aligned data to obtain analyzed data;

[0020] The analyzed data are temporally fused using a spatiotemporal autoregressive model and a dynamic Bayesian network to obtain the second data set.

[0021] In an optional implementation, the extracting and fusing the features of the spatial data set to obtain the fused data includes:

[0022] Extracting characteristic information of the data in the second data set;

[0023] Performing feature selection on the feature information to obtain multiple key features;

[0024] The plurality of key features are spatially fused using feature integration and feature-level splicing to obtain the fused data.

[0025] In an optional implementation, after acquiring the multiple data, the method further includes: preprocessing the multiple data; the preprocessing includes: format conversion and noise removal.

[0026] A second aspect of the embodiment of the present application provides a reservoir area floating object monitoring device based on multi-source data fusion, comprising:

[0027] Acquisition module: used to acquire multiple data in the target area and target time period; the multiple data include: multi-spectral satellite remote sensing data, drone aerial photography data, ground sensor data and meteorological data; the drone aerial photography data is obtained by photographing floating objects in the target area; the ground sensor data include: water quality parameters, the number and density of floating objects;

[0028] Data fusion module: used for fusing the multiple data to obtain fused data; the data fusion includes: spatial fusion, temporal fusion and feature extraction and fusion;

[0029] Identification processing module: used for inputting the fusion data into the floating object identification model to identify the floating objects and obtain monitoring results; the monitoring results include: the location and type of the floating objects at each time point within the target time period;

[0030] Analysis and display module: used to perform data analysis according to the monitoring results to obtain data analysis results; and to display according to the data analysis results; the data analysis results include: floating object distribution map, statistical data and change trend map; the statistical data include: the number of each type of floating objects existing in the target area and the target time period and the total number of floating objects;

[0031] Alarm module: used to determine whether the total number of the floating objects exceeds the preset threshold; if the total number of the floating objects exceeds the preset threshold, output warning information.

[0032] In an optional implementation, the data fusion module includes:

[0033] A spatial fusion module, used for performing spatial fusion on the multiple data to obtain a first data set;

[0034] A time fusion module, used for performing time fusion on the first data set to obtain a second data set;

[0035] The feature extraction and fusion module is used to extract and fuse the features of the second data set to obtain the fused data.

[0036] In an optional implementation, the spatial fusion module is specifically used to:

[0037] Using an image registration method to spatially align the multiple data to obtain spatially aligned data, the image registration method comprising: feature point matching and interpolation;

[0038] The aligned data are spatially fused using a data fusion method to obtain the first data set, wherein the data fusion method includes: weighted average method and minimum variance fusion.

[0039] In an optional implementation, the time fusion module is specifically used to:

[0040] Using an interpolation method or a resampling method to time-align the data in the first data set to obtain time-aligned data;

[0041] Performing time series analysis on the time-aligned data to obtain analyzed data;

[0042] The analyzed data are temporally fused using a spatiotemporal autoregressive model and a dynamic Bayesian network to obtain the second data set.

[0043] In an optional implementation, the feature extraction and fusion module is specifically used for:

[0044] Extracting characteristic information of the data in the second data set;

[0045] Performing feature selection on the feature information to obtain multiple key features;

[0046] The plurality of key features are spatially fused using feature integration and feature-level splicing to obtain the fused data.

[0047] In an optional embodiment, the device further comprises:

[0048] The preprocessing module is used to preprocess the multiple data; the preprocessing includes: format conversion and noise removal.

[0049] According to a third aspect of an embodiment of the present application, an electronic device is provided, including: a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of the method for monitoring floating objects in a reservoir area based on multi-source data fusion according to the first aspect.

[0050] According to a fourth aspect of the embodiments of the present application, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps of the method for monitoring floating objects in a reservoir area based on multi-source data fusion are implemented as described in the first aspect.

[0051] According to a fifth aspect of the embodiments of the present application, a computer program product is provided, including a computer program, which, when executed by a processor, implements the steps of the method for monitoring floating objects in a reservoir area based on multi-source data fusion according to the first aspect.

[0052] In the disclosed embodiment, by acquiring multiple floating object data, spatial fusion, temporal fusion, feature extraction and fusion are performed, and after obtaining the fused data, the floating object recognition model is input to identify the floating objects and obtain the monitoring results; data analysis is performed based on the monitoring results to obtain the floating object distribution map, statistical data and change trend map; it is determined whether the total number of floating objects in the statistical data exceeds the preset threshold, and if it exceeds the preset threshold, an early warning message is output. Comprehensive, efficient and accurate monitoring of floating objects in the reservoir area is achieved, and the multiple fusions of data help to improve the accuracy of floating object identification, and it has a real-time early warning function, which can timely clean up floating objects in the water area and promote the sustainable development of the aquatic ecosystem. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the description of the embodiments of the present application will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0054] Figure 1 It is a flowchart of the steps of a reservoir area floating object monitoring method based on multi-source data fusion proposed in an embodiment of the present application;

[0055] Figure 2 It is a structural schematic diagram of a reservoir area floating object monitoring device based on multi-source data fusion proposed in one embodiment of the present application;

[0056] Figure 3 is a schematic diagram of an electronic device in the present application. DETAILED DESCRIPTION

[0057] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0058] In the drawings, the size of the constituent elements, the thickness of the layer or the area may be exaggerated for the sake of clarity. Therefore, any implementation of the present disclosure is not necessarily limited to the size shown in the drawings, and the shapes and sizes of the components in the drawings do not reflect the true proportions. In addition, the drawings schematically show ideal examples, and any implementation of the present disclosure is not limited to the shapes or values ​​shown in the drawings.

[0059] Monitoring floating objects in reservoir areas is an important part of environmental protection and water resources management. Traditional monitoring methods mainly rely on manual inspections and single sensor data, which have problems such as low efficiency, limited coverage, and insufficient data accuracy. With the development of sensor technology and data processing technology, multi-source data fusion has become an effective way to solve these problems. Multi-source data fusion can improve monitoring accuracy, expand monitoring range, and realize real-time monitoring and dynamic early warning by integrating multiple sensor data.

[0060] Reference Figure 1 , Figure 1 1 is a flowchart of the steps of the reservoir area floating object monitoring method based on multi-source data fusion proposed in Example 1 of the present application. Figure 1 As shown, the method includes the following steps S11 to S15:

[0061] Step S11: Acquire multiple data in the target area and target time period; the multiple data include: multi-spectral satellite remote sensing data, drone aerial photography data, ground sensor data and meteorological data; the drone aerial photography data is obtained by photographing floating objects in the target area; the ground sensor data include: water quality parameters, the number and density of floating objects;

[0062] Multispectral satellite remote sensing data: Select satellites equipped with multispectral imagers, such as Landsat series, Sentinel series, etc. Plan satellite observation tasks based on parameters such as the target area range and observation time period range. After the satellite completes the observation, the multispectral satellite remote sensing data is received through the ground receiving station.

[0063] Drone aerial photography data: Use a drone equipped with a high-definition camera and a stable gimbal to take aerial photos of the target area, capture images or videos of floating objects, and obtain drone aerial photography data of the floating objects after processing by the graphics processor inside the drone.

[0064] Ground sensor data: Sensors are deployed in the waters or surrounding areas of the target area, such as water quality monitoring instruments, flow meters, particulate matter sensors, etc. The sensors automatically collect ground sensor data such as water quality parameters, the number and density of floating objects at a set frequency.

[0065] Meteorological data: Meteorological data of the target area can be obtained from the Meteorological Bureau, meteorological satellites, meteorological stations and other channels, including temperature, humidity, wind speed, wind direction, precipitation and other information.

[0066] Obtain a variety of data in the target area within the target time period, including multi-spectral satellite remote sensing data, drone aerial photography data, ground sensor data and meteorological data. The target area is mainly the reservoir area where floating objects need to be monitored, and the target time period can be selected according to needs. The comprehensive application of multiple data sources has a wide coverage of floating object monitoring, and can provide more comprehensive and accurate floating object information in the target area within the target time period, providing a solid foundation for subsequent analysis.

[0067] Furthermore, after acquiring the multiple data, the method further includes: preprocessing the multiple data; the preprocessing includes: format conversion and noise removal.

[0068] After acquiring a variety of data, preprocessing is performed to help improve the quality of the data and the accuracy of subsequent analysis. Preprocessing usually includes format conversion and noise removal. Format conversion: converting data acquired from different data sources into the same format for subsequent unified processing and analysis. For example, converting multispectral satellite remote sensing data and drone aerial photography data into common image formats (such as JPEG or TIFF), and converting ground sensor data into standard data file formats (such as CSV or Excel). Noise removal: processing and removing noise in the data to improve data quality and accuracy. Noise may come from interference during data acquisition, sensor errors, atmospheric interference, and other reasons. Common noise removal methods include filtering, interpolation, smoothing, and so on.

[0069] Step S12: performing data fusion on the multiple data to obtain fused data; the data fusion includes: spatial fusion, temporal fusion and feature extraction and fusion.

[0070] Data fusion is performed on the acquired data (including multispectral satellite remote sensing data, drone aerial data, ground sensor data and meteorological data), and data from different sources are integrated and processed to provide more comprehensive, diverse and accurate data and enhance the availability and operability of data. Data fusion specifically includes: spatial fusion, temporal fusion and feature extraction and fusion.

[0071] The data fusion process is usually carried out in the order of space, time, and features. First, spatial fusion is used to align data from different geographic sources. Then, temporal fusion integrates and analyzes data at different times. Finally, feature extraction and fusion extract and merge key feature information on a unified data basis. Spatial and temporal fusion provides a reliable data basis for feature extraction, allowing feature fusion to more accurately capture the physical characteristics and changing trends of floating objects, ultimately improving the overall monitoring accuracy. They work together to provide comprehensive and accurate floating object monitoring results.

[0072] Furthermore, the performing data fusion on the multiple data to obtain fused data includes:

[0073] Performing spatial fusion on the multiple data to obtain a first data set;

[0074] Performing time fusion on the first data set to obtain a second data set;

[0075] Feature extraction and fusion are performed on the second data set to obtain the fused data.

[0076] The first data set is obtained by spatially fusing multiple data and aligning data from different geographic sources to ensure accurate matching of various data in geographic space. This enables information from different data sources to be compared and analyzed in the same geographic framework, thereby improving the accuracy and reliability of floating object monitoring.

[0077] The first data set is temporally fused, the data obtained at different time points in the first data set are unified to the same time base, and then time series analysis is performed to analyze the correlation between data at different time points to obtain the second data set. The changing trend of floating objects, such as the movement trajectory and diffusion speed of floating objects, can be observed and analyzed, providing information of the time dimension for the prediction and decision-making of floating objects.

[0078] The obtained second data set is subjected to feature extraction and fusion, focusing on the integration of the intrinsic information of the data, and then extracting the features useful for monitoring floating objects from the data and performing fusion processing. These feature information can be the color, texture, shape and other features of floating objects, as well as the physical properties of floating objects (such as density, volume, etc.), and meteorological factors that affect the diffusion of floating objects (such as wind speed, wind direction, etc.). It can conduct comprehensive analysis from multiple dimensions and multiple levels, improving the accuracy and comprehensiveness of monitoring.

[0079] Furthermore, the spatial fusion of the multiple data to obtain a first data set includes:

[0080] Using an image registration method to spatially align the multiple data to obtain spatially aligned data, the image registration method comprising: feature point matching and interpolation;

[0081] The aligned data are spatially fused using a data fusion method to obtain the first data set, wherein the data fusion method includes: weighted average method and minimum variance fusion.

[0082] Use image registration methods to spatially align multiple data to obtain spatially aligned data, ensuring that multi-source data can be compared and analyzed in the same spatial coordinate system. Image registration methods include at least: feature point matching and interpolation methods. Feature point matching: For image data (such as multispectral satellite images and drone images), feature point matching techniques such as SIFT (scale-invariant feature transform) and SURF (speeded up robust features) are often used to find and match feature points in two images, thereby achieving image alignment. Interpolation method: For ground sensor data, such as point data (such as GPS measurement points, meteorological station observation data, etc.), point data can be mapped to raster data through interpolation methods (such as Kriging interpolation, inverse distance weighted method, etc.) to achieve spatial alignment with image data.

[0083] The data fusion method is to integrate multi-source spatial data according to certain rules or algorithms to obtain more comprehensive and accurate information. Data fusion methods include at least: weighted average method and minimum variance fusion. Weighted average method: assign weights according to the accuracy and confidence of different data sources, and then perform weighted average on the data values ​​of each data source. Minimum variance fusion: fuse multi-source data by minimizing the error between the fused data and the real data, establish an error model, and use an optimization algorithm to solve the optimal fusion parameters. Through registration and fusion technology, the inconsistency between multi-source data can be eliminated and the accuracy and reliability of the data can be improved.

[0084] Further, performing time fusion on the first data set to obtain a second data set includes:

[0085] Using an interpolation method or a resampling method to time-align the data in the first data set to obtain time-aligned data;

[0086] Performing time series analysis on the time-aligned data to obtain analyzed data;

[0087] The analyzed data are temporally fused using a spatiotemporal autoregressive model and a dynamic Bayesian network to obtain the second data set.

[0088] Time alignment refers to adjusting the data at different time points in the first data set to a unified time scale, which can be done using interpolation or resampling methods. Since different data sources may have different acquisition frequencies and timestamps, time alignment is the first step in time fusion. Interpolation methods: For data points with large time intervals, interpolation methods (such as linear interpolation, polynomial interpolation, spline interpolation, etc.) can be used to estimate the data values ​​of the missing time points, thereby generating a continuous and smooth time series so that the data is consistent in time. Resampling methods: If the time resolution of the first data set does not match the time resolution required for subsequent analysis, resampling methods can be used to adjust the sampling rate of the data. Resampling may include downsampling (reducing the number of data points to reduce the time resolution) or upsampling (increasing the number of data points to increase the time resolution) to align the data in time with the target analysis framework.

[0089] Perform time series analysis on the time-aligned data. Time series analysis aims to reveal the laws, trends, periodicity and randomness of data changes over time. This usually involves Kalman filtering and particle filtering techniques to smooth data and remove noise. Kalman filtering: Applicable to time series data of linear systems. It estimates the state of the system in a recursive manner. It combines prediction and observation information to minimize the variance of the estimation error. It can not only smooth data but also make predictions. Particle filtering: Applicable to time series data of nonlinear and non-Gaussian systems. It approximates the posterior probability distribution of the system through a set of weighted particles. Each particle represents a possible state of the system. As the observation data arrives, the particle filter will update the weight and position of the particle to approximate the true system state.

[0090] The analyzed data are temporally fused using the spatiotemporal autoregressive model and the dynamic Bayesian network to obtain the second data set. Spatiotemporal autoregressive model: Combining the characteristics of spatial autoregression and temporal autoregression, the spatial and temporal correlation of data can be considered at the same time. In floating object monitoring, the spatiotemporal autoregressive model can capture the spatial distribution pattern of floating objects and their temporal changing trends. Dynamic Bayesian network: The dynamic Bayesian network is a probabilistic graphical model for representing and reasoning about time-varying systems. It can describe the changes of system states over time and the relationship between observed data and states. In floating object monitoring, a dynamic Bayesian network can be constructed to simulate the dynamic behavior of floating objects, and the current state of floating objects can be estimated and the future state can be predicted through Bayesian reasoning. The process of temporal fusion is to integrate and fuse data from different data sources or different time points under the framework of the spatiotemporal autoregressive model and the dynamic Bayesian network to generate a more comprehensive and accurate fused data set. Better capture the dynamic changing characteristics of floating objects, and provide scientific basis and decision support for floating object governance, environmental management and emergency response.

[0091] Furthermore, the extracting and fusing the features of the spatial data set to obtain the fused data includes:

[0092] Extracting characteristic information of the data in the second data set;

[0093] Performing feature selection on the feature information to obtain multiple key features;

[0094] The plurality of key features are spatially fused using feature integration and feature-level splicing to obtain the fused data.

[0095] Useful feature information is extracted from the data in the second dataset, such as color and texture features in images, band reflectance features in spectral data, and physical and chemical parameter features in water quality data. For image data, convolutional neural networks can be used to automatically extract deep features.

[0096] Perform feature selection on feature information, select the most useful features, remove redundant information, and finally obtain multiple key features. Feature selection is the process of selecting the most useful feature subset for classification or regression tasks from the extracted features. This is usually achieved through dimensionality reduction techniques such as principal component analysis and linear discriminant analysis. Principal component analysis: extracts principal components by maximizing data variance, while linear discriminant analysis selects features by maximizing inter-class differences and minimizing intra-class differences.

[0097] Spatial fusion of multiple key features, combining features from different data sources or different feature extraction methods to form a unified feature vector, and finally obtaining fused data, can be achieved by weighted fusion, feature-level splicing and other methods. Weighted fusion: For different types of features or data from different sources, different weights can be given according to their importance and reliability, and then the weighted features are fused. Feature-level splicing: Feature vectors from different sources are directly spliced ​​together to form a longer feature vector. By fusing multiple features, more comprehensive information can be captured, thereby more accurately identifying floating objects, which helps to build an efficient and accurate model.

[0098] Step S13: inputting the fused data into a floating object recognition model to identify the floating objects and obtain monitoring results; the monitoring results include: the location and type of the floating objects at each time point within the target time period.

[0099] The fused data is input into the floating object recognition model to identify the floating objects, and the monitoring results of the location and type of floating objects at each time point in the target time period are obtained. Among them, the floating object recognition model is trained. Specifically, based on the fused data, a machine learning algorithm (such as support vector machine SVM, convolutional neural network CNN, etc.) is used to optimize and establish a machine learning model; the fused data set is divided into a training set, a validation set, and a test set; the training set is used to train the model, the validation set is used to adjust the model parameters and perform early stopping to prevent overfitting; the test set is used to evaluate the final performance of the model. During the training process, the model parameters are updated using optimization algorithms such as gradient descent to minimize the loss function (such as cross entropy loss, mean square error, etc.), and a trained floating object recognition model is obtained after repeated training. By continuously collecting new and various data, self-learning and model optimization can be performed to improve the accuracy and efficiency of the monitoring method. Machine learning algorithms and deep learning models can process complex data, further improve the accuracy of floating object recognition, and help to formulate scientific and reasonable floating object cleaning and management strategies, and improve the protection and management level of the water environment.

[0100] Step S14: Perform data analysis based on the monitoring results to obtain data analysis results; display the data analysis results; the data analysis results include: a floating object distribution map, statistical data and a change trend map; the statistical data include: the number of each type of floating objects existing in the target area and the total number of floating objects within the target time period.

[0101] According to the monitoring results (the location and type of floating objects at each time point in the target time period), after data analysis, the floating object distribution map, statistical data and change trend map are displayed on the user interface, where the floating object distribution map refers to the location of each type of floating objects in the target area and the target time period; the statistical data refers to the number of each type of floating objects and the total number of floating objects in the target area and the target time period, and also includes the density of floating objects; the change trend map refers to the overall change trend of the number of floating objects in the target area and the target time period. The visual display provides users with intuitive and comprehensive data support, allowing them to view monitoring results in real time, understand the status of floating objects in a timely manner, and help users take appropriate treatment measures in a timely manner to reduce environmental risks and impacts.

[0102] Step S15: Determine whether the total number of the floating objects exceeds a preset threshold; if the total number of the floating objects exceeds the preset threshold, output warning information.

[0103] According to actual needs and the characteristics of monitoring data, considering various factors such as region, season, and weather, a preset threshold for the number of floating objects is set, and the preset threshold can be adjusted according to actual conditions. When the total number of floating objects in the target area and target time period exceeds the preset threshold, an early warning message is output. It can also be judged by the density of floating objects and the corresponding preset threshold. According to the early warning information, an early warning notification is issued to relevant personnel by email, SMS, pop-up window, alarm, etc. By setting a preset threshold and monitoring the number of floating objects in real time, it can ensure timely detection and response at the early stage of the problem, which helps to reduce the impact of floating objects on the environment and ecology.

[0104] Based on the same inventive concept, the present application embodiment discloses a reservoir area floating object monitoring device based on multi-source data fusion, such as Figure 2 As shown, the device comprises:

[0105] Acquisition module: used to acquire various data within the target area and target time period; the various data include: multi-spectral satellite remote sensing data, drone aerial photography data, ground sensor data and meteorological data; the drone aerial photography data is obtained by photographing floating objects in the target area; the ground sensor data includes: water quality parameters, the number and density of floating objects.

[0106] Data fusion module: used for fusing the multiple data to obtain fused data; the data fusion includes: spatial fusion, temporal fusion and feature extraction and fusion.

[0107] Identification processing module: used for inputting the fusion data into the floating object identification model to identify the floating objects and obtain monitoring results; the monitoring results include: the location and type of the floating objects at each time point within the target time period.

[0108] Analysis and display module: used to perform data analysis according to the monitoring results to obtain data analysis results; and to display according to the data analysis results; the data analysis results include: a floating object distribution map, statistical data and a change trend map; the statistical data include: the number of each type of floating objects existing in the target area and the target time period and the total number of floating objects.

[0109] Alarm module: used to determine whether the total number of the floating objects exceeds the preset threshold; if the total number of the floating objects exceeds the preset threshold, output warning information.

[0110] In an optional implementation, the data fusion module includes:

[0111] A spatial fusion module, used for performing spatial fusion on the multiple data to obtain a first data set;

[0112] A time fusion module, used for performing time fusion on the first data set to obtain a second data set;

[0113] The feature extraction and fusion module is used to extract and fuse the features of the second data set to obtain the fused data.

[0114] In an optional implementation, the spatial fusion module is specifically used to:

[0115] Using an image registration method to spatially align the multiple data to obtain spatially aligned data, the image registration method comprising: feature point matching and interpolation;

[0116] The aligned data are spatially fused using a data fusion method to obtain the first data set, wherein the data fusion method includes: weighted average method and minimum variance fusion.

[0117] In an optional implementation, the time fusion module is specifically used to:

[0118] Using an interpolation method or a resampling method to time-align the data in the first data set to obtain time-aligned data;

[0119] Performing time series analysis on the time-aligned data to obtain analyzed data;

[0120] The analyzed data are temporally fused using a spatiotemporal autoregressive model and a dynamic Bayesian network to obtain the second data set.

[0121] In an optional implementation, the feature extraction and fusion module is specifically used for:

[0122] Extracting characteristic information of the data in the second data set;

[0123] Performing feature selection on the feature information to obtain multiple key features;

[0124] The plurality of key features are spatially fused using feature integration and feature-level splicing to obtain the fused data.

[0125] In an optional embodiment, the device further comprises:

[0126] The preprocessing module is used to preprocess the multiple data; the preprocessing includes: format conversion and noise removal.

[0127] The method for monitoring floating objects in a reservoir area based on multi-source data fusion provided by the disclosed embodiment obtains multiple floating object data in a target area and a target time period, performs spatial fusion, temporal fusion, and feature extraction and fusion, and inputs the fused data into a floating object recognition model to identify the floating objects and obtain monitoring results; performs data analysis based on the monitoring results to obtain a floating object distribution map, statistical data, and a change trend map; determines whether the total number of floating objects in the statistical data exceeds a preset threshold, and outputs warning information if it exceeds the preset threshold. Efficient and accurate monitoring of floating objects in the reservoir area is achieved; the fusion of multiple data helps to improve the accuracy of floating object identification; and it has a real-time warning function, which can timely clean up floating objects in the water area and promote the sustainable development of the aquatic ecosystem.

[0128] It should be noted that, for the method embodiments, for the sake of simplicity, they are all described as a series of action combinations, but those skilled in the art should be aware that the embodiments of the present disclosure are not limited by the order of the actions described, because according to the embodiments of the present disclosure, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily required by the embodiments of the present disclosure.

[0129] The present disclosure also provides an electronic device, referring to Figure 3 , Figure 3 Schematic diagram of an electronic device shown in an embodiment of the present disclosure. Figure 3 As shown, the electronic device 100 includes: a memory 110 and a processor 120. The memory 110 and the processor 120 are connected via a bus communication. The memory 110 stores a computer program, which can be run on the processor 120 to implement the steps in the reservoir area floating object monitoring method based on multi-source data fusion disclosed in the embodiment of the present disclosure.

[0130] The embodiment of the present disclosure also provides a computer-readable storage medium. When the instructions in the computer-readable storage medium are executed by a processor of a computer device, the computer device can execute the steps in the method for monitoring floating objects in a reservoir area based on multi-source data fusion as disclosed in the embodiment of the present disclosure.

[0131] The embodiment of the present disclosure also provides a computer program product, including a computer program. When the computer program is executed by a processor of a computer device, the computer program can execute the steps in the reservoir area floating object monitoring method based on multi-source data fusion as disclosed in the embodiment of the present disclosure.

[0132] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0133] The embodiments of the present application are described with reference to the flowcharts and / or block diagrams of the methods, apparatuses, electronic devices, and computer program products according to the embodiments of the present application. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of the processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing terminal device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0134] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing terminal device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0135] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device so that a series of operating steps are executed on the computer or other programmable terminal device to produce a computer-implemented process, thereby providing instructions for executing on the computer or other programmable terminal device to implement the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0136] Although the preferred embodiments of the present application have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the embodiments of the present application.

[0137] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or terminal device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or terminal device. In the absence of further restrictions, the elements defined by the sentence "including one..." do not exclude the existence of other identical elements in the process, method, article or terminal device including the elements.

[0138] The above is a detailed introduction to the reservoir area floating object monitoring method, device, equipment, storage medium and product with multi-source data fusion provided by the present application. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea; at the same time, for general technical personnel in this field, according to the idea of ​​the present application, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A method for monitoring floating objects in a reservoir area based on multi-source data fusion, characterized in that: include: Obtain various data in the target area and target time period; The multiple data include: multi-spectral satellite remote sensing data, drone aerial photography data, ground sensor data and meteorological data; the drone aerial photography data is obtained by photographing floating objects in the target area; the ground sensor data include: water quality parameters, the number and density of floating objects; Performing data fusion on the multiple data to obtain fused data; the data fusion includes: spatial fusion, temporal fusion and feature extraction and fusion; Input the fused data into a floating object recognition model to identify the floating objects and obtain monitoring results; the monitoring results include: the location and type of the floating objects at each time point within the target time period; According to the monitoring results, data analysis is performed to obtain data analysis results; and the data analysis results are displayed; the data analysis results include: a floating object distribution map, statistical data and a change trend map; the statistical data include: the number of each type of floating objects existing in the target area and the total number of floating objects within the target time period; Determine whether the total number of the floating objects exceeds a preset threshold; if the total number of the floating objects exceeds the preset threshold, output warning information.

2. The method for monitoring floating objects in a reservoir area based on multi-source data fusion according to claim 1 is characterized in that: The step of fusing the multiple data to obtain fused data includes: Performing spatial fusion on the multiple data to obtain a first data set; Performing time fusion on the first data set to obtain a second data set; Feature extraction and fusion are performed on the second data set to obtain the fused data.

3. The reservoir area floating object monitoring method based on multi-source data fusion according to claim 2 is characterized in that: The spatial fusion of the multiple data to obtain a first data set includes: Using an image registration method to spatially align the multiple data to obtain spatially aligned data, the image registration method comprising: feature point matching and interpolation; The aligned data are spatially fused using a data fusion method to obtain the first data set, wherein the data fusion method includes: weighted average method and minimum variance fusion.

4. The method for monitoring floating objects in a reservoir area based on multi-source data fusion according to claim 3 is characterized in that: The performing time fusion on the first data set to obtain a second data set includes: Using an interpolation method or a resampling method to time-align the data in the first data set to obtain time-aligned data; Performing time series analysis on the time-aligned data to obtain analyzed data; The analyzed data are temporally fused using a spatiotemporal autoregressive model and a dynamic Bayesian network to obtain the second data set.

5. The method for monitoring floating objects in a reservoir area based on multi-source data fusion according to claim 4 is characterized in that: The extracting and fusing the features of the spatial data set to obtain the fused data includes: Extracting characteristic information of the data in the second data set; Performing feature selection on the feature information to obtain multiple key features; The plurality of key features are spatially fused using feature integration and feature-level splicing to obtain the fused data.

6. The method for monitoring floating objects in a reservoir area based on multi-source data fusion according to claim 1 is characterized in that: After acquiring the multiple data, the method further includes: preprocessing the multiple data; the preprocessing includes: format conversion and noise removal.

7. A reservoir area floating object monitoring device based on multi-source data fusion, characterized in that: The device comprises: The acquisition module is used to acquire multiple data in the target area and target time period; the multiple data include: multi-spectral satellite remote sensing data, drone aerial photography data, ground sensor data and meteorological data; the drone aerial photography data is obtained by photographing floating objects in the target area; the ground sensor data include: water quality parameters, the number and density of floating objects; A data fusion module is used to fuse the multiple data to obtain fused data; the data fusion includes: spatial fusion, temporal fusion and feature extraction and fusion; The identification processing module is used to input the fusion data into a floating object identification model to identify the floating objects and obtain monitoring results; the monitoring results include: the location and type of the floating objects at each time point within the target time period; The analysis and display module is used to perform data analysis according to the monitoring results to obtain data analysis results; and to display according to the data analysis results; the data analysis results include: a floating object distribution map, statistical data and a change trend map; the statistical data include: the number of each type of floating objects existing in the target area and the total number of floating objects within the target time period; Alarm module: used to determine whether the total number of the floating objects exceeds the preset threshold; if the total number of the floating objects exceeds the preset threshold, output warning information.

8. The reservoir area floating object monitoring device based on multi-source data fusion according to claim 7 is characterized in that: The data fusion module comprises: A spatial fusion module, used for performing spatial fusion on the multiple data to obtain a first data set; A time fusion module, used for performing time fusion on the first data set to obtain a second data set; The feature extraction and fusion module is used to extract and fuse the features of the second data set to obtain the fused data.

9. The reservoir area floating object monitoring device based on multi-source data fusion according to claim 8 is characterized in that: The spatial fusion module is specifically used for: Using an image registration method to spatially align the multiple data to obtain spatially aligned data, the image registration method comprising: feature point matching and interpolation; The aligned data are spatially fused using a data fusion method to obtain the first data set, wherein the data fusion method includes: weighted average method and minimum variance fusion.

10. The reservoir area floating object monitoring device based on multi-source data fusion according to claim 8, characterized in that: The time fusion module is specifically used for: Using an interpolation method or a resampling method to time-align the data in the first data set to obtain time-aligned data; Performing time series analysis on the time-aligned data to obtain analyzed data; The analyzed data are temporally fused using a spatiotemporal autoregressive model and a dynamic Bayesian network to obtain the second data set.

11. The reservoir area floating object monitoring device based on multi-source data fusion according to claim 8, characterized in that: The feature extraction and fusion module is specifically used for: Extracting characteristic information of the data in the second data set; Performing feature selection on the feature information to obtain multiple key features; The plurality of key features are spatially fused using feature integration and feature-level splicing to obtain the fused data.

12. The reservoir area floating object monitoring device based on multi-source data fusion according to claim 7 is characterized in that: The device also includes: The preprocessing module is used to preprocess the multiple data; the preprocessing includes: format conversion and noise removal.

13. An electronic device, characterized in that: include: A processor, a memory, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the method for monitoring floating objects in a reservoir area based on multi-source data fusion as claimed in any one of claims 1 to 6 is implemented.

14. A computer-readable storage medium, characterized in that: When the instructions in the storage medium are executed by a processor of an electronic device, the electronic device is enabled to execute the reservoir area floating object monitoring method based on multi-source data fusion as described in any one of claims 1 to 6.

15. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the method for monitoring floating objects in a reservoir area based on multi-source data fusion as described in any one of claims 1 to 6 is implemented.

Images