River flood warning method based on machine learning and multi-meteorological modal fusion

Abstract

A river flood warning method based on machine learning and multi-meteorological modal fusion is disclosed. First, the time series of river water level and the time series of rainfall are arranged as input vectors respectively, and then associated coding is performed to obtain a rainfall-river water level time series associated feature vector. Then, the satellite remote sensing image of the river basin is feature extracted by a river basin surface morphology feature extractor and processed by a feature map enhancer based on a heavy parameterization layer to obtain a river basin surface morphology enhanced feature map. Then, the river basin surface morphology enhanced feature map and the rainfall-river water level time series associated feature vector are multi-modally fused to obtain a multi-meteorological modal fusion feature. Finally, based on the multi-meteorological modal fusion feature, it is determined whether to generate a river flood warning prompt. In this way, the accuracy and real-time performance of flood warnings can be improved, providing effective decision support for flood defense and rescue.

Description

Technical Field

[0001] The present application relates to the field of river flood warning, and more specifically, to a river flood warning method based on machine learning and multi-meteorological modality fusion. Background Art

[0002] River floods are a common natural disaster that causes serious damage to human society and the ecological environment. In order to effectively prevent and mitigate flood disasters, it is necessary to establish a scientific flood warning system to provide timely and accurate flood information and warning signals. However, traditional flood warning methods are usually based on empirical formulas or hydrological models, ignoring the complex nonlinear relationship between river water level and rainfall, as well as the diversity of the impact of river basin surface morphology on floods, resulting in instability and unreliability of warning results.

[0003] Therefore, a river flood warning scheme based on machine learning and multi-meteorological modal fusion is desired. Summary of the invention

[0004] In order to solve the above technical problems, this application is proposed. This application provides a river flood warning method based on machine learning and multi-meteorological modal fusion, which can use multiple data sources such as satellite remote sensing and meteorological stations to establish a correlation model between river water level and rainfall through machine learning algorithms to predict and judge river water level, and can also combine river basin surface morphological information such as topography, soil, and vegetation in the river basin to assess the possibility and severity of flood disasters, thereby comprehensively generating flood warning signals of different levels.

[0005] According to one aspect of the present application, a river flood early warning method based on machine learning and multi-meteorological modal fusion is provided, which includes:

[0006] Obtain satellite remote sensing images of river basins;

[0007] Acquire weather station data, wherein the weather station data includes a time series of river water level and a time series of rainfall;

[0008] Arranging the time series of the river water level and the time series of the rainfall into a river water level time series input vector and a rainfall time series input vector respectively according to the time dimension;

[0009] Performing association coding on the rainfall time series input vector and the river water level time series input vector to obtain a rainfall-river water level time series association feature vector;

[0010] Extracting features from the satellite remote sensing image of the river basin by using a river basin surface morphology feature extractor based on a deep neural network model to obtain a river basin surface morphology feature map;

[0011] Using a feature map enhancer based on a re-parameterized layer to process the river basin surface morphology feature map to obtain a river basin surface morphology enhanced feature map;

[0012] Performing multi-modal fusion on the river basin surface morphology enhancement feature map and the rainfall-river water level time series correlation feature vector to obtain multi-meteorological modal fusion features; and

[0013] Based on the multi-meteorological modal fusion features, it is determined whether to generate a river flood warning prompt.

[0014] Compared with the prior art, the river flood warning method based on machine learning and multi-meteorological modal fusion provided by the present application first arranges the time series of river water level and the time series of rainfall as input vectors respectively and then performs association coding to obtain rainfall-river water level time series association feature vectors, then extracts features from the satellite remote sensing image of the river basin through the river basin surface morphology feature extractor and processes it using a feature map enhancer based on a heavy parameterization layer to obtain a river basin surface morphology enhanced feature map, then performs multi-modal fusion on the river basin surface morphology enhanced feature map and the rainfall-river water level time series association feature vector to obtain multi-meteorological modal fusion features, and finally, based on the multi-meteorological modal fusion features, determines whether to generate a river flood warning prompt. In this way, the accuracy and real-time performance of flood warnings can be improved, providing effective decision support for flood defense and rescue. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. The following drawings are not intentionally scaled to the actual size, and the focus is on illustrating the main purpose of the present application.

[0016] Figure 1 This is a flow chart of a river flood warning method based on machine learning and multi-meteorological modality fusion according to an embodiment of the present application.

[0017] Figure 2 Schematic diagram of the architecture of a river flood warning method based on machine learning and multi-meteorological modality fusion according to an embodiment of the present application.

[0018] Figure 3 It is a block diagram of a river flood warning system based on machine learning and multi-meteorological modality fusion according to an embodiment of the present application.

[0019] Figure 4 This is an application scenario diagram of a river flood warning method based on machine learning and multi-meteorological modal fusion according to an embodiment of the present application. DETAILED DESCRIPTION

[0020] The technical solutions in the embodiments of the present application will be described clearly and completely below in conjunction with the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, not all embodiments. Based on the embodiments of the present application, all other embodiments obtained by ordinary technicians in this field without creative work also fall within the scope of protection of the present application.

[0021] As shown in this application and claims, unless the context clearly indicates an exception, the words "a", "an", "an" and / or "the" do not refer to the singular and may also include the plural. Generally speaking, the terms "include" and "comprise" only indicate the inclusion of the steps and elements that have been clearly identified, and these steps and elements do not constitute an exclusive list. The method or device may also include other steps or elements.

[0022] Although the present application makes various references to certain modules in the system according to the embodiments of the present application, any number of different modules can be used and run on the user terminal and / or server. The modules are only illustrative, and different aspects of the system and method can use different modules.

[0023] Flowcharts are used in the present application to illustrate the operations performed by the system according to the embodiments of the present application. It should be understood that the preceding or following operations are not necessarily performed accurately in order. On the contrary, various steps may be processed in reverse order or simultaneously as required. Meanwhile, other operations may also be added to these processes, or a certain step or several steps of operations may be removed from these processes.

[0024] Below, the exemplary embodiments according to the present application will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application, and it should be understood that the present application is not limited to the exemplary embodiments described here.

[0025] In view of the above technical problems, in the technical solution of this application, a river flood warning method based on machine learning and multi-meteorological modal fusion is proposed. This method uses multiple data sources such as satellite remote sensing and meteorological stations, and establishes a correlation model between river water level and rainfall through a machine learning algorithm to predict and judge the river water level. In addition, it also combines the river basin's topography, soil, vegetation and other river basin surface morphological information to assess the possibility and severity of flood disasters, thereby comprehensively generating flood warning signals of different levels. In this way, the accuracy and real-time performance of flood warnings can be improved, providing effective decision-making support for flood defense and rescue.

[0026] Figure 1This is a flow chart of a river flood warning method based on machine learning and multi-meteorological modality fusion according to an embodiment of the present application. Figure 2 FIG. 1 is a schematic diagram of the architecture of a river flood warning method based on machine learning and multi-meteorological modality fusion according to an embodiment of the present application. Figure 1 and Figure 2 As shown, according to the embodiment of the present application, the river flood warning method based on machine learning and multi-meteorological modal fusion includes the following steps: S110, obtaining a satellite remote sensing image of a river basin; S120, obtaining meteorological station data, wherein the meteorological station data includes a time series of a river water level and a time series of a rainfall; S130, arranging the time series of the river water level and the time series of the rainfall into a river water level time series input vector and a rainfall time series input vector according to the time dimension respectively; S140, performing association coding on the rainfall time series input vector and the river water level time series input vector to obtain a rainfall-river water level time series association feature vector ; S150, extracting features from the satellite remote sensing image of the river basin through a river basin surface morphology feature extractor based on a deep neural network model to obtain a river basin surface morphology feature map; S160, processing the river basin surface morphology feature map using a feature map enhancer based on a heavy parameterization layer to obtain a river basin surface morphology enhanced feature map; S170, performing multimodal fusion on the river basin surface morphology enhanced feature map and the rainfall-river water level time series correlation feature vector to obtain a multi-meteorological modal fusion feature; and, S180, determining whether to generate a river flood warning prompt based on the multi-meteorological modal fusion feature.

[0027] Specifically, in the technical solution of the present application, first, a satellite remote sensing image of the river basin is obtained, and weather station data is obtained, wherein the weather station data includes a time series of river water level and a time series of rainfall. Next, considering that in river flood warning, the river water level and the rainfall have a dynamic change law of time series in the time dimension, and there is a complex correlation between the river water level and the rainfall. Therefore, in order to establish a correlation between the river water level and the rainfall, so as to realize the prediction of the future water level, in the technical solution of the present application, it is necessary to arrange the time series of the river water level and the time series of the rainfall into a river water level time series input vector and a rainfall time series input vector according to the time dimension respectively.

[0028] Then, the sample covariance correlation matrix of the rainfall time series input vector relative to the river water level time series input vector is calculated to obtain the rainfall-river water level time series correlation matrix. It should be understood that in flood warning, rainfall is one of the main factors leading to river water level changes. By calculating the sample covariance correlation matrix between the rainfall and river water level time series input vectors, the correlation pattern and change trend between rainfall and river water level can be revealed, providing a basis for subsequent feature extraction and model building.

[0029] Then, the rainfall-river water level time series correlation matrix is ​​subjected to feature mining in a rainfall-river water level time series correlation feature extractor based on a convolutional neural network model to extract the time series collaborative correlation feature information between the rainfall and the river water level, thereby obtaining a rainfall-river water level time series correlation feature vector.

[0030] Accordingly, in step S140, the rainfall time series input vector and the river water level time series input vector are associated and encoded to obtain a rainfall-river water level time series associated feature vector, including: calculating the sample covariance association matrix of the rainfall time series input vector relative to the river water level time series input vector to obtain a rainfall-river water level time series associated matrix; and, passing the rainfall-river water level time series associated matrix through a rainfall-river water level time series associated feature extractor based on a convolutional neural network model to obtain the rainfall-river water level time series associated feature vector.

[0031] It should be understood that the satellite remote sensing image can provide a wide range of surface information, including the terrain, soil, vegetation and other features of the river basin. These surface features are of great significance for flood warning because they can affect the distribution and flow rate of water flow, thereby affecting the formation and development of floods. Therefore, in order to extract surface morphological features to further improve the accuracy of river flood warning, in the technical solution of the present application, the satellite remote sensing image of the river basin is further subjected to feature mining in a river basin surface morphological feature extractor based on a convolutional neural network model to extract the characteristic distribution information about the surface morphology of the river basin in the satellite remote sensing image, thereby obtaining a surface morphological feature map of the river basin. By processing with the river basin surface morphological feature extractor based on a convolutional neural network model, features related to the surface morphology can be extracted from the satellite remote sensing image. Therefore, the surface morphological feature map of the river basin can provide more detailed and comprehensive river basin surface morphological feature information, providing more comprehensive and accurate information for flood warning.

[0032] Correspondingly, in step S150, the deep neural network model is a convolutional neural network model, that is, the river basin surface morphology feature extractor based on the deep neural network model is a river basin surface morphology feature extractor based on the convolutional neural network model. It is worth mentioning that the convolutional neural network (CNN) is a deep learning model mainly used to process data with a grid structure. It is a feedforward neural network with multiple levels, including a convolution layer, a pooling layer and a fully connected layer. The core idea of ​​the convolutional neural network is to extract the features of the input data through a convolution operation. The convolution operation uses a set of learnable filters (also called convolution kernels) to perform element-by-element multiplication and summation operations on the input data to generate a feature map. These feature maps capture the local features of the input data, such as edges, textures, etc. After the convolution layer, a pooling layer is usually applied to reduce the size of the feature map and reduce the model's sensitivity to the location of the input data. Common pooling operations include maximum pooling and average pooling. Finally, the feature map is converted into a vector with predictive power through a fully connected layer for classification, regression or other tasks. Each neuron in a fully connected layer is connected to all neurons in the previous layer.

[0033] Further, considering that in flood warning, the surface morphology feature map of the river basin contains information on the surface morphology, such as elevation, slope, etc. However, these feature maps may still contain some implicit information. Therefore, in order to enhance and enrich the expressive ability of these features of the surface morphology of the river basin and make it more discriminative and robust, in the technical solution of the present application, the surface morphology feature map of the river basin is further processed using a feature map enhancer based on a reparameterized layer to obtain an enhanced feature map of the surface morphology of the river basin. Through the processing of the feature map enhancer based on the reparameterized layer, randomness can be introduced, and the original feature map can be reparameterized into a richer feature representation, thereby enhancing the expressive ability of the surface morphology feature map of the river basin, so that it can better capture the details and changes of the surface morphology, thereby extracting more discriminative and important feature information, such as the shape of the river, the width of the river channel, the undulation of the terrain, etc., so as to more accurately assess the possibility and severity of the flood. In this process, the mean and variance of each surface morphology feature map of the river basin are extracted and used to generate a new feature map. This form of reparameterization can be seen as a way to perform data augmentation in the semantic feature space, which helps to improve the classifier's ability to perceive and recognize surface morphological features of different river basins and further improve the accuracy of flood warnings.

[0034] Accordingly, in step S160, the river basin surface morphology feature map is processed using a feature map enhancer based on a reparameterized layer to obtain a river basin surface morphology enhanced feature map, including: using a feature map enhancer based on a reparameterized layer to process the river basin surface morphology feature map using the following enhancement formula to obtain a river basin surface morphology enhanced feature map; wherein the enhancement formula is:

[0035]

[0036] Among them, F is the surface morphological characteristic map of the river basin, F' is the enhanced surface morphological characteristic map of the river basin, μ and σ are the eigenvalue set f i The mean and variance of ∈F, f i is the characteristic value of the ith position of the surface morphological characteristic map of the river basin, f i ' is the characteristic value of the i-th position of the river basin surface morphology enhancement feature map, and f i '∈F', log represents the logarithmic function with base 2, arccos represents the inverse cosine function, and arcsin represents the inverse sine function.

[0037] Next, considering that the river basin surface morphology enhancement feature map provides information about the surface morphology characteristics of the river basin, and the rainfall-river water level time series correlation feature vector reflects the temporal synergistic correlation between rainfall and water level. However, using these two types of information alone may not be able to fully describe the occurrence and development of floods, because the formation and evolution of floods are affected by multiple factors. Based on this, in order to be able to fuse the information of different meteorological modes together, so as to comprehensively consider the impact of multiple factors on flood warning, in the technical solution of the present application, a multi-meteorological modal fusion device based on a meta-network is further used to perform multi-modal fusion on the river basin surface morphology enhancement feature map and the rainfall-river water level time series correlation feature vector to obtain a multi-meteorological modal fusion feature map. It should be understood that the multi-meteorological modal fusion device based on a meta-network is a network structure for learning the relationship between different modes, which can fuse the features of different modes channel by channel to obtain a more comprehensive and integrated feature representation. By using the meta-network-based multi-meteorological modal fuser, the river basin surface morphology enhanced feature map and the rainfall-river water level time series correlation feature vector can be effectively fused, so that when conducting river flood warning, the information of multiple factors such as surface morphology, rainfall and water level are comprehensively considered, and the possibility and severity of floods can be more comprehensively described, thereby improving the accuracy and reliability of flood warnings and providing more effective decision-making support for flood defense and rescue.

[0038] Correspondingly, in step S170, multimodal fusion is performed on the river basin surface morphology enhancement feature map and the rainfall-river water level time series correlation feature vector to obtain a multi-meteorological modal fusion feature, including: using a meta-network-based multi-meteorological modal fuser to perform multimodal fusion on the river basin surface morphology enhancement feature map and the rainfall-river water level time series correlation feature vector to obtain a multi-meteorological modal fusion feature map as the multi-meteorological modal fusion feature.

[0039] In one example, a meta-network-based multi-meteorological modal fuser is used to perform multi-modal fusion on the river basin surface morphology enhancement feature map and the rainfall-river water level time series correlation feature vector to obtain a multi-meteorological modal fusion feature map as the multi-meteorological modal fusion feature, including: passing the rainfall-river water level time series correlation feature vector through a point convolution layer to obtain a first convolution feature vector; passing the first convolution feature vector through a corrected linear unit based on a ReLU function to obtain a first corrected convolution feature vector; passing the first corrected convolution feature vector through a point convolution layer to obtain a second convolution feature vector; passing the second convolution feature vector through a corrected linear unit based on a Sigmoid function to obtain a second corrected convolution feature vector; and, fusing the second corrected convolution feature vector with the river basin surface morphology enhancement feature map to obtain the multi-meteorological modal fusion feature map.

[0040] Then, the multi-meteorological modal fusion feature map is passed through a classifier to obtain a classification result, and the classification result is used to indicate whether a river flood warning prompt is generated. In other words, the fusion feature information of the multi-meteorological modal data is used for classification processing to comprehensively generate flood warning signals of different levels. In this way, the accuracy and real-time performance of flood warnings can be improved, and effective decision support can be provided for flood defense and rescue.

[0041] Accordingly, in step S180, based on the multi-meteorological modal fusion features, it is determined whether to generate a river flood warning prompt, including: passing the multi-meteorological modal fusion feature map through a classifier to obtain a classification result, and the classification result is used to indicate whether to generate a river flood warning prompt.

[0042] Furthermore, in the technical solution of the present application, the river flood warning method based on machine learning and multi-meteorological modal fusion also includes a training step: for training the rainfall-river water level time series correlation feature extractor based on the convolutional neural network model, the river basin surface morphology feature extractor based on the convolutional neural network model, the feature map enhancer based on the re-parameterization layer, the meta-network-based multi-meteorological modal fuser and the classifier.

[0043] In one example, the training step includes: acquiring training data, the training data includes training satellite remote sensing images of the river basin, training meteorological station data, and whether a real value of a river flood warning prompt is generated, wherein the training meteorological station data includes a time series of training river water levels and a time series of training rainfall; arranging the time series of the training river water levels and the time series of the training rainfall into a training river water level time series input vector and a training rainfall time series input vector respectively according to the time dimension; calculating the sample covariance association matrix of the training rainfall time series input vector relative to the training river water level time series input vector to obtain a training rainfall-river water level time series association matrix; passing the training rainfall-river water level time series association matrix through the rainfall-river water level time series association feature extractor based on the convolutional neural network model to obtain a training rainfall-river water level time series association feature vector; and extracting the training satellite remote sensing images of the river basin through the river basin surface morphology feature extractor based on the convolutional neural network model. The method comprises the following steps: extracting features from the sensing image to obtain a surface morphology feature map of a training river basin; processing the surface morphology feature map of the training river basin using the feature map enhancer based on the reparameterization layer to obtain an enhanced surface morphology feature map of the training river basin; performing multimodal fusion on the enhanced surface morphology feature map of the training river basin and the training rainfall-river water level time series correlation feature vector using the meta-network-based multi-meteorological modality fusion to obtain a training multi-meteorological modality fusion feature map; optimizing the training multi-meteorological modality fusion feature map to obtain an optimized training multi-meteorological modality fusion feature map; passing the optimized training multi-meteorological modality fusion feature map through the classifier to obtain a classification loss function value; and training the rainfall-river water level time series correlation feature extractor based on the convolutional neural network model, the river basin surface morphology feature extractor based on the convolutional neural network model, the feature map enhancer based on the reparameterization layer, the meta-network-based multi-meteorological modality fusion and the classifier using the classification loss function value.

[0044] In the above technical solution, each feature matrix of the training river basin surface morphology feature map expresses the image semantic features of the training satellite remote sensing image of the river basin, and each feature matrix follows the channel distribution of the convolutional neural network model. When the training river basin surface morphology feature map is processed using a feature map enhancer based on a reparameterization layer, the feature map enhancer based on a reparameterization layer performs feature map enhancement on the training river basin surface morphology feature map based on prior information, but it does not change the channel distribution pattern between each feature matrix of the obtained river basin surface morphology enhanced feature map. In the above technical solution, the training rainfall-river water level time series correlation feature vector represents the local time domain semantic correlation feature between rainfall and river water level based on convolution coding of the full time domain sample covariance correlation distribution of the training rainfall and the training river water level.

[0045] In this way, after using the meta-network-based multi-meteorological modal fuser to perform multi-modal fusion on the training river basin surface morphological feature map and the training rainfall-river water level time series correlation feature vector, the training river basin surface morphological feature map will be constrained in the channel dimension based on the local time domain semantic association features between the training rainfall-river water level expressed by the training rainfall-river water level time series correlation feature vector, which also makes the training multi-meteorological modal fusion feature map have a mixed distribution in the channel dimension with a weakened association with the image semantic distribution of the feature matrix, so that the overall feature distribution of the training multi-meteorological modal fusion feature map based on the image semantic feature spatial distribution and the channel dimension distribution is sparse in the probability density representation in the probability density domain, thereby affecting the regression convergence effect of the classification result when classification is performed through the classifier.

[0046] Based on this, the applicant of the present application trains the multi-meteorological modality fusion feature map F 1 Accordingly, in one example, the training multi-meteorological modal fusion feature graph is optimized to obtain an optimized training multi-meteorological modal fusion feature graph, including: optimizing the training multi-meteorological modal fusion feature graph using the following optimization formula to obtain the optimized training multi-meteorological modal fusion feature graph; wherein the optimization formula is:

[0047]

[0048]

[0049] Among them, F 1 is the training multi-meteorological modality fusion feature map, F 1 ⊙2 Represents the position-by-position square graph of the feature map, W m is an intermediate weight map with trainable parameters, for example, based on the training multi-meteorological modality fusion feature map F1 The spatial distribution of image semantic features and channel dimension distribution properties are initially set to the eigenvalue of each feature matrix as the eigenvalue mean of the corresponding feature matrix of the training river basin surface morphological feature map, and then weighted along the channel by the training rainfall-river water level time series correlation feature vector. In addition, W I is a unit graph with all eigenvalues ​​1, ⊕ represents feature graph addition, ⊙ represents point multiplication by position, and F 1 ' is the optimized training multi-meteorological modal fusion feature map.

[0050] Here, in order to optimize the training multi-meteorological modality fusion feature map F 1 The distribution uniformity and consistency of the sparse probability density in the overall probability space are enhanced by the tail distribution reinforcement mechanism of the standard Cauchy distribution to train the multi-meteorological modal fusion feature map F 1 The distance distribution in the high-dimensional feature space is optimized based on the spatial angle tilt to achieve the training of the multi-meteorological mode fusion feature map F 1 The distances of the local feature distributions are weakly correlated with the feature distribution space resonance, thereby improving the training multi-meteorological modal fusion feature map F 1 The overall uniformity and consistency of the probability density distribution relative to the regression probability convergence improves the classification regression convergence effect, that is, the speed and accuracy of classification convergence. In this way, satellite remote sensing and meteorological data can be used to comprehensively carry out river flood warnings, thereby improving the accuracy and real-time nature of flood warnings and providing effective decision-making support for flood defense and rescue.

[0051] In summary, the river flood warning method based on machine learning and multi-meteorological modal fusion according to the embodiment of the present application is explained, which can improve the accuracy and real-time performance of flood warnings and provide effective decision-making support for flood defense and rescue.

[0052] Figure 3 FIG. 1 is a block diagram of a river flood warning system 100 based on machine learning and multi-meteorological modality fusion according to an embodiment of the present application. Figure 3As shown, according to the embodiment of the present application, the river flood warning system 100 based on machine learning and multi-meteorological modal fusion includes: a satellite data acquisition module 110, which is used to acquire satellite remote sensing images of the river basin; a meteorological station data acquisition module 120, which is used to acquire meteorological station data, wherein the meteorological station data includes a time series of river water level and a time series of rainfall; a vectorization module 130, which is used to arrange the time series of the river water level and the time series of the rainfall into a river water level time series input vector and a rainfall time series input vector according to the time dimension respectively; an association coding module 140, which is used to perform association coding on the rainfall time series input vector and the river water level time series input vector to obtain a rainfall-river water level time series association feature vector; a river A river basin surface morphology feature extraction module 150 is used to extract features from the satellite remote sensing image of the river basin through a river basin surface morphology feature extractor based on a deep neural network model to obtain a river basin surface morphology feature map; a feature map enhancement module 160 is used to process the river basin surface morphology feature map using a feature map enhancer based on a heavy parameterization layer to obtain a river basin surface morphology enhanced feature map; a multimodal fusion module 170 is used to perform multimodal fusion on the river basin surface morphology enhanced feature map and the rainfall-river water level time series correlation feature vector to obtain a multi-meteorological modal fusion feature; and an early warning analysis module 180 is used to determine whether to generate a river flood early warning prompt based on the multi-meteorological modal fusion feature.

[0053] Here, those skilled in the art can understand that the specific functions and operations of each module in the above-mentioned river flood warning system 100 based on machine learning and multi-meteorological modal fusion have been described in the above reference. Figure 1 to Figure 2 It has been introduced in detail in the description of the river flood early warning method based on machine learning and multi-meteorological modal fusion, and therefore, its repeated description will be omitted.

[0054] As described above, the river flood warning system 100 based on machine learning and multi-meteorological modal fusion according to the embodiment of the present application can be implemented in various wireless terminals, such as a server with a river flood warning algorithm based on machine learning and multi-meteorological modal fusion. In one example, the river flood warning system 100 based on machine learning and multi-meteorological modal fusion according to the embodiment of the present application can be integrated into the wireless terminal as a software module and / or hardware module. For example, the river flood warning system 100 based on machine learning and multi-meteorological modal fusion can be a software module in the operating system of the wireless terminal, or it can be an application developed for the wireless terminal; of course, the river flood warning system 100 based on machine learning and multi-meteorological modal fusion can also be one of the many hardware modules of the wireless terminal.

[0055] Alternatively, in another example, the river flood warning system 100 based on machine learning and multi-meteorological modal fusion and the wireless terminal may also be separate devices, and the river flood warning system 100 based on machine learning and multi-meteorological modal fusion may be connected to the wireless terminal via a wired and / or wireless network, and transmit interactive information in accordance with an agreed data format.

[0056] Figure 4 FIG. 1 is an application scenario diagram of a river flood warning method based on machine learning and multi-meteorological modal fusion according to an embodiment of the present application. Figure 4 As shown, in this application scenario, first, a satellite remote sensing image of the river basin is obtained (for example, Figure 4 D1 as shown in the figure) and weather station data (e.g. Figure 4 D2 as shown in the figure), wherein the weather station data includes a time series of river water level and a time series of rainfall, and then the satellite remote sensing image of the river basin and the weather station data are input into a server (for example, Figure 4 In S) shown in , the server is able to use the river flood warning algorithm based on machine learning and multi-meteorological modality fusion to process the satellite remote sensing images of the river basin and the meteorological station data to obtain a classification result indicating whether a river flood warning prompt is generated.

[0057] According to another aspect of the present application, a non-volatile computer-readable storage medium is provided, on which computer-readable instructions are stored. When the instructions are executed by a computer, the above-mentioned method can be executed.

[0058] The program part of the technology can be considered as a "product" or "manufactured product" in the form of executable code and / or related data, which is participated in or realized by computer-readable media. Tangible and permanent storage media can include any memory or storage used by any computer, processor, or similar device or related module. For example, various semiconductor memories, tape drives, disk drives or any similar devices that can provide storage functions for software.

[0059] In addition, it will be appreciated by those skilled in the art that various aspects of the present application may be illustrated and described by a number of patentable categories or situations, including any new and useful process, machine, product or combination of substances, or any new and useful improvements thereto. Accordingly, various aspects of the present application may be performed entirely by hardware, entirely by software (including firmware, resident software, microcode, etc.), or by a combination of hardware and software. The above hardware or software may all be referred to as "data blocks", "modules", "engines", "units", "components" or "systems". In addition, various aspects of the present application may be represented as a computer product located in one or more computer-readable media, which includes computer-readable program code.

[0060] Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by those of ordinary skill in the art to which this application belongs. It should also be understood that terms such as those defined in common dictionaries should be interpreted as having a meaning consistent with their meaning in the context of the relevant technology, and should not be interpreted in an idealized or extremely formal sense, unless explicitly defined as such herein.

[0061] The above is an explanation of the present application and should not be considered as a limitation thereof. Although several exemplary embodiments of the present application are described, it will be readily understood by those skilled in the art that many modifications may be made to the exemplary embodiments without departing from the novel teachings and advantages of the present application. Therefore, all of these modifications are intended to be included within the scope of the present application as defined by the claims. It should be understood that the above is an explanation of the present application and should not be considered to be limited to the specific embodiments disclosed, and modifications to the disclosed embodiments and other embodiments are intended to be included within the scope of the appended claims. The present application is defined by the claims and their equivalents.

Claims

1. A river flood warning method based on machine learning and multi-meteorological modal fusion, characterized in that: include: Obtain satellite remote sensing images of river basins; Acquire weather station data, wherein the weather station data includes a time series of river water level and a time series of rainfall; Arranging the time series of the river water level and the time series of the rainfall into a river water level time series input vector and a rainfall time series input vector respectively according to the time dimension; The rainfall time series input vector and the river water level time series input vector are associated and encoded to obtain a rainfall-river water level time series associated feature vector, wherein the sample covariance association matrix between the rainfall and river water level time series input vectors is calculated to reveal the association pattern and change trend between the rainfall and river water level, and feature mining is performed by a rainfall-river water level time series associated feature extractor based on a convolutional neural network model to extract the time series collaborative association feature information between the rainfall and the river water level, thereby obtaining the rainfall-river water level time series associated feature vector; Extracting features from the satellite remote sensing image of the river basin by using a river basin surface morphology feature extractor based on a deep neural network model to obtain a river basin surface morphology feature map; Using a feature map enhancer based on a re-parameterized layer to process the river basin surface morphology feature map to obtain a river basin surface morphology enhanced feature map; The river basin surface morphology enhancement feature map and the rainfall-river water level time series correlation feature vector are multimodally fused to obtain a multi-meteorological modality fusion feature, wherein a multi-meteorological modality fuser based on a meta-network is used to perform multimodal fusion on the river basin surface morphology enhancement feature map and the rainfall-river water level time series correlation feature vector to obtain a multi-meteorological modality fusion feature map, so as to fuse the features of different meteorological modalities channel by channel; and Based on the multi-meteorological modal fusion features, it is determined whether to generate a river flood warning prompt.

2. The river flood early warning method based on machine learning and multi-meteorological modal fusion according to claim 1 is characterized in that: The rainfall time series input vector and the river water level time series input vector are associated and encoded to obtain a rainfall-river water level time series associated feature vector, including: Calculating a sample covariance correlation matrix of the rainfall time series input vector relative to the river water level time series input vector to obtain a rainfall-river water level time series correlation matrix; and The rainfall-river water level time series correlation matrix is ​​passed through a rainfall-river water level time series correlation feature extractor based on a convolutional neural network model to obtain the rainfall-river water level time series correlation feature vector.

3. The river flood early warning method based on machine learning and multi-meteorological modal fusion according to claim 2 is characterized in that: The deep neural network model is a convolutional neural network model.

4. The river flood early warning method based on machine learning and multi-meteorological modal fusion according to claim 3 is characterized in that: The river basin surface morphology feature map is processed using a feature map enhancer based on a re-parameterized layer to obtain a river basin surface morphology enhanced feature map, including: The river basin surface morphology feature map is processed using a feature map enhancer based on a reparameterized layer using the following enhancement formula to obtain a river basin surface morphology enhanced feature map; wherein the enhancement formula is: Among them, F is the surface morphological characteristic map of the river basin, F ′ is the enhanced characteristic map of the river basin surface morphology, μ and σ are the eigenvalue sets f i The mean and variance of ∈F, f i is the characteristic value of the ith position of the surface morphological characteristic map of the river basin, f i ′ is the characteristic value of the i-th position of the river basin surface morphology enhancement feature map, and f i ′ ∈F ′ , log represents the logarithmic function with base 2, arccos represents the inverse cosine function, and arcsin represents the inverse sine function.

5. The river flood early warning method based on machine learning and multi-meteorological modal fusion according to claim 4 is characterized in that: The river basin surface morphology enhancement feature map and the rainfall-river water level time series correlation feature vector are subjected to multi-modal fusion to obtain multi-meteorological modal fusion features, including: A multi-meteorological modal fuser based on a meta-network is used to perform multi-modal fusion on the river basin surface morphology enhancement feature map and the rainfall-river water level time series correlation feature vector to obtain a multi-meteorological modal fusion feature map as the multi-meteorological modal fusion feature.

6. The river flood early warning method based on machine learning and multi-meteorological modal fusion according to claim 5 is characterized in that: The multi-meteorological modal fusion device based on the meta-network is used to perform multi-modal fusion on the river basin surface morphology enhancement feature map and the rainfall-river water level time series correlation feature vector to obtain a multi-meteorological modal fusion feature map as the multi-meteorological modal fusion feature, including: Passing the rainfall-river water level time series correlation feature vector through a point convolution layer to obtain a first convolution feature vector; Passing the first convolution feature vector through a rectified linear unit based on a ReLU function to obtain a first rectified convolution feature vector; Passing the first modified convolutional feature vector through a point convolution layer to obtain a second convolutional feature vector; Passing the second convolution feature vector through a rectified linear unit based on a Sigmoid function to obtain a second rectified convolution feature vector; and The second corrected convolution feature vector is fused with the river basin surface morphology enhancement feature map to obtain the multi-meteorological modality fusion feature map.

7. The river flood early warning method based on machine learning and multi-meteorological modal fusion according to claim 6 is characterized in that: Based on the multi-meteorological modal fusion features, determining whether to generate a river flood warning prompt includes: The multi-meteorological modal fusion feature map is passed through a classifier to obtain a classification result, and the classification result is used to indicate whether a river flood warning prompt is generated.

8. The river flood early warning method based on machine learning and multi-meteorological modal fusion according to claim 7 is characterized in that: It also includes a training step: for training the rainfall-river water level time series correlation feature extractor based on the convolutional neural network model, the river basin surface morphology feature extractor based on the convolutional neural network model, the feature map enhancer based on the re-parameterization layer, the meta-network-based multi-meteorological modality fusion device and the classifier.

9. The river flood early warning method based on machine learning and multi-meteorological modal fusion according to claim 8 is characterized in that: The training step comprises: Acquiring training data, the training data includes training satellite remote sensing images of the river basin, training weather station data, and whether a real value of a river flood warning prompt is generated, wherein the training weather station data includes a time series of training river water levels and a time series of training rainfall; Arranging the time series of the training river water level and the time series of the training rainfall into a training river water level time series input vector and a training rainfall time series input vector respectively according to the time dimension; Calculating a sample covariance association matrix of the training rainfall time series input vector relative to the training river water level time series input vector to obtain a training rainfall-river water level time series association matrix; The training rainfall-river water level time series association matrix is ​​passed through the rainfall-river water level time series association feature extractor based on the convolutional neural network model to obtain a training rainfall-river water level time series association feature vector; Performing feature extraction on the training satellite remote sensing image of the river basin by the river basin surface morphology feature extractor based on the convolutional neural network model to obtain a training river basin surface morphology feature map; Using the feature map enhancer based on the re-parameterized layer to process the training river basin surface morphology feature map to obtain a training river basin surface morphology enhanced feature map; Using the meta-network-based multi-meteorological modality fuser, multi-modal fusion is performed on the training river basin surface morphology enhancement feature map and the training rainfall-river water level time series correlation feature vector to obtain a training multi-meteorological modality fusion feature map; Optimizing the training multi-meteorological modality fusion feature map to obtain an optimized training multi-meteorological modality fusion feature map; Passing the optimized trained multi-meteorological modal fusion feature map through the classifier to obtain a classification loss function value; and The rainfall-river water level time series correlation feature extractor based on the convolutional neural network model, the river basin surface morphology feature extractor based on the convolutional neural network model, the feature map enhancer based on the re-parameterization layer, the meta-network-based multi-meteorological modality fusion device and the classifier are trained with the classification loss function value.

Images