Flood submerging position extraction method and system based on image retrieval

Abstract

The invention discloses a flood submerging position extraction method and system based on image retrieval. The method comprises the following steps: acquiring a street view image of a flood event; performing first feature extraction on the building part in the streetscape image to obtain a first image, and inputting the first image into a neural network model for secondary feature extraction to obtain a first image feature; and calculating the spatial similarity between the first image feature and the street view image database image feature of the flood city, and obtaining a database image position with the highest similarity with the first image as a positioning result. The invention provides an efficient and accurate flood inundation position extraction method, the problem of disaster situation positioning is effectively solved, powerful data and information support is provided for emergency response, disaster situation evaluation and disaster prevention and reduction work of flood disasters, and the disaster management efficiency and effect are greatly improved.

Description

Technical Field

[0001] The present invention relates to the technical field of flood disaster assessment, and particularly to a method and system for extracting flood inundation locations based on image retrieval. Background Art

[0002] The acquisition of flood disaster information and emergency management have become an urgent need for national development and an important guarantee for the sustainable development of the social economy and ecological environment. Related tasks such as the acquisition of disaster information and emergency response are inseparable from the support of data and information.

[0003] Traditional remote sensing satellite data and widely used social media data in recent years have both become common data sources in flood disaster research. Different data sources have their own advantages and disadvantages in flood application scenarios. For example, remote sensing data has a wide coverage area, and its accuracy is guaranteed as measured data, but it is limited by the revisit cycle and meteorological conditions, and the application of remote sensing data is restricted in some scenarios. In contrast, social media data has advantages such as a large amount and strong timeliness, but as non-authoritative data, its authenticity and accuracy remain to be verified. Therefore, according to the availability of data during actual floods, comprehensively using multi-source data and breaking through the performance bottleneck of a single data source helps to extract more accurate and comprehensive disaster information, which has important practical significance for disaster emergency response and disaster prevention and mitigation. Summary of the Invention

[0004] The purpose of this part is to outline some aspects of the embodiments of the present invention and briefly introduce some preferred embodiments. Simplifications or omissions may be made in this part, as well as in the abstract and title of the present application, to avoid obscuring the purpose of this part, the abstract, and the title, but such simplifications or omissions cannot be used to limit the scope of the present invention.

[0005] In view of the above existing problems, the present invention is proposed. Therefore, the present invention provides a method for extracting flood inundation locations based on image retrieval to solve the problem of difficult and rapid accurate positioning of inundation locations in flood disasters.

[0006] To solve the above technical problems, the present invention provides the following technical solutions:

[0007] In a first aspect, the present invention provides a method for extracting flood inundation locations based on image retrieval, including:

[0008] Obtain street view images of flood events;

[0009] Perform primary feature extraction on the building part in the street view image to obtain a first image, and input the first image into a neural network model for secondary feature extraction to obtain first image features;

[0010] Calculate the spatial similarity between the first image feature and the image features of the street view image database of the flood-affected city, and take the position of the database image with the highest similarity to the first image as the positioning result.

[0011] As a preferred solution of the method for extracting flood inundation positions based on image retrieval according to the present invention, wherein: obtaining the first image includes,

[0012] Input the street view image of the flood event into the trained segmentation model for semantic segmentation, extract the features of the buildings in the street view image, and intercept a rectangular image according to the extracted area to obtain the first image.

[0013] As a preferred solution of the method for extracting flood inundation positions based on image retrieval according to the present invention, wherein: inputting the first image into a neural network model for secondary feature extraction, and obtaining the first image feature includes,

[0014] Use the neural network model to extract the global feature and local feature of the first image;

[0015] Obtaining the global feature of the first image includes: performing a first convolution on the first image to obtain a feature descriptor, performing a second convolution on the feature descriptor to obtain a convolution feature, clustering the convolution feature, calculating the weighted average residual of each clustering center and its associated convolution feature, and normalizing the weighted average residual of each clustering center and all features to obtain a stable global feature of the first image;

[0016] Obtaining the local feature of the first image includes: using multiple different rectangles to extract features at a preset step length to obtain multi-scale local features of the first image.

[0017] As a preferred solution of the method for extracting flood inundation positions based on image retrieval according to the present invention, wherein: calculating the spatial similarity between the first image feature and the image features of the street view image database of the flood-affected city includes,

[0018] Calculate the Euclidean distance between the global feature of the first image and the global feature of the street view image database of the flood-affected city, perform a first screening, and obtain an image set with a relatively high similarity to the first image;

[0019] Calculate the similarity between the local feature descriptors of the image set and the local feature descriptors of the street view image database, perform a second screening, and use an intelligent algorithm to perform a third screening on the matching points to obtain a similarity score;

[0020] Take the position of the database image with the highest similarity score as the position of the flood event.

[0021] As a preferred solution of the method for extracting flood inundation locations based on image retrieval according to the present invention, wherein: performing a first screening includes,

[0022] Calculating the Euclidean distance between the global features of the first image and the global features of the images in the street view image database of the flood-affected city is expressed as:

[0023]

[0024] wherein, x i represents the i-th local feature of the first image, and x i represents the i-th local feature of the image in the street view image database, n is the feature dimension, and d is the calculated Euclidean distance result.

[0025] As a preferred solution of the method for extracting flood inundation locations based on image retrieval according to the present invention, wherein: clustering the convolutional features includes,

[0026] Using softmax to achieve soft assignment clustering;

[0027]

[0028] wherein, W*H*D is the feature descriptor, x i represents the i-th local feature, k represents the number of clustering centers, N represents the number of features, and w k represents the weight vector of the k-th clustering center, and b k represents the bias term of the k-th clustering center, and e is the natural constant.

[0029] As a preferred solution of the method for extracting flood inundation locations based on image retrieval according to the present invention, wherein: calculating the weighted average residual of each clustering center and its associated convolutional features is expressed by the formula:

[0030]

[0031] wherein, W*H*D is the feature descriptor, x i represents the i-th local feature, k represents the number of clustering centers, N represents the total number of local features, and w k represents the weight vector of the k-th clustering center, and b k represents the bias term of the k-th clustering center, V(:,k) is the weighted average residual, and c k represents a certain clustering center in the accumulation process.

[0032] In a second aspect, the present invention provides a system for extracting flood inundation locations based on image retrieval, including,

[0033] An image data acquisition module, configured to acquire street view images of flood events;

[0034] A feature extraction module, configured to perform first feature extraction on the building part in the street view image to obtain a first image, and input the first image into a neural network model for second feature extraction to obtain first image features;

[0035] A comparison and positioning module, configured to calculate the spatial similarity between the first image features and the image features of the street view image database of the flood-affected city, and use the position of the database image with the highest similarity to the first image as the positioning result.

[0036] In a third aspect, the present invention provides a computing device, including:

[0037] A memory and a processor;

[0038] The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions. When the computer-executable instructions are executed by the processor, the steps of the method for extracting flood inundation positions based on image retrieval are implemented.

[0039] In a fourth aspect, the present invention provides a computer-readable storage medium, which stores computer-executable instructions. When the computer-executable instructions are executed by a processor, the steps of the method for extracting flood inundation positions based on image retrieval are implemented.

[0040] Compared with the prior art, the beneficial effects of the present invention are as follows: By comprehensively using social media images and the urban street view image database, and combining advanced technologies such as image preprocessing, building segmentation, feature extraction, and similarity calculation, the present invention provides an efficient and accurate method for extracting flood inundation positions, effectively solving the problem of disaster situation positioning, and providing strong data and information support for flood disaster emergency response, disaster situation assessment, and disaster prevention and mitigation work, greatly improving the efficiency and effect of disaster management. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the following drawings are only some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts. Among them:

[0042] Figure 1 is a schematic diagram of the overall process of the method for extracting flood inundation positions based on image retrieval according to an embodiment of the present invention;

[0043] Figure 2 is a schematic diagram of the extraction of street view database points in a certain city by the method for extracting flood inundation positions based on image retrieval according to an embodiment of the present invention;

[0044] Figure 3 Schematic diagram of point 24 of the flood inundation location extraction method based on image retrieval according to an embodiment of the present invention;

[0045] Figure 4 Schematic diagram of the image building segmentation and cropping method flow of the flood inundation location extraction method based on image retrieval according to an embodiment of the present invention;

[0046] Figure 5 Schematic diagram of the image global feature extraction process of the flood inundation location extraction method based on image retrieval according to an embodiment of the present invention;

[0047] Figure 6 Schematic diagram of the method for predicting using local feature similarity ranking of the flood inundation location extraction method based on image retrieval according to an embodiment of the present invention; Detailed implementation manners

[0048] In order to make the above objects, features, and advantages of the present invention more obvious and understandable, the following will describe the detailed implementation manners of the present invention with reference to the accompanying drawings of the specification. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0049] In the following description, many specific details are set forth in order to fully understand the present invention. However, the present invention can also be implemented in other ways different from those described herein. Those skilled in the art can make similar generalizations without departing from the connotation of the present invention. Therefore, the present invention is not limited by the specific embodiments disclosed below.

[0050] Secondly, the so-called "one embodiment" or "embodiment" herein refers to a specific feature, structure, or characteristic that can be included in at least one implementation manner of the present invention. The "in one embodiment" that appears in different places in this specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment that excludes other embodiments.

[0051] The present invention is described in detail with reference to the schematic diagrams. When detailing the embodiments of the present invention, for the sake of clarity, the cross-sectional views showing the device structure will be enlarged locally out of the general scale, and the schematic diagrams are only examples and should not limit the protection scope of the present invention herein. In addition, in actual production, three-dimensional spatial dimensions including length, width, and depth should be included.

[0052] In the description of the present invention, it should be noted that the orientation or positional relationship indicated by terms such as "upper, lower, inner, and outer" is based on the orientation or positional relationship shown in the drawings. It is only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the system or component referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be construed as a limitation to the present invention. In addition, the terms "first, second, or third" are only used for descriptive purposes and cannot be understood as indicating or implying relative importance.

[0053] Unless otherwise clearly defined and limited in the present invention, the terms "installation, connection, and coupling" shall be understood in a broad sense. For example, it can be a fixed connection, a detachable connection, or an integral connection; it can also be a mechanical connection, an electrical connection, or a direct connection, and can also be indirectly connected through an intermediate medium, or can be the communication inside two components. For those of ordinary skill in the art, the specific meanings of the above terms in the present invention can be understood according to specific situations.

[0054] Embodiment 1

[0055] Referring to Figure 1 , an embodiment of the present invention provides a method for extracting flood inundation positions based on image retrieval, including:

[0056] S100: Obtain the street view images of the flood event;

[0057] S102: Perform primary feature extraction on the building part in the street view image to obtain a first image, and input the first image into a neural network model for secondary feature extraction to obtain the first image features;

[0058] S104: Calculate the spatial similarity between the first image features and the image features of the street view image database of the flood city, and obtain the position of the database image with the highest similarity to the first image as the positioning result;

[0059] It should be noted that through the primary feature extraction and the secondary feature extraction of the neural network model, the present invention can efficiently capture the key information of the buildings in the street view image and form highly recognizable first image features. Subsequently, by calculating the spatial similarity between these features and the image features in the street view image database of the flood city, the present invention can quickly lock the position of the database image that best matches the first image, thereby achieving accurate and rapid positioning of the flood inundation position. Through the steps of S100 to S104, the present invention realizes the precise processing and analysis of the street view images in the flood event.

[0060] The method for obtaining the street view images of the flood event in step S100 above can be through official channels such as government open data, emergency management departments, web crawler technology, cooperation with map service providers and professional image libraries, etc.;

[0061] In a possible embodiment, when the image acquisition method is web crawler technology, the specific process of obtaining the street view images of flood events is as follows: Use web crawler technology to obtain the Sina Weibo data related to floods during a specific flood occurrence cycle. The Weibo fields include the posting time, posting location, and Weibo image content. Clean and deduplicate the data to obtain the flood-related images. Through data screening and verification criteria, obtain the street view images of flood events that can be used for geolocation;

[0062] In a possible embodiment, the data screening and verification criteria may include but are not limited to:

[0063] ① There are landmark buildings in the Weibo image;

[0064] ② There are detailed address texts in the Weibo image;

[0065] ③ Deduplicate the duplicate data of the same street and delete the duplicate point data;

[0066] ④ There should be obvious flood characteristics in the Weibo image, which can clearly reflect the actual inundation situation for verifying the authenticity and accuracy of the data.

[0067] In the embodiment of the present application, the above step S102 of obtaining the first image feature may include the following sub-steps A1 - A2;

[0068] In A1: Input the street view images of flood events into the trained segmentation model for semantic segmentation, extract the features of the buildings in the street view images, and intercept a rectangular image according to the extracted area to obtain the first image;

[0069] The above-mentioned segmentation model includes a semantic segmentation network based on deep learning, such as PSPNet, U-Net, Mask R-CNN, DeepLab series, etc. These models are trained with a large number of street view images and can accurately identify and segment the building areas in the images.

[0070] In a possible embodiment, when the segmentation model selects the PSPNet model, the process of obtaining the first image is as follows: Use a large number of street view images with labeled building areas as the training data set, and train the PSPNet model through supervised learning; after training, input the street view images of flood events to be segmented into the trained PSPNet model for semantic segmentation. The PSPNet model automatically identifies and segments the buildings in the image. According to the four boundary values of the top, bottom, left, and right of the building area in the segmentation result, intercept a rectangular image that includes the entire building and avoids interference from irrelevant backgrounds as the final preprocessed image, that is, the first image;

[0071] It should be noted that the PSPNet model fuses context information of different scales through the pyramid pooling module, so as to be able to understand complex scenes in images more accurately and accurately segment the building area.

[0072] In A2: Use the neural network model to extract the global features and local features of the first image to obtain the first image features;

[0073] In the embodiment of the present application, obtaining the global features of the first image includes: performing a first convolution on the first image to obtain a feature descriptor, performing a second convolution on the feature descriptor to obtain a convolution feature, clustering the convolution feature, calculating the weighted average residual of each cluster center and its associated convolution feature, and normalizing the weighted average residual of each cluster center and all features to obtain stable global features of the first image;

[0074] The above-mentioned normalization methods include Min-Max normalization, L2 norm normalization, Z-score normalization, etc.;

[0075] In the embodiment of the present application, obtaining the local features of the first image includes: using a plurality of different rectangles to extract the features according to a preset step size to obtain multi-scale local features of the first image.

[0076] In the embodiment of the present application, softmax is used to implement soft assignment clustering, which is expressed by the formula:

[0077]

[0078] Among them, W*H*D is the feature descriptor, x i represents the i-th local feature, k represents the number of cluster centers, N represents the number of features, w k represents the weight vector of the k-th cluster center, b k represents the bias term of the k-th cluster center, and e is the natural constant.

[0079] In the embodiment of the present application, calculating the weighted average residual of each cluster center and its associated convolution feature is expressed by the formula:

[0080]

[0081] Among them, W*H*D is the feature descriptor, x i represents the i-th local feature, k represents the number of cluster centers, N represents the total number of local features, w k represents the weight vector of the k-th cluster center, b k represents the bias term of the k-th cluster center, V(:,k) is the weighted average residual, and c k represents a certain cluster center in the accumulation process.

[0082] In a possible embodiment, when the segmentation model selects the PSPNet model and the normalization method selects the L2 norm, the process of obtaining the first image features is as follows: perform semantic segmentation on the street view images of all flood events, find the four boundary points of the upper, lower, left, and right of the predicted pixel points as buildings in the image, and use these four points as the upper, lower, left, and right boundaries to cut the image to obtain the buildings in the image, so as to obtain the first image. Divide all the first images into two types: a query set and a database set, and match positive and negative samples for each first image of the flood event in the query set. The positive sample is a random image within 10 m around the query image, and the negative samples are five images outside 50 m around the query image;

[0083] Use the conv5 layer to perform preliminary feature extraction on all the first images and calculate the clustering centers at the same time; use the soft clustering method to compress the image features into a global feature of a specific size, use a smooth weight function to calculate the distance between each local feature and the clustering center, then normalize the features of each center point respectively to erase the size of the residuals in the clustering, and finally perform the last normalization on the overall data to obtain the final global feature. Based on the global feature, set rectangular patch blocks of different sizes and the step sizes of each patch block, and extract local features according to these patch blocks in the global feature.

[0084] It should be noted that the sizes and clarity of the images obtained when collecting street view data are the same. However, since the images were cut by the building part in the previous step, the sizes of each input image are different. Therefore, all the images are uniformly sized before using the neural network to extract image features.

[0085] It should also be noted that the building area is accurately segmented from the image by the segmentation model, removing the interference of irrelevant backgrounds, providing high-quality image data for subsequent feature extraction work; based on the segmented building images, use the neural network model to deeply extract global and local features. The global feature extraction process focuses on capturing the overall structural information of the building, while the local feature extraction pays more attention to the detailed texture of the image. The complementarity of the global feature and the local feature jointly constitutes a comprehensive description of the building features. This comprehensive and in-depth feature extraction method not only improves the understanding ability of the building features but also provides a more reliable and efficient feature basis for subsequent image matching.

[0086] In the embodiment of the present application, the positioning result of the flood street view image position obtained in the above step S106 further includes the following sub-steps B1 - B3:

[0087] In B1: Calculate the Euclidean distance between the global features of the first image and the global features of the images in the street view image database of the flood-affected city, perform a primary screening, and obtain an image set with a relatively high similarity to the first image;

[0088] It should be noted that in terms of the urban street view image database, by combining the street view map with physical perception, determine the disaster-affected locations of social media images, use ARCGIS PRO to divide the severely disaster-affected areas as the database point range, then generate street points at an interval of 15m, then convert the WGS84 coordinates to the Baidu coordinate system (BD-09), and finally use web crawlers to obtain the street view images of the street sampling points on Baidu Map.

[0089] In B2: Calculate the similarity between the local feature descriptors of the image set and the local feature descriptors of the images in the street view image database, and perform a secondary screening;

[0090] It should be noted that the secondary screening is to calculate the spatial similarity between images using the local features of the first image intercepted by patch blocks of different rectangle sizes;

[0091] In B3: Use an intelligent algorithm to perform a tertiary screening on the matching points to obtain a similarity score, and take the position of the database image with the highest similarity score as the position of the flood event.

[0092] In the embodiment of the present application, the calculation of the Euclidean distance between the global features of the first image and the global features of the images in the street view image database of the flood-affected city is expressed by the formula:

[0093]

[0094] where x i represents the i-th local feature of the first image, x i represents the i-th local feature of the image in the street view image database, n is the feature dimension, and d is the calculated Euclidean distance result.

[0095] The above-mentioned use of an intelligent algorithm for tertiary screening, and the intelligent algorithm can be methods such as the RANSAC algorithm, convolutional neural network CNN, recurrent neural network RNN or its variants such as long short-term memory network LSTM, etc.;

[0096] In a possible embodiment, when the intelligent algorithm selects the RANSAC algorithm, the process of triple screening is as follows: randomly extract a set of sample points from the key point sets of the first image and the candidate database images, calculate a preliminary homography matrix using these sample points, and the RANSAC algorithm uses the unitary matrix to test other key points in the database images to determine whether they can be correctly mapped to the corresponding positions in the first image, thereby classifying these points as inliers or outliers. Through multiple iterations, the algorithm will continuously update and record the homography matrix containing the most inliers and its corresponding inlier set;

[0097] To evaluate the matching degree between each database image and the first image, the number of inliers in each inlier set is also calculated and normalized to obtain a final score. After completing the matching calculations for all database images, the image with the highest score is output as the final matching image, and the location of the database image is queried to obtain the specific location of the flood disaster.

[0098] It should be noted that the present invention adopts a multi-step strategy to accurately locate flood street view images. By calculating the Euclidean distance of global features, an image set with high similarity to the first image is quickly screened; local features are intercepted using different rectangular patches, and the similarity is calculated for secondary screening to enhance the matching accuracy; then an intelligent algorithm is used for triple screening to achieve the accurate location of the flood street view image position.

[0099] The above is a schematic solution of a method for extracting flood inundation locations based on image retrieval in this embodiment. It should be noted that the technical solution of the system for extracting flood inundation locations based on image retrieval belongs to the same concept as the technical solution of the above method for extracting flood inundation locations based on image retrieval. For the details not described in detail in the technical solution of the system for extracting flood inundation locations based on image retrieval in this embodiment, reference can be made to the description of the technical solution of the method for extracting flood inundation locations based on image retrieval.

[0100] The system for extracting flood inundation locations based on image retrieval in this embodiment includes:

[0101] An image data acquisition module for acquiring street view images of flood events;

[0102] A feature extraction module for performing primary feature extraction on the building part in the street view image to obtain a first image, and inputting the first image into a neural network model for secondary feature extraction to obtain first image features;

[0103] A comparison and positioning module for calculating the spatial similarity between the first image features and the database image features of the street view images in the flood city, and obtaining the location of the database image with the highest similarity to the first image as the positioning result.

[0104] This embodiment also provides a computing device, which is applicable to the situation of flood inundation location extraction based on image retrieval, and includes:

[0105] A memory and a processor; the memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions to implement the method for flood inundation location extraction based on image retrieval as proposed in the above embodiment.

[0106] This embodiment also provides a storage medium, on which a computer program is stored, and when the program is executed by a processor, it implements the method for flood inundation location extraction based on image retrieval as proposed in the above embodiment.

[0107] The storage medium proposed in this embodiment and the method for flood inundation location extraction based on image retrieval proposed in the above embodiment belong to the same inventive concept. For technical details not described in detail in this embodiment, reference can be made to the above embodiment, and this embodiment has the same beneficial effects as the above embodiment.

[0108] From the above description of the embodiments, those skilled in the art can clearly understand that the present invention can be implemented by means of software and necessary general-purpose hardware. Of course, it can also be implemented by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as a floppy disk, read-only memory (ROM), random access memory (RAM), flash memory (FLASH), hard disk, or optical disc of a computer, etc., including several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods of the various embodiments of the present invention.

[0109] Embodiment 2

[0110] Referring to Figures 2 - 6 , as an embodiment of the present invention, a method for flood inundation location extraction based on image retrieval is provided. In order to verify its beneficial effects, the verification results of the method of the present invention are provided.

[0111] Taking an event in a certain area as an example, affected by the large amount of water vapor transported by Typhoon In-Fa and the combined action of local topography, a certain province has encountered extreme heavy rainfall, and many places in the province have encountered extremely heavy rainstorms rarely seen in history.

[0112] To specifically locate the disaster area, it is necessary to first collect all the street view images in the surrounding areas where the disaster is likely to occur. First, a certain number of flood disaster images in a certain city are collected, and the images on Sina Weibo are used as the main source for collecting disaster images.

[0113] Collect 20 flood microblog images of areas that can be accurately pinpointed as verification images. Divide and equally sample the surrounding areas of these flood points as 3024 collection points for the street view database, as Figure 2 shown. In this way, all flood image points can be included in the database. In the collection of database images, the present invention uses the Baidu Map Developer API to collect images. Each point will have 24 pictures, as Figure 3 shown, for a total of 72,576 street view images as database images.

[0114] Use a segmentation model to preprocess the image data for building segmentation, as Figure 4 shown. The PSPNET model used for semantic segmentation preprocessing of images and the convolutional neural network model used for graphic feature extraction have both been pre-trained on the ImageNet dataset. The convolutional neural network model is as Figure 5 shown, and then used in the present invention to accelerate model convergence. All experiments are carried out using PyTorch on the Windows system. First, during the training process, all street view image datasets are divided into a training set and a test set in a ratio of 8:2. At the same time, the data in the training set and the test set is further divided into query images and database images in a ratio of 8:2. When preprocessing the images before inputting them into the network, all images are adjusted to a size of 640*480*3. The model is trained using the SGD optimizer for 50 epochs, with a learning rate of 1e-4. The loss function uses TripletMarginLoss. When using clustering, the clustering size is set to 16, the number of negative samples for each image is set to 5, and when extracting local features, the patch sizes are set to 2*2, 5*5, 8*8, and the stride is 1 for all. In the selection of the feature extraction network, the present invention compares networks such as convolutional neural network models and ResNet, and finally selects the convolutional neural network model with better performance as the feature extraction network. At the same time, the present invention conducts a sensitivity analysis of the basic parameters for multiple groups of values to determine the specific values of each hyperparameter. When using the hyperparameters listed above, the model can obtain better performance. The results obtained using the convolutional neural network model are as Figure 6 shown.

[0115] The present invention mainly analyzes the flood disaster situation in a certain city. The recall rate is selected as the evaluation index in the proposed street view database of the city. In the prediction results, the prediction within 10 meters of the true coordinates is regarded as a correct prediction. At the same time, the top one, top five, and top ten similarity ranking scales are used to observe the prediction accuracy of this method. The final recall rates of the three scales are 60.37%, 77.88%, and 84.33% respectively.

[0116] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit them. Although the present invention has been described in detail with reference to the preferred embodiments, those of ordinary skill in the art should understand that the technical solutions of the present invention can be modified or equivalently replaced without departing from the spirit and scope of the technical solutions of the present invention, and they should all be covered within the scope of the claims of the present invention.

Claims

1. A method for extracting flood inundation locations based on image retrieval, characterized in that, Including: Obtain the street view images of flood events; Perform primary feature extraction on the building part in the street view image to obtain a first image, input the first image into a neural network model for secondary feature extraction, and obtain first image features; Calculate the spatial similarity between the first image features and the image features of the street view image database in the flood city, and use the position of the database image with the highest similarity to the first image as the positioning result.

2. The method for extracting flood inundation locations based on image retrieval according to claim 1, wherein Obtaining the first image includes, Input the street view image of the flood event into a trained segmentation model for semantic segmentation, extract features of the buildings in the street view image, and intercept a rectangular image according to the extracted area to obtain the first image.

3. The method for extracting flood inundation locations based on image retrieval according to claim 1 or 2, characterized in that, Inputting the first image into the neural network model for secondary feature extraction and obtaining first image features includes, Using the neural network model to extract the global features and local features of the first image; Obtaining the global features of the first image includes: performing a first convolution on the first image to obtain a feature descriptor, performing a second convolution on the feature descriptor to obtain convolution features, clustering the convolution features, calculating the weighted average residual between each cluster center and its associated convolution features, and normalizing the weighted average residuals of each cluster center and all features to obtain stable global features of the first image; Obtaining the local features of the first image includes: using multiple different rectangles to extract features at a preset step size to obtain multi-scale local features of the first image.

4. The method for extracting flood inundation locations based on image retrieval according to claim 3, characterized in that, Calculating the spatial similarity between the first image features and the image features of the street view image database in the flood city includes, Calculating the Euclidean distance between the global features of the first image and the global features of the street view image database in the flood city, performing a first screening, and obtaining an image set with a relatively high similarity to the first image; Calculating the similarity between the local feature descriptors of the image set and the local feature descriptors of the street view image database in the flood city, performing a second screening, and using an intelligent algorithm to perform a third screening on the matching points to obtain a similarity score; Using the position of the database image with the highest similarity score as the position of the flood event.

5. The method for extracting flood inundation locations based on image retrieval according to claim 4, characterized in that, Performing a first screening includes, Calculating the Euclidean distance between the global features of the first image and the global features of the street view image database in the flood city is expressed as: where x i represents the i-th local feature of the first image, and x i represents the i-th local feature of the image in the street view image database. n is the feature dimension, and d is the calculated Euclidean distance result.

6. The method for extracting flood inundation locations based on image retrieval according to claim 3, characterized in that Clustering the convolution features includes, Using softmax to implement soft assignment clustering; Among them, W*H*D is the feature descriptor, x i represents the i-th local feature, k represents the number of clustering centers, N represents the number of features, w k represents the weight vector of the k-th clustering center, b k represents the bias term of the k-th clustering center, and e is the natural constant.

7. The method for extracting flood inundation locations based on image retrieval according to claim 6, wherein Calculating the weighted average residual between each cluster center and its associated convolution features is expressed by the formula: Among them, W*H*D is the feature descriptor, x i represents the i-th local feature, k represents the number of cluster centers, N represents the total number of local features, w k represents the weight vector of the k-th cluster center, b k represents the bias term of the k-th cluster center, V(:,k) is the weighted average residual, c k represents a certain cluster center in the accumulation process.

8. A system for extracting flood inundation locations based on image retrieval, characterized in that, Including, An image data acquisition module, used to obtain the street view images of flood events; A feature extraction module, used to perform primary feature extraction on the building part in the street view image to obtain a first image, input the first image into a neural network model for secondary feature extraction, and obtain first image features; A comparison and positioning module, used to calculate the spatial similarity between the first image features and the image features of the street view image database in the flood city, and use the position of the database image with the highest similarity to the first image as the positioning result.

9. An electronic device, including: A memory and a processor; The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions. When the computer-executable instructions are executed by the processor, the steps of the method for extracting flood inundation locations based on image retrieval according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium stores computer-executable instructions. When the computer-executable instructions are executed by a processor, the steps of the method for extracting flood inundation locations based on image retrieval according to any one of claims 1 to 7 are implemented.

Images