Water body identification method, device, electronic device and storage medium

Abstract

The present invention provides a water body recognition method, device, electronic device and storage medium, wherein the method comprises: obtaining visible light image data to be detected; inputting the visible light image data to be detected into a water body recognition model to obtain the confidence that each pixel point output by the water body recognition model is a water body; determining the target water body area in the visible light image data to be detected based on the confidence that each pixel point is a water body; wherein the water body recognition model is obtained by training an initial network model based on image sample data; the initial network model includes network layer connections of multiple preset paths. The water body recognition method, device, electronic device and storage medium provided by the present invention can increase the number of connection layers in the network, and the increase in the number of connection layers means that more convolutional features are retained, thereby improving the accuracy of water body recognition.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a water body recognition method, device, electronic equipment and storage medium. Background Art

[0002] Water resources are not only closely related to human production and life, but also to the sustainable and stable development of urban economy. Therefore, it is particularly important to identify urban water bodies and dynamically analyze the status of urban water bodies.

[0003] In related technologies, water areas in image data are usually identified through deep learning models such as Convolution Neural Network (CNN) or Image Semantic Segmentation Network (SegNet). However, the above deep learning models are limited by the depth of the network layer and learn fewer features, which reduces the accuracy of water body identification by the deep learning models. Summary of the invention

[0004] In view of the problems existing in the prior art, the present invention provides a water body identification method, device, electronic device and storage medium.

[0005] The present invention provides a water body identification method, comprising:

[0006] Acquire visible light image data to be detected;

[0007] Inputting the visible light image data to be detected into a water body recognition model to obtain the confidence that each pixel point output by the water body recognition model is a water body;

[0008] Determine the target water body area in the visible light image data to be detected based on the confidence that each pixel point is a water body;

[0009] Wherein, the water body recognition model is obtained by training an initial network model based on image sample data; the initial network model includes network layer connections of multiple preset paths.

[0010] According to a water body recognition method provided by the present invention, before inputting the visible light image data to be detected into a water body recognition model to obtain the confidence that each pixel point output by the water body recognition model is a water body, the method further includes:

[0011] Construct a recurrent neural network based on a cross-network path of multiple upsampling layers and multiple pooling layers;

[0012] The initial network model is constructed based on the recurrent neural network and multiple convolutional layers.

[0013] According to a water body recognition method provided by the present invention, the cross network path of the multiple upsampling layers and the multiple pooling layers includes a first upsampling layer, a first pooling layer, a second pooling layer, a second upsampling layer, a third pooling layer, a third upsampling layer, a fourth upsampling layer and a fourth pooling layer in sequence;

[0014] The method of constructing a recurrent neural network based on a cross network path of multiple upsampling layers and multiple pooling layers includes:

[0015] A combination of the first upsampling layer, the first pooling layer, the second pooling layer, the second upsampling layer, the third pooling layer, the third upsampling layer, the fourth upsampling layer and the fourth pooling layer is determined as the recurrent neural network.

[0016] According to a water body recognition method provided by the present invention, the initial network model is constructed based on the recurrent neural network and multiple convolutional layers, including:

[0017] A first convolutional layer and a second convolutional layer are sequentially arranged between the first upsampling layer and the network input layer;

[0018] A third convolutional layer and a fourth convolutional layer are sequentially arranged between the first upsampling layer and the first pooling layer;

[0019] A fifth convolutional layer and a sixth convolutional layer are sequentially arranged between the first pooling layer and the second pooling layer;

[0020] A seventh convolutional layer and an eighth convolutional layer are sequentially arranged between the second pooling layer and the second upsampling layer;

[0021] A ninth convolutional layer and a tenth convolutional layer are sequentially arranged between the second upsampling layer and the third pooling layer;

[0022] An eleventh convolutional layer and a twelfth convolutional layer are sequentially arranged between the third pooling layer and the third upsampling layer;

[0023] A thirteenth convolutional layer and a fourteenth convolutional layer are sequentially arranged between the third upsampling layer and the fourth upsampling layer;

[0024] A fifteenth convolutional layer and a sixteenth convolutional layer are sequentially arranged between the fourth upsampling layer and the fourth pooling layer;

[0025] A seventeenth convolutional layer, an eighteenth convolutional layer and a nineteenth convolutional layer are sequentially arranged between the fourth pooling layer and the network output layer.

[0026] According to a water body identification method provided by the present invention, the network layer connection of the multiple preset paths includes:

[0027] Connecting the second convolutional layer to the first pooling layer;

[0028] Connecting the sixth convolutional layer to the second upsampling layer;

[0029] Connecting the tenth convolutional layer to the third upsampling layer;

[0030] Connecting the fourteenth convolutional layer to the fourth pooling layer;

[0031] The fourth convolutional layer is connected to the fourth upsampling layer.

[0032] According to a water body identification method provided by the present invention, after constructing the initial network model based on the recurrent neural network and multiple convolutional layers, the method further includes:

[0033] Processing the image sample data to obtain a gray level co-occurrence matrix sample of each pixel in the image sample data;

[0034] The initial network model is trained based on the entropy of the gray-level co-occurrence matrix sample of each pixel and the image sample data to obtain the water body recognition model.

[0035] According to a water body recognition method provided by the present invention, the image sample data is processed to obtain a gray level co-occurrence matrix sample of each pixel in the image sample data, including:

[0036] Preprocessing the image sample data to obtain an orthophoto sample;

[0037] Performing grayscale processing on the orthophoto sample to obtain a grayscale image sample;

[0038] The gray level co-occurrence matrix sample of each pixel is determined based on the gray level image sample.

[0039] According to a water body recognition method provided by the present invention, the gray level co-occurrence matrix sample of each pixel point is determined based on the gray level image sample, comprising:

[0040] Performing linear grayscale compression on the grayscale image samples;

[0041] The gray-level co-occurrence matrix sample of each pixel is determined based on the compressed gray-level image sample.

[0042] According to a water body recognition method provided by the present invention, the gray level co-occurrence matrix sample of each pixel is determined based on the compressed gray level image sample, comprising:

[0043] Determine the target neighborhood range of each pixel in the compressed grayscale image sample;

[0044] For each of the target neighborhoods of the pixel points, determining the number of occurrences of each pixel pair on a preset vector within the target neighborhood;

[0045] The gray level co-occurrence matrix samples of the corresponding pixel points are determined based on the number of occurrences of each pixel pair.

[0046] According to a water body recognition method provided by the present invention, the visible light image data to be detected is input into a water body recognition model to obtain the confidence that each pixel point output by the water body recognition model is a water body, including:

[0047] Processing the visible light image data to be detected to obtain a gray level co-occurrence matrix of each pixel in the visible light image data to be detected;

[0048] The entropy of each gray-level co-occurrence matrix and the visible light image data to be detected are input into the water body recognition model to obtain the confidence that each pixel point output by the water body recognition model is a water body.

[0049] The present invention also provides a water body identification device, comprising:

[0050] An acquisition unit, used for acquiring visible light image data to be detected;

[0051] An identification unit, used for inputting the visible light image data to be detected into a water body identification model to obtain the confidence that each pixel point output by the water body identification model is a water body;

[0052] A determination unit, configured to determine a target water body region in the visible light image data to be detected based on the confidence level that each pixel point is a water body;

[0053] The water body recognition model is obtained by training an initial network model based on image sample data; the initial network model includes a cross network path of multiple upsampling layers and multiple pooling layers.

[0054] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the steps of any of the above-mentioned water body identification methods are implemented.

[0055] The present invention also provides a non-transitory computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps of any of the water body identification methods described above are implemented.

[0056] The present invention also provides a computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the steps of any of the above-mentioned water body identification methods are implemented.

[0057] The present invention provides a water body recognition method, device, electronic device and storage medium, which inputs the acquired visible light image data to be detected into a water body recognition model to obtain the confidence that each pixel point output by the water body recognition model is a water body. Since the water body recognition model in the present invention is obtained by training based on an initial network model including network layer connections of multiple preset paths, and the network layer connections of multiple preset paths can increase the number of connection layers in the network, the increase in the number of connection layers means that more convolution features are retained, thereby improving the accuracy of water body recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0059] Figure 1 This is one of the flow charts of the water body identification method provided by the present invention;

[0060] Figure 2 This is the second flow chart of the water body identification method provided by the present invention;

[0061] Figure 3 It is a structural schematic diagram of the initial network model provided by the present invention;

[0062] Figure 4 This is the third flow chart of the water body identification method provided by the present invention;

[0063] Figure 5 It is one of the schematic diagrams of grayscale image sample processing provided by the present invention;

[0064] Figure 6 This is the second schematic diagram of grayscale image sample processing provided by the present invention;

[0065] Figure 7 It is a structural schematic diagram of the water body identification device provided by the present invention;

[0066] Figure 8 It is a schematic diagram of the physical structure of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0067] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0068] Combine the following Figure 1-Figure 6 The water body identification method of the present invention is described.

[0069] Figure 1 It is one of the flow charts of the water body identification method provided by the present invention, such as Figure 1 As shown, the water body identification method includes the following steps:

[0070] Step 101: Obtain visible light image data to be detected.

[0071] For example, a drone is used to collect visible light image data of a certain area (such as an urban area or a suburban area), that is, the visible light image data to be detected is obtained.

[0072] Step 102: input the visible light image data to be detected into a water body recognition model to obtain the confidence that each pixel point output by the water body recognition model is a water body.

[0073] Wherein, the water body recognition model is obtained by training an initial network model based on image sample data; the initial network model includes network layer connections of multiple preset paths.

[0074] For example, when the visible light image data to be detected is obtained, the visible light image data to be detected is input into a pre-trained water body recognition model, and the water body recognition model extracts the features of each pixel point in the visible light image data to be detected, and determines the probability of each pixel point being a water body based on all the extracted features, and finally outputs the confidence that each pixel point is a water body.

[0075] It should be noted that the water body recognition model is pre-trained based on image sample data. The specific training method is: first, an initial network model is constructed, and then the image sample data is input into the initial network model to obtain the predicted confidence that each pixel point output by the initial network model is water; then the difference between the predicted confidence that each pixel point is water and the true value of whether each pixel point is water is determined, and the initial network model is optimized based on the difference until the model parameters of the initial network model reach the convergence conditions to obtain the water body recognition model.

[0076] Step 103: determine the target water body area in the visible light image data to be detected based on the confidence that each pixel point is a water body.

[0077] For example, when the confidence that each pixel is a water body is obtained, the confidence that each pixel is a water body is compared with the preset confidence, and the pixel points whose confidence that the water body is greater than or equal to the preset confidence are determined to be water bodies, and the pixel points whose confidence that the water body is less than the preset confidence are determined to be not water bodies. Finally, all the pixel points determined to be water bodies are combined to obtain the target water body area.

[0078] It is understandable that the area of ​​urban areas or suburban areas is large, so when identifying water bodies in urban areas or suburban areas, the visible light image data to be detected in a certain area collected by the drone can be divided into data corresponding to multiple images, and then the data corresponding to each image can be input into the water body recognition model for water body recognition. Finally, the water body areas corresponding to each image are combined to obtain the target water body area corresponding to the overall visible light image data to be detected.

[0079] Furthermore, when the target water body area corresponding to the visible light image data to be detected is obtained, the target water body area is loaded into mapping software such as the Geographic Information System (GIS). In the mapping software, the target water body area can be divided into multiple areas, and each area can be marked with a different color; map components (for example, a compass, a scale, and a legend, etc.) can also be added; the water body recognition accuracy can also be marked, and finally a water system map of the corresponding area is obtained.

[0080] The water body recognition method provided by the present invention inputs the acquired visible light image data to be detected into the water body recognition model to obtain the confidence that each pixel point output by the water body recognition model is a water body. Since the water body recognition model in the present invention is obtained by training based on an initial network model including a network layer connection of multiple preset paths, and the network layer connection of multiple preset paths can increase the number of connection layers in the network, the increase in the number of connection layers means that more convolution features are retained, thereby improving the accuracy of water body recognition.

[0081] Optionally, Figure 2 This is the second flow chart of the water body identification method provided by the present invention, such as Figure 2 As shown, before executing step 102, the following steps are also included:

[0082] Step 104: construct a recurrent neural network based on the cross network paths of multiple upsampling layers and multiple pooling layers.

[0083] Among them, the cross-network path of the multiple upsampling layers and the multiple pooling layers includes a first upsampling layer, a first pooling layer, a second pooling layer, a second upsampling layer, a third pooling layer, a third upsampling layer, a fourth upsampling layer and a fourth pooling layer in sequence.

[0084] Optionally, a combination of the first upsampling layer, the first pooling layer, the second pooling layer, the second upsampling layer, the third pooling layer, the third upsampling layer, the fourth upsampling layer and the fourth pooling layer is determined as the recurrent neural network.

[0085] Step 105: construct the initial network model based on the recurrent neural network and multiple convolutional layers.

[0086] Optionally, a first convolutional layer and a second convolutional layer are sequentially arranged between the first upsampling layer and the network input layer; a third convolutional layer and a fourth convolutional layer are sequentially arranged between the first upsampling layer and the first pooling layer; a fifth convolutional layer and a sixth convolutional layer are sequentially arranged between the first pooling layer and the second pooling layer; a seventh convolutional layer and an eighth convolutional layer are sequentially arranged between the second pooling layer and the second upsampling layer; a ninth convolutional layer and a tenth convolutional layer are sequentially arranged between the second upsampling layer and the third pooling layer; an eleventh convolutional layer and a twelfth convolutional layer are sequentially arranged between the third pooling layer and the third upsampling layer; a thirteenth convolutional layer and a fourteenth convolutional layer are sequentially arranged between the third upsampling layer and the fourth upsampling layer; a fifteenth convolutional layer and a sixteenth convolutional layer are sequentially arranged between the fourth upsampling layer and the fourth pooling layer; and a seventeenth convolutional layer, an eighteenth convolutional layer and a nineteenth convolutional layer are sequentially arranged between the fourth pooling layer and the network output layer.

[0087] Among them, the upsampling layer is a deconvolution layer. Each convolution layer and each upsampling layer is followed by a linear rectifier function (Rectified Linear Unit, ReLU) operation. ReLU, as the activation function of the neuron, defines the nonlinear output result of the neuron after linear transformation; the upsampling layer is used to enlarge the image; each convolution layer is used to extract the feature information of the data input into the network input layer, and the pooling layer is used to speed up the convergence of the model.

[0088] For example, Figure 3 It is a structural diagram of the initial network model provided by the present invention, such as Figure 3As shown in the figure, the first convolutional layer to the nineteenth convolutional layer are numbered J1 to J19, and all convolutional layers are represented by strip boxes filled with dots; the first pooling layer to the fourth pooling layer are numbered C1 to C4, and all pooling layers are represented by strip boxes filled with oblique grids; the first upsampling layer to the fourth upsampling layer are numbered S1 to S4, and all upsampling layers are represented by white strip boxes, SR represents the network input layer, and the network input layer can be a 256*256*c image, where c is the number of channels, and the value of c is different for different experiments; SC represents the network output layer, and each feature vector is mapped using a 1*1 convolutional layer. The network output layer finally outputs the confidence that each pixel of 256*256*1 is a water body.

[0089] It should be noted that step 104 and step 105 may be performed before step 101 or before step 102, and the present invention does not limit this.

[0090] The water body recognition method provided by the present invention constructs an initial network model based on a cross-network path of multiple upsampling layers and multiple pooling layers, so that the water body recognition model finally trained can obtain the scale of different image sizes, thereby improving the recognition accuracy of the water body recognition model.

[0091] Furthermore, if Figure 3 As shown, the network layer connections of the multiple preset paths specifically include: the second convolution layer is connected to the first pooling layer, the sixth convolution layer is connected to the second upsampling layer, the tenth convolution layer is connected to the third upsampling layer, the fourteenth convolution layer is connected to the fourth pooling layer, and the fourth convolution layer is connected to the fourth upsampling layer. Figure 3 The solid arrows in the middle represent all the connection operations. More connections mean more convolutional features are retained.

[0092] Specifically, the initial network model of the present invention is improved on the basis of the U-Net model, and the operations of multiple pooling layers and multiple upsampling layers are cross-performed, and the network layers of the above multiple preset paths are connected, so that more connection fusion can occur to retain more convolution feature information. For example, Figure 3 The second convolutional layer J2 is connected to the first pooling layer C1, and the first pooling layer C1 combines the feature information of the second convolutional layer J2; Figure 3 The fourth convolution layer J4 is connected to the fourth upsampling layer S4, and the fourth upsampling layer S4 combines the feature information of the fourth convolution layer J4; Figure 3 The sixth convolutional layer J6 is connected to the second upsampling layer S2, and the second upsampling layer S2 combines the feature information of the sixth convolutional layer J6; Figure 3The tenth convolutional layer J10 is connected to the third upsampling layer S3, and the third upsampling layer S3 combines the feature information of the tenth convolutional layer J10; Figure 3 The fourteenth convolutional layer J14 is connected to the fourth pooling layer C4, and the fourth pooling layer C4 combines the feature information of the fourteenth convolutional layer J14.

[0093] The water body recognition method provided by the present invention combines multiple upsampling layers, multiple pooling layers and multiple convolutional layers, and connects multiple network layers of preset paths, so that the final water body recognition model can extract more convolution feature information and improve the recognition accuracy of the water body recognition model.

[0094] Figure 4 FIG. 3 is a flow chart of the water body identification method provided by the present invention. Figure 4 As shown, after executing step 105, the following steps are also included:

[0095] Step 106: Process the image sample data to obtain a gray level co-occurrence matrix sample of each pixel in the image sample data.

[0096] Optionally, the image sample data is preprocessed to obtain orthophoto samples; the orthophoto samples are grayscale processed to obtain grayscale image samples; and the grayscale co-occurrence matrix samples of each pixel are determined based on the grayscale image samples.

[0097] For example, the image sample data is preprocessed by distortion correction, aerial triangulation, mosaicking and color matching to obtain orthophoto samples so that the image sample data has consistent spectral reflectance and geometric shape; then the orthophoto samples are gray-scale processed, and the grayscale co-occurrence matrix samples of each pixel are determined based on the obtained grayscale image samples.

[0098] Specifically, the grayscale image samples are linearly compressed; and the grayscale co-occurrence matrix samples of each pixel are determined based on the compressed grayscale image samples.

[0099] For example, when a grayscale image sample is obtained, the grayscale image sample is linearly compressed to reduce the data amount of the grayscale co-occurrence matrix finally generated. The present invention compresses the grayscale value of the grayscale image sample within the range of 0 to 8 by a linear compression method; Figure 5 is one of the schematic diagrams of grayscale image sample processing provided by the present invention, such as Figure 5As shown, the left side is the grayscale image sample corresponding to the visible light image data to be detected, and the right side is the compressed grayscale image sample obtained by linearly compressing the grayscale image sample. The specific linear compression method can be: divide the difference between the smallest grayscale value and the largest grayscale value in the grayscale image sample into eight intervals, and number the eight intervals from 1 to 8 from small to large; then determine which interval each grayscale value in the grayscale image sample falls into, and replace the grayscale value in the grayscale image sample with the number of the corresponding interval; for example, if the grayscale value 245 and the grayscale value 242 in the grayscale image sample both fall into the largest interval, then the grayscale value 245 and the grayscale value 242 are both replaced by 8; if the grayscale value 0 in the grayscale image sample falls into the smallest interval, then the grayscale value 0 is replaced by 0. The specific linear compression results are shown as follows. Figure 5 shown.

[0100] The water body recognition method provided by the present invention performs linear grayscale compression on grayscale image samples, thereby reducing the data volume of the grayscale co-occurrence matrix finally generated.

[0101] Furthermore, determining the gray level co-occurrence matrix sample of each pixel based on the compressed gray level image sample can be achieved in the following manner:

[0102] Determine the target neighborhood range of each pixel point in the compressed grayscale image sample; for the target neighborhood range of each pixel point, determine the number of occurrences of each pixel pair on a preset vector within the target neighborhood range; and determine the grayscale co-occurrence matrix sample of the corresponding pixel point based on the number of occurrences of each pixel pair.

[0103] The preset vector is a predetermined specified vector, for example, the preset vector is a vector with a length of d and a direction of θ.

[0104] For example, taking the central pixel point (x, y) of the grayscale image sample as an example, the neighborhood range n of the central pixel point (x, y) is determined. For example, n=5, then the neighborhood range of the central pixel point (x, y) is a 5*5 pixel area including the pixel point (x, y); Figure 6 This is the second schematic diagram of grayscale image sample processing provided by the present invention. Figure 6As shown, the neighborhood range of the central pixel point (x, y) is the area within the dotted box, the central pixel point (x, y) is represented by a slash, and the preset vector is represented by a black arrow; then the number of times the pixel pair on the preset vector appears within the neighborhood range is counted, for example, the number of times the pixel pair (2,2) on the preset vector appears within the neighborhood range is 1, then fill 1 in the second row and second case position of the gray-level co-occurrence matrix sample; the number of times the pixel pair (4,5) on the preset vector appears within the neighborhood range is 2, then fill 2 in the fourth row and fifth case position of the gray-level co-occurrence matrix sample; the number of times the pixel pair (7,3) on the preset vector appears within the neighborhood range is 1, then fill 1 in the seventh row and third case position of the gray-level co-occurrence matrix sample; according to this method, the number of times each pixel pair on the preset vector within the neighborhood range is counted, and the final gray-level co-occurrence matrix sample is as follows Figure 6 As shown, in Figure 6 In the figure, the left side shows the compressed grayscale image samples with marked neighborhood ranges, and the right side shows the gray-level co-occurrence matrix samples.

[0105] Step 107: training the initial network model based on the entropy of the gray-level co-occurrence matrix sample of each pixel and the image sample data to obtain the water body recognition model.

[0106] For example, after the gray level co-occurrence matrix sample is determined, the entropy of the gray level co-occurrence matrix sample is calculated based on the following formula (1), and the entropy of the gray level co-occurrence matrix sample is used as the texture feature value of the central pixel point (x, y).

[0107] Ent=-∑ i ∑ j p(i, j)logp(i, j) (1)

[0108] Among them, Ent represents the entropy of the gray-level co-occurrence matrix sample, and p(i, j) represents the pixel point with abscissa i and ordinate j.

[0109] After obtaining the entropy of the gray-level co-occurrence matrix samples of each pixel in the image sample data, the entropy of the gray-level co-occurrence matrix samples of each pixel in the image sample data and the image sample data are fused and input into the initial network model to obtain the predicted confidence that each pixel output by the initial network model is water; then determine the difference between the predicted confidence that each pixel is water and the true value of whether each pixel is water, and optimize the initial network model based on the difference until the model parameters of the initial network model reach the convergence condition to obtain the water body recognition model.

[0110] In addition, after the water body recognition model is trained, the water body recognition model can also be evaluated qualitatively, quantitatively and efficiently to ensure the water body recognition quality of the water body recognition model; wherein, the qualitative evaluation is to determine whether the water body recognition model can recognize water body areas that the previous model could not recognize; the quantitative evaluation is to determine how much percentage the water body recognition model has improved its accuracy compared to the previous model; and the efficiency evaluation is to determine the length of time the water body recognition model takes to identify water bodies.

[0111] It should be noted that, for each pixel in the compressed grayscale image sample, it is necessary to calculate the entropy of the grayscale co-occurrence matrix sample of each pixel according to the calculation method of the entropy of the grayscale co-occurrence matrix sample of the central pixel. In this way, all texture feature quantities corresponding to the compressed grayscale image sample can be obtained. The texture feature quantity is what the user can perceive visually, so a visual feature layer of the corresponding area is obtained.

[0112] The water body recognition method provided by the present invention uses the entropy of the gray level co-occurrence matrix sample of each pixel as the texture feature of the pixel, and fuses it with the image sample data and inputs it into the initial network model, so that the recognition accuracy of the water body recognition model finally obtained by training is improved.

[0113] Optionally, Figure 1 Step 102 in the embodiment can be specifically implemented in the following manner:

[0114] The visible light image data to be detected is processed to obtain a gray level co-occurrence matrix of each pixel in the visible light image data to be detected; the entropy of each gray level co-occurrence matrix and the visible light image data to be detected are input into the water body recognition model to obtain the confidence that each pixel output by the water body recognition model is a water body.

[0115] For example, when the visible light image data to be detected is obtained, the visible light image data to be detected can also be processed. The specific processing steps can refer to the processing steps of the above-mentioned image data sample, and finally the gray level co-occurrence matrix of each pixel in the visible light image data to be detected is obtained, and then the entropy of the gray level co-occurrence matrix of each pixel is calculated based on the above-mentioned formula (1). Finally, the entropy of the gray level co-occurrence matrix of each pixel is used as a texture feature and fused with the visible light image data to be detected. The fused data is input into the water body recognition model, and then the confidence that each pixel output by the water body recognition model is a water body is obtained.

[0116] The water body recognition method provided by the present invention uses the entropy of the grayscale co-occurrence matrix of each pixel as the texture feature of the pixel, uses the visible light image data to be detected as the spectral feature, and inputs the texture feature and the spectral feature together into the water body recognition model, so that the water body recognition model can effectively distinguish between low-reflectivity objects and mixed water bodies in complex landscapes, thereby improving the recognition accuracy of the water body recognition model.

[0117] The water body identification device provided by the present invention is described below. The water body identification device described below and the water body identification method described above can be referenced to each other.

[0118] Figure 7 Schematic diagram of the structure of the water body identification device provided by the present invention. Figure 7 As shown, the water body identification device includes an acquisition unit 701, an identification unit 702 and a determination unit 703; wherein,

[0119] An acquisition unit 701 is used to acquire visible light image data to be detected;

[0120] The recognition unit 702 is used to input the visible light image data to be detected into a water body recognition model to obtain the confidence that each pixel point output by the water body recognition model is a water body;

[0121] A determination unit 703 is used to determine a target water body area in the visible light image data to be detected based on the confidence that each pixel point is a water body;

[0122] Wherein, the water body recognition model is obtained by training an initial network model based on image sample data; the initial network model includes network layer connections of multiple preset paths.

[0123] The present invention provides a water body recognition device, which inputs the acquired visible light image data to be detected into a water body recognition model to obtain the confidence that each pixel point output by the water body recognition model is a water body. Since the water body recognition model in the present invention is obtained by training based on an initial network model including a network layer connection of multiple preset paths, and the network layer connection of multiple preset paths can increase the number of connection layers in the network, the increase in the number of connection layers means that more convolution features are retained, thereby improving the accuracy of water body recognition.

[0124] Based on any of the above embodiments, the device further includes a first building unit and a second building unit;

[0125] The first construction unit is used to construct a recurrent neural network based on a cross network path of multiple upsampling layers and multiple pooling layers;

[0126] The second construction unit is used to construct the initial network model based on the recurrent neural network and multiple convolutional layers.

[0127] Based on any of the foregoing embodiments, the cross network path of the multiple upsampling layers and the multiple pooling layers sequentially includes a first upsampling layer, a first pooling layer, a second pooling layer, a second upsampling layer, a third pooling layer, a third upsampling layer, a fourth upsampling layer and a fourth pooling layer;

[0128] The first building block is specifically used for:

[0129] A combination of the first upsampling layer, the first pooling layer, the second pooling layer, the second upsampling layer, the third pooling layer, the third upsampling layer, the fourth upsampling layer and the fourth pooling layer is determined as the recurrent neural network.

[0130] Based on any of the above embodiments, the second building unit is specifically used for:

[0131] A first convolutional layer and a second convolutional layer are sequentially arranged between the first upsampling layer and the network input layer;

[0132] A third convolutional layer and a fourth convolutional layer are sequentially arranged between the first upsampling layer and the first pooling layer;

[0133] A fifth convolutional layer and a sixth convolutional layer are sequentially arranged between the first pooling layer and the second pooling layer;

[0134] A seventh convolutional layer and an eighth convolutional layer are sequentially arranged between the second pooling layer and the second upsampling layer;

[0135] A ninth convolutional layer and a tenth convolutional layer are sequentially arranged between the second upsampling layer and the third pooling layer;

[0136] An eleventh convolutional layer and a twelfth convolutional layer are sequentially arranged between the third pooling layer and the third upsampling layer;

[0137] A thirteenth convolutional layer and a fourteenth convolutional layer are sequentially arranged between the third upsampling layer and the fourth upsampling layer;

[0138] A fifteenth convolutional layer and a sixteenth convolutional layer are sequentially arranged between the fourth upsampling layer and the fourth pooling layer;

[0139] A seventeenth convolutional layer, an eighteenth convolutional layer and a nineteenth convolutional layer are sequentially arranged between the fourth pooling layer and the network output layer.

[0140] Based on any of the above embodiments, the network layer connections of the multiple preset paths include:

[0141] Connecting the second convolutional layer to the first pooling layer;

[0142] Connecting the sixth convolutional layer to the second upsampling layer;

[0143] Connecting the tenth convolutional layer to the third upsampling layer;

[0144] Connecting the fourteenth convolutional layer to the fourth pooling layer;

[0145] The fourth convolutional layer is connected to the fourth upsampling layer.

[0146] Based on any of the above embodiments, the device further includes a processing unit and a training unit;

[0147] The processing unit is used to process the image sample data to obtain a gray level co-occurrence matrix sample of each pixel in the image sample data;

[0148] The training unit is used to train the initial network model based on the entropy of the gray-level co-occurrence matrix sample of each pixel and the image sample data to obtain the water body recognition model.

[0149] Based on any of the above embodiments, the processing unit is specifically used for:

[0150] Preprocessing the image sample data to obtain an orthophoto sample;

[0151] Performing grayscale processing on the orthophoto sample to obtain a grayscale image sample;

[0152] The gray level co-occurrence matrix sample of each pixel is determined based on the gray level image sample.

[0153] Based on any of the above embodiments, the processing unit is further specifically configured to:

[0154] Performing linear grayscale compression on the grayscale image samples;

[0155] The gray-level co-occurrence matrix sample of each pixel is determined based on the compressed gray-level image sample.

[0156] Based on any of the above embodiments, the processing unit is further specifically configured to:

[0157] Determine the target neighborhood range of each pixel in the compressed grayscale image sample;

[0158] For each of the target neighborhoods of the pixel points, determining the number of occurrences of each pixel pair on a preset vector within the target neighborhood;

[0159] The gray level co-occurrence matrix samples of the corresponding pixel points are determined based on the number of occurrences of each pixel pair.

[0160] Based on any of the above embodiments, the identification unit 702 is specifically used for:

[0161] Processing the visible light image data to be detected to obtain a gray level co-occurrence matrix of each pixel in the visible light image data to be detected;

[0162] The entropy of each gray-level co-occurrence matrix and the visible light image data to be detected are input into the water body recognition model to obtain the confidence that each pixel point output by the water body recognition model is a water body.

[0163] The present invention provides a water body recognition device, which inputs the acquired visible light image data to be detected into a water body recognition model to obtain the confidence that each pixel point output by the water body recognition model is a water body. Since the water body recognition model in the present invention is obtained by training an initial network model based on a cross network path including multiple upsampling layers and multiple pooling layers, and the cross network path of multiple upsampling layers and multiple pooling layers can increase the number of connection layers in the network, the increase in the number of connection layers means that more convolution features are retained, thereby improving the accuracy of water body recognition.

[0164] Figure 8 is a schematic diagram of the physical structure of the electronic device provided by the present invention, such as Figure 8 As shown, the electronic device may include: a processor 810, a communications interface 820, a memory 830 and a communication bus 840, wherein the processor 810, the communications interface 820 and the memory 830 communicate with each other via the communication bus 840. The processor 810 may call the logic instructions in the memory 830 to execute the water body recognition method, which includes: obtaining visible light image data to be detected;

[0165] Inputting the visible light image data to be detected into a water body recognition model to obtain the confidence that each pixel point output by the water body recognition model is a water body;

[0166] Determine the target water body area in the visible light image data to be detected based on the confidence that each pixel point is a water body;

[0167] Wherein, the water body recognition model is obtained by training an initial network model based on image sample data; the initial network model includes network layer connections of multiple preset paths.

[0168] In addition, the logic instructions in the above-mentioned memory 830 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0169] On the other hand, the present invention further provides a computer program product, the computer program product includes a computer program, the computer program can be stored in a non-transitory computer-readable storage medium, when the computer program is executed by a processor, the computer can execute the water body identification method provided by the above methods, the method includes: obtaining visible light image data to be detected;

[0170] Inputting the visible light image data to be detected into a water body recognition model to obtain the confidence that each pixel point output by the water body recognition model is a water body;

[0171] Determine the target water body area in the visible light image data to be detected based on the confidence that each pixel point is a water body;

[0172] Wherein, the water body recognition model is obtained by training an initial network model based on image sample data; the initial network model includes network layer connections of multiple preset paths.

[0173] In another aspect, the present invention further provides a non-transitory computer-readable storage medium having a computer program stored thereon, which is implemented when the computer program is executed by a processor to perform the water body identification method provided by the above methods, the method comprising: obtaining visible light image data to be detected;

[0174] Inputting the visible light image data to be detected into a water body recognition model to obtain the confidence that each pixel point output by the water body recognition model is a water body;

[0175] Determine the target water body area in the visible light image data to be detected based on the confidence that each pixel point is a water body;

[0176] Wherein, the water body recognition model is obtained by training an initial network model based on image sample data; the initial network model includes network layer connections of multiple preset paths.

[0177] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Those of ordinary skill in the art may understand and implement it without creative work.

[0178] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0179] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A water body identification method, characterized in that: include: Acquire visible light image data to be detected; Inputting the visible light image data to be detected into a water body recognition model to obtain the confidence that each pixel point output by the water body recognition model is a water body; Determine the target water body area in the visible light image data to be detected based on the confidence that each pixel point is a water body; The water body recognition model is obtained by training an initial network model based on image sample data; the initial network model includes a plurality of network layer connections of preset paths; the method further includes: Construct a recurrent neural network based on a cross-network path of multiple upsampling layers and multiple pooling layers; Constructing the initial network model based on the recurrent neural network and multiple convolutional layers; The cross network path of the multiple upsampling layers and the multiple pooling layers includes, in sequence, a first upsampling layer, a first pooling layer, a second pooling layer, a second upsampling layer, a third pooling layer, a third upsampling layer, a fourth upsampling layer, and a fourth pooling layer; The method of constructing a recurrent neural network based on a cross network path of multiple upsampling layers and multiple pooling layers includes: Determine a combination of the first upsampling layer, the first pooling layer, the second pooling layer, the second upsampling layer, the third pooling layer, the third upsampling layer, the fourth upsampling layer and the fourth pooling layer as the recurrent neural network; The constructing the initial network model based on the recurrent neural network and multiple convolutional layers includes: A first convolutional layer and a second convolutional layer are sequentially arranged between the first upsampling layer and the network input layer; A third convolutional layer and a fourth convolutional layer are sequentially arranged between the first upsampling layer and the first pooling layer; A fifth convolutional layer and a sixth convolutional layer are sequentially arranged between the first pooling layer and the second pooling layer; A seventh convolutional layer and an eighth convolutional layer are sequentially arranged between the second pooling layer and the second upsampling layer; A ninth convolutional layer and a tenth convolutional layer are sequentially arranged between the second upsampling layer and the third pooling layer; An eleventh convolutional layer and a twelfth convolutional layer are sequentially arranged between the third pooling layer and the third upsampling layer; A thirteenth convolutional layer and a fourteenth convolutional layer are sequentially arranged between the third upsampling layer and the fourth upsampling layer; A fifteenth convolutional layer and a sixteenth convolutional layer are sequentially arranged between the fourth upsampling layer and the fourth pooling layer; A seventeenth convolutional layer, an eighteenth convolutional layer and a nineteenth convolutional layer are sequentially arranged between the fourth pooling layer and the network output layer; The network layer connections of the plurality of preset paths include: Connecting the second convolutional layer to the first pooling layer; Connecting the sixth convolutional layer to the second upsampling layer; Connecting the tenth convolutional layer to the third upsampling layer; Connecting the fourteenth convolutional layer to the fourth pooling layer; The fourth convolutional layer is connected to the fourth upsampling layer.

2. The water body identification method according to claim 1, characterized in that: After constructing the initial network model based on the recurrent neural network and multiple convolutional layers, the method further includes: Processing the image sample data to obtain a gray level co-occurrence matrix sample of each pixel in the image sample data; The initial network model is trained based on the entropy of the gray-level co-occurrence matrix sample of each pixel and the image sample data to obtain the water body recognition model.

3. The water body identification method according to claim 2, characterized in that: The processing of the image sample data to obtain a gray level co-occurrence matrix sample of each pixel in the image sample data includes: Preprocessing the image sample data to obtain an orthophoto sample; Performing grayscale processing on the orthophoto sample to obtain a grayscale image sample; The gray level co-occurrence matrix sample of each pixel is determined based on the gray level image sample.

4. The water body identification method according to claim 3, characterized in that: The step of determining the gray level co-occurrence matrix sample of each pixel based on the gray level image sample comprises: Performing linear grayscale compression on the grayscale image samples; The gray-level co-occurrence matrix sample of each pixel is determined based on the compressed gray-level image sample.

5. The water body identification method according to claim 4, characterized in that: The step of determining the gray level co-occurrence matrix sample of each pixel based on the compressed gray level image sample comprises: Determine the target neighborhood range of each pixel in the compressed grayscale image sample; For each of the target neighborhoods of the pixel points, determining the number of occurrences of each pixel pair on a preset vector within the target neighborhood; The gray level co-occurrence matrix samples of the corresponding pixel points are determined based on the number of occurrences of each pixel pair.

6. The water body identification method according to any one of claims 1 to 5, characterized in that: The step of inputting the visible light image data to be detected into a water body recognition model to obtain the confidence level of each pixel point output by the water body recognition model as a water body includes: Processing the visible light image data to be detected to obtain a gray level co-occurrence matrix of each pixel in the visible light image data to be detected; The entropy of each gray-level co-occurrence matrix and the visible light image data to be detected are input into the water body recognition model to obtain the confidence that each pixel point output by the water body recognition model is a water body.

7. A water body identification device, characterized in that: include: An acquisition unit, used for acquiring visible light image data to be detected; An identification unit, used for inputting the visible light image data to be detected into a water body identification model to obtain the confidence that each pixel point output by the water body identification model is a water body; A determination unit, configured to determine a target water body region in the visible light image data to be detected based on the confidence level that each pixel point is a water body; Wherein, the water body recognition model is obtained by training the initial network model based on the image sample data; the initial network model includes a plurality of network layer connections of preset paths; The device also includes a first building unit and a second building unit; The first construction unit is used to construct a recurrent neural network based on a cross network path of multiple upsampling layers and multiple pooling layers; The second construction unit is used to construct the initial network model based on the recurrent neural network and multiple convolutional layers; The cross network path of the multiple upsampling layers and the multiple pooling layers includes, in sequence, a first upsampling layer, a first pooling layer, a second pooling layer, a second upsampling layer, a third pooling layer, a third upsampling layer, a fourth upsampling layer, and a fourth pooling layer; The first building block is specifically used for: Determine a combination of the first upsampling layer, the first pooling layer, the second pooling layer, the second upsampling layer, the third pooling layer, the third upsampling layer, the fourth upsampling layer and the fourth pooling layer as the recurrent neural network; The second building block is specifically used for: A first convolutional layer and a second convolutional layer are sequentially arranged between the first upsampling layer and the network input layer; A third convolutional layer and a fourth convolutional layer are sequentially arranged between the first upsampling layer and the first pooling layer; A fifth convolutional layer and a sixth convolutional layer are sequentially arranged between the first pooling layer and the second pooling layer; A seventh convolutional layer and an eighth convolutional layer are sequentially arranged between the second pooling layer and the second upsampling layer; A ninth convolutional layer and a tenth convolutional layer are sequentially arranged between the second upsampling layer and the third pooling layer; An eleventh convolutional layer and a twelfth convolutional layer are sequentially arranged between the third pooling layer and the third upsampling layer; A thirteenth convolutional layer and a fourteenth convolutional layer are sequentially arranged between the third upsampling layer and the fourth upsampling layer; A fifteenth convolutional layer and a sixteenth convolutional layer are sequentially arranged between the fourth upsampling layer and the fourth pooling layer; A seventeenth convolutional layer, an eighteenth convolutional layer and a nineteenth convolutional layer are sequentially arranged between the fourth pooling layer and the network output layer; The network layer connections of the plurality of preset paths include: Connecting the second convolutional layer to the first pooling layer; Connecting the sixth convolutional layer to the second upsampling layer; Connecting the tenth convolutional layer to the third upsampling layer; Connecting the fourteenth convolutional layer to the fourth pooling layer; The fourth convolutional layer is connected to the fourth upsampling layer.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the steps of the water body identification method according to any one of claims 1 to 6 are implemented.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the water body identification method according to any one of claims 1 to 6 are implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the water body identification method according to any one of claims 1 to 6 are implemented.

Images