Method and system for identifying waterlogging ponding range based on binocular vision
Through binocular vision technology and deep learning methods, real-time and accurate monitoring and trend prediction of water accumulation are achieved, and the problems of insufficient accuracy and applicability in traditional methods are solved, which is of great significance to urban flood control and emergency management.
Patent Information
- Application Number
- CN202411981889.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-12-31
- Publication Date
- 2025-05-13
AI Technical Summary
Traditional water accumulation monitoring methods have insufficient accuracy, untimely response, and are difficult to apply to a variety of environments and scenarios. The real-time and accuracy of existing computer vision methods in complex environments are limited.
The waterlogging range recognition method based on binocular vision is used to collect video data through binocular cameras, and the dry and wet period discrimination and waterlogging range recognition is performed using convolutional neural networks and U-Net networks. The waterlogging area is calculated based on binocular camera parameters, and the development trend of waterlogging area is predicted through long and short-term memory networks.
Real-time and accurate monitoring of water accumulation conditions has been achieved, and the problems of insufficient three-dimensional spatial information acquisition and weak dynamic environmental adaptability in traditional methods have been overcome, and the accuracy and robustness of water accumulation scope assessment has been improved, and it has the important significance of urban flood control and emergency management.
Smart Images
Figure CN119992409A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the field of computer vision and intelligent monitoring technology, and in particular relates to a method and system for identifying the scope of waterlogging based on binocular vision. Background Art
[0002] With the acceleration of urbanization, urban waterlogging is becoming more and more serious. Traditional waterlogging monitoring methods mainly rely on artificial or static sensors. The waterlogging monitoring accuracy is insufficient and the response is not timely, making it difficult to accurately monitor the waterlogging situation in a short time. Secondly, traditional cameras have problems such as insufficient three-dimensional spatial information acquisition, plane assumption limitations, and weak dynamic environment adaptability when obtaining the scope of waterlogging area, making it difficult to apply to a variety of different environments and scenarios. In addition, existing computer vision methods have certain limitations in real-time and accuracy in dealing with complex environments. Therefore, a waterlogging range recognition and trend testing system based on video data is proposed. Combined with binocular camera technology, it can effectively monitor the waterlogging situation in real time, which is of great significance to urban flood control and emergency management. Summary of the invention
[0003] The purpose of the present invention is to provide a method for identifying the range of urban flooding based on binocular camera technology and a system for identifying the range of urban flooding using the method, so as to solve the problems of insufficient waterlogging monitoring accuracy, untimely response and limited application scenarios in the prior art.
[0004] To achieve the above purpose, the present invention proposes a method for identifying the range of waterlogging based on binocular vision, and adopts the following technical solutions:
[0005] A method for identifying waterlogging range based on binocular vision comprises the following steps:
[0006] Step 1: collect real-time video data of the target area through a binocular camera installed in the target area, extract video frames, and pre-process each video frame;
[0007] Step 2: Taking the preprocessed video frames as input, based on the pre-trained dry and wet period discrimination model, it is judged whether there is a waterlogging area in each video frame. The video frames without waterlogging areas are judged as dry periods, and the video frames with waterlogging areas are judged as wet periods.
[0008] Step 3: Taking each video frame identified as a wet period as input, based on the pre-trained water accumulation range recognition model, identify and segment the water accumulation area image;
[0009] Step 4: Count the number of water-logged pixels in the water-logged area image, and calculate the actual area of the water-logged area based on the parameters of the binocular camera system.
[0010] Furthermore, the preprocessing of each video frame in step 1 includes using a background difference method to remove interference information in non-waterlogged areas of each video frame, and using a denoising algorithm to remove noise.
[0011] Furthermore, the dry and wet period discrimination model is obtained by training based on the convolutional neural network model structure. The convolutional neural network model structure includes a convolutional layer, a pooling layer, a fully connected layer, and an output layer. The output layer uses a sigmoid activation function to output a binary classification result of the dry period or the wet period. The preprocessed video frames are used as the input of the convolutional neural network model structure, and the corresponding dry period or wet period labels are used as the output to train the convolutional neural network model structure to obtain the trained convolutional neural network model, namely the dry and wet period discrimination model.
[0012] Furthermore, a waterlogging range recognition model is obtained based on U-Net network model structure training. The U-Net network model structure includes an encoder, a decoder and a jump connection. The output layer uses a softmax function to output the probability that each pixel belongs to the waterlogging category. Each wet period video frame is used as the input of the U-Net network model structure, and the identified segmented waterlogging area image is used as the output. The U-Net network model structure is trained to obtain a trained U-Net network model, namely, the waterlogging range recognition model.
[0013] Furthermore, the number of water-logged pixels in the output water-logged area image is counted, and the actual area of the water-logged area is calculated by combining the internal and external parameters of the binocular camera; the internal parameter is the focal length f of the binocular camera in the x and y directions. x and f y , the coordinates of the center point of the image c x and c y ; The external parameter is the pixel depth Z of the binocular camera, which is the distance from the actual object corresponding to the pixel to the camera; combined with the coordinates p of the pixel in the image x and p y , calculate the actual area of the waterlogged area. The specific calculation process is as follows:
[0014] The actual size S of the pixel in the x and y directions x and S y They are:
[0015]
[0016] The actual area of each pixel is A:
[0017] A=S x *S y
[0018] When calculating the actual area of the waterlogged area, it is assumed that the actual size of the pixel in the x and y directions is the same, that is, S x=S y =S, the expression of S is:
[0019]
[0020] Where f is the average focal length and f = (f x +f y ) / 2, the actual area of each pixel is:
[0021]
[0022] The actual area of the entire waterlogged area is obtained as follows:
[0023]
[0024] Among them, A i is the actual area of the ith pixel, Z i is the depth of the i-th pixel, p xi is the x-coordinate of the ith pixel in the image.
[0025] Furthermore, the present invention also protects a system for identifying the range of waterlogging based on the above method, which mainly includes a video data acquisition module, a data preprocessing module located in a data processing center, a dry and wet period discrimination module, and a waterlogging range recognition module; the video data acquisition module collects real-time video data of the target area through a binocular camera, and transmits the collected real-time video data to the data processing center to extract video frames; each video frame preprocessed by the data preprocessing module is input into the dry and wet period discrimination module to discriminate whether the video frame is a dry period or a wet period; each video frame judged as a wet period is input into the waterlogging range recognition module to realize the recognition and segmentation of the waterlogging area image.
[0026] Furthermore, the waterlogging range identification system further includes a waterlogging trend prediction module, which predicts the trend of waterlogging area in the target area over time based on a pre-trained waterlogging trend prediction model. The prediction formula for the waterlogging area trend is:
[0027] A(t+1)=f(A(t),ΔA)
[0028] Among them, A(t+1) is the predicted waterlogging area of the target area at time t+1, A(t) is the waterlogging area of the target area at time t, ΔA is the area change, and f is the waterlogging trend prediction model.
[0029] Furthermore, the waterlogging trend prediction model is obtained based on the long short-term memory network (LSTM) model structure training, and the specific steps of obtaining the waterlogging trend prediction model and the prediction result include:
[0030] Step 1: Collect the time series data of waterlogged area in the target area within the historical period;
[0031] Step 2: Preprocess the collected data, including cleaning data, supplementing missing data, and standardizing data;
[0032] Step 3: construct a long short-term memory (LSTM) model structure, wherein the LSTM model structure includes two LSTM layers, and the number of cell units is 128 and 64 respectively; using the preprocessed waterlogging area time series data as input and the corresponding predicted waterlogging area time series data as output, the constructed LSTM model structure is trained to obtain a waterlogging trend prediction model;
[0033] Step 4: Taking the time series data of the waterlogged area in the target area to be predicted during the target time period as input, the waterlogging trend prediction model is used to obtain the predicted result of the waterlogged area of the target area changing over time.
[0034] Compared with the prior art, the present invention has the following beneficial effects:
[0035] The present invention collects video data through a binocular camera in real time, uses neural network deep learning and image segmentation technology to identify the waterlogged area, and combines the binocular camera parameters to evaluate the scope of the waterlogging. It further predicts the development trend of the waterlogged area through a long short-term memory network, and overcomes the problems of insufficient three-dimensional spatial information acquisition, reliance on plane assumptions, and weak adaptability to dynamic environments that exist in traditional cameras when acquiring waterlogged areas. The accuracy and robustness of waterlogging scope assessment are improved through depth information, and future trends can be predicted while real-time monitoring of the waterlogging situation is achieved. This is of great significance to urban flood control and emergency management. BRIEF DESCRIPTION OF THE DRAWINGS
[0036] Figure 1 A flowchart of a method for identifying a waterlogging range provided by a specific embodiment of the present invention;
[0037] Figure 2 A video data processing flow chart provided for a specific embodiment of the present invention;
[0038] Figure 3 A schematic diagram of the convolutional neural network model structure provided for a specific embodiment of the present invention;
[0039] Figure 4 A schematic diagram of the U-Net network model structure provided for a specific embodiment of the present invention;
[0040] Figure 5 A flow chart for predicting the trend of waterlogged area provided for a specific implementation manner of the present invention. DETAILED DESCRIPTION
[0041] The technical solution of the present invention is further described in detail below in conjunction with the accompanying drawings and specific implementation methods.
[0042] like Figure 1 , Figure 2 As shown, the present invention provides a method for identifying the scope of waterlogging based on binocular vision, comprising the following steps:
[0043] Step 1: collect real-time video data of the target area through a binocular camera installed in the target area, extract video frames, and pre-process each video frame, including using a background difference method to remove interference information in non-waterlogged areas in each video frame, and using a denoising algorithm to remove noise;
[0044] Step 2: Taking the preprocessed video frames as input, based on the pre-trained dry and wet period discrimination model, it is judged whether there is a waterlogging area in each video frame. The video frames without waterlogging areas are judged as dry periods, and the video frames with waterlogging areas are judged as wet periods.
[0045] Step 3: Taking each video frame identified as a wet period as input, based on the pre-trained water accumulation range recognition model, identify and segment the water accumulation area image;
[0046] Step 4: Count the number of water-logged pixels in the water-logged area image, and calculate the actual area of the water-logged area based on the parameters of the binocular camera system.
[0047] like Figure 3 As shown, the dry and wet period discrimination model in step 2 is obtained by training based on the convolutional neural network model structure, and the convolutional neural network model structure is connected to a fully connected layer output after three convolutional pooling layers; each convolutional layer uses a 3×3 convolution kernel, and the number of convolution kernels is 32, 64, and 128 respectively, and each convolutional layer is followed by a maximum pooling layer, using a 2×2 pooling kernel with a step size of 2, and the output layer uses a sigmoid activation function to output a binary classification result of the dry period or the wet period; the preprocessed video frames are used as the input of the convolutional neural network model structure, and the convolutional neural network model structure is trained with the corresponding dry period or wet period label as the output to obtain the trained convolutional neural network model, that is, the dry and wet period discrimination model.
[0048] like Figure 4As shown, the waterlogging range recognition model in step 3 is obtained based on the U-Net network model structure training, and the U-Net network model structure includes an encoder, a decoder and a jump connection; the encoder consists of four downsampling operations, each of which includes a convolution layer, a ReLU activation layer, and a maximum pooling layer. Each convolution layer uses a 3×3 convolution kernel with a step size of 1, and the number of convolution kernels is 16, 32, 64, and 128 respectively. The maximum pooling layer uses a 2×2 pooling kernel with a step size of 2; the decoder consists of four upsampling operations, each of which consists of a transposed convolution layer, a convolution layer, and a ReLU activation function layer. The jump connection connects the feature maps of the corresponding layers of the encoder and the decoder, and the output layer uses a softmax function to output the probability that each pixel belongs to the waterlogging category; each video frame identified as a wet period is used as the input of the U-Net network model structure, and the image of the waterlogging area identified and segmented is used as the output, and the U-Net network model structure is trained to obtain a trained U-Net network model, that is, a waterlogging range recognition model.
[0049] Step 4 counts the number of water-logged pixels in the water-logged area image, and calculates the actual area of the water-logged area by combining the internal and external parameters of the binocular camera system:
[0050] The internal parameter is the focal length f of the binocular camera in the x and y directions x and f y , the coordinates of the center point of the image c x and c y ; The external parameter is the pixel depth Z, which is the distance from the actual object corresponding to the pixel to the camera; combined with the coordinates p of the pixel in the image x and p y , calculate the actual area of the waterlogged area. The specific calculation process is as follows:
[0051] The actual size S of the pixel in the x and y directions x and S y They are:
[0052]
[0053] The actual area of each pixel is A:
[0054] A=S x *S y
[0055] When calculating the actual area of the waterlogged area, it is assumed that the actual size of the pixel in the x and y directions is the same, that is, S x =S y =S, the expression of S is:
[0056]
[0057] Where f is the average focal length and f = f x +f y / 2, the actual area of each pixel is:
[0058]
[0059] The actual area of the entire waterlogged area is:
[0060]
[0061] Among them, A i is the actual area of the ith pixel, Z i is the depth of the i-th pixel, p xi is the x-coordinate of the ith pixel in the image.
[0062] The present invention also protects a system for identifying the range of waterlogging based on the above method, which mainly includes a video data acquisition module, a data preprocessing module located in a data processing center, a dry and wet period discrimination module, and a waterlogging range recognition module; the video data acquisition module collects real-time video data of the target area through a binocular camera, and transmits the collected real-time video data to the data processing center to extract video frames; each video frame preprocessed by the data preprocessing module is input into the dry and wet period discrimination module to discriminate whether the video frame is in a dry period or a wet period; each video frame judged as a wet period is input into the waterlogging range recognition module to realize the recognition and segmentation of the waterlogging area image.
[0063] Furthermore, the waterlogging range identification system also includes a waterlogging trend prediction module, which uses a pre-trained waterlogging trend prediction model to predict the trend of waterlogging area in the target area over time. The prediction formula for the waterlogging area trend is:
[0064] A(t+1)=f(A(t),ΔA)
[0065] Among them, A(t+1) is the predicted waterlogging area of the target area at time t+1, A(t) is the waterlogging area of the target area at time t, ΔA is the area change, and f is the waterlogging trend prediction model.
[0066] like Figure 5 As shown, the water accumulation trend prediction model is obtained based on the long short-term memory network (LSTM) model structure training, and the specific steps of obtaining the water accumulation trend prediction model and the prediction result include:
[0067] Step 1: Collect the time series data of waterlogged area in the target area within the historical period;
[0068] Step 2: Preprocess the collected data, including cleaning data, processing missing data, and standardizing data;
[0069] Step 3: construct a long short-term memory (LSTM) model structure, wherein the LSTM model structure includes two LSTM layers, and the number of cell units is 128 and 64 respectively; using the preprocessed waterlogging area time series data as input and the corresponding predicted waterlogging area time series data as output, the constructed LSTM model structure is trained to obtain a waterlogging trend prediction model;
[0070] Step 4: Taking the time series data of the waterlogged area in the target area to be predicted during the target time period as input, the waterlogging trend prediction model is used to obtain the predicted result of the waterlogged area of the target area changing over time.
[0071] The above description is only a preferred specific embodiment of the present invention and is not intended to limit the present invention. Any modification, equivalent replacement, improvement, etc. made by any person skilled in the art within the technical scope disclosed by the present invention and based on the technical solution and inventive concept of the present invention shall be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.
Claims
1. A method for identifying the scope of waterlogging based on binocular vision, characterized in that: The following steps are involved: Step 1: collect real-time video data of the target area through a binocular camera installed in the target area, extract video frames, and pre-process each video frame; Step 2: Taking the preprocessed video frames as input, based on the pre-trained dry and wet period discrimination model, it is judged whether there is a waterlogging area in each video frame. The video frames without waterlogging areas are judged as dry periods, and the video frames with waterlogging areas are judged as wet periods. Step 3: Taking each video frame identified as a wet period as input, based on the pre-trained water accumulation range recognition model, identify and segment the water accumulation area image; Step 4: Count the number of water-logged pixels in the water-logged area image, and calculate the actual area of the water-logged area based on the parameters of the binocular camera system.
2. According to the binocular vision-based method for identifying the scope of waterlogging in urban areas, the method is characterized in that: The preprocessing of each video frame includes using the background difference method to remove the interference information of the non-water accumulation area in each video frame, and using the denoising algorithm to remove the noise.
3. The method for identifying waterlogging range based on binocular vision according to claim 1 is characterized in that: The dry and wet period discrimination model is obtained by training based on the convolutional neural network model structure. The convolutional neural network model structure includes a convolutional layer, a pooling layer, a fully connected layer, and an output layer. The output layer uses a sigmoid activation function to output a binary classification result of the dry period or the wet period. The preprocessed video frames are used as the input of the convolutional neural network model structure, and the corresponding dry period or wet period labels are used as the output to train the convolutional neural network model structure to obtain the trained convolutional neural network model, namely the dry and wet period discrimination model.
4. The method for identifying waterlogging range based on binocular vision according to claim 1 is characterized in that: The waterlogging range recognition model is obtained based on the training of the U-Net network model structure. The U-Net network model structure includes an encoder, a decoder and a skip connection. The output layer uses a softmax function to output the probability that each pixel belongs to the waterlogging category. The U-Net network model structure is trained with each wet period video frame as the input and the corresponding waterlogged area image as the output to obtain a trained U-Net network model, namely, the waterlogging range assessment model.
5. The method for identifying waterlogging range based on binocular vision according to claim 1 is characterized in that: The parameters of the binocular camera system include internal and external parameters. The internal parameter is the focal length f of the binocular camera in the x and y directions. x and f y , the coordinates of the center point of the image c x and c y ; The external parameter is the pixel depth Z of the binocular camera, which is the distance from the actual object corresponding to the pixel to the camera; combined with the coordinates p of the pixel in the image x and p y , calculate the actual area of the waterlogged area. The specific calculation process is as follows: The actual size S of the pixel in the x and y directions x and S y They are: S x =Z*f x / p x S y =Z*f x / p y The actual area of each pixel is A: A=S x *S y When calculating the actual area of the waterlogged area, it is assumed that the actual size of the pixel in the x and y directions is the same, that is, S x =S y =S, the expression of S is: S=Z*f / p x =Z*f / p y Where f is the average focal length and f×f x +f y / 2, the actual area of each pixel is: A=S 2 =(z*f / p x ) 2 =(z*f / p y ) 2 The actual area of the entire waterlogged area is obtained as follows: Among them, A i is the actual area of the ith pixel, Z i is the depth of the i-th pixel, p xi is the x-coordinate of the ith pixel in the image.
6. The waterlogging range recognition system of the waterlogging range recognition method based on binocular vision according to claim 1 is characterized in that: It mainly includes a video data acquisition module, a data preprocessing module located in a data processing center, a dry and wet period discrimination module, and a water accumulation range assessment module; the video data acquisition module collects real-time video data of the target area through a binocular camera, and transmits the collected real-time video data to the data processing center to extract video frames; each video frame preprocessed by the data preprocessing module is input into the dry and wet period discrimination module to discriminate whether each video frame is a dry period or a wet period; each video frame judged as a wet period is input into the water accumulation range recognition module to realize the recognition and segmentation of the water accumulation area image.
7. The waterlogging range identification system according to claim 6 is characterized in that: It also includes a water accumulation trend prediction module, which predicts the trend of water accumulation area in the target area over time based on a pre-trained water accumulation trend prediction model. The prediction formula for the water accumulation area trend is: A(t+1)=f(A(t),ΔA) Among them, A(t+1) is the predicted waterlogging area of the target area at time t+1, A(t) is the waterlogging area of the target area at time t, ΔA is the area change, and f is the waterlogging trend prediction model.
8. The waterlogging range identification system according to claim 7 is characterized in that: The waterlogging trend prediction model is obtained based on the long short-term memory network (LSTM) model structure training. The specific steps to obtain the waterlogging trend prediction model and prediction results include: Step 1: Collect the time series data of waterlogged area in the target area during the historical period; Step 2: Preprocess the collected data, including cleaning data, supplementing missing data, and standardizing data; Step 3: construct a long short-term memory (LSTM) model structure, wherein the LSTM model structure includes two LSTM layers, and the number of cell units is 128 and 64 respectively; using the preprocessed waterlogging area time series data as input and the corresponding predicted waterlogging area time series data as output, the constructed LSTM model structure is trained to obtain a waterlogging trend prediction model; Step 4: Taking the time series data of the waterlogged area in the target area to be predicted during the target time period as input, the waterlogging trend prediction model is used to obtain the predicted result of the waterlogged area of the target area changing over time.