Water area contour recognition method, system, electronic equipment and medium

Abstract

The present invention relates to a method, system, electronic device, and medium for identifying water area contours, and relates to the technical field of contour recognition. The method comprises dividing the image to be identified into a grid to obtain multiple small grid maps; processing each small grid map using an SVM classifier to obtain recognition results for each small grid map, wherein the recognition results include: water areas and non-water areas; obtaining the water area of ​​the image to be identified based on the recognition results of each small grid map; cropping the image to be identified based on the water area to obtain a water area image; and inputting the water area image into a convolutional neural network to obtain the water area contour in the image to be identified. The present invention can improve the accuracy of contour recognition results.

Description

Technical Field

[0001] The present invention relates to the field of contour recognition technology, and in particular to a water area contour recognition method, system, electronic equipment and medium. Background Art

[0002] In order to protect the edge of water bodies such as rivers and lakes and prevent accidents, it is necessary to identify the specific location of the water edge in the image. The OpenCV contour recognition method can roughly identify some contours of the water edge, but it is greatly affected by weather interference, resulting in inaccurate contour results. Summary of the Invention

[0003] The purpose of the present invention is to provide a water area contour recognition method, system, electronic equipment and medium, which can improve the accuracy of contour recognition results.

[0004] To achieve the above object, the present invention provides the following solutions:

[0005] A water area contour recognition method, comprising:

[0006] Obtain the image to be recognized;

[0007] Dividing the image to be identified into a grid to obtain a plurality of small grid images;

[0008] Using an SVM classifier to process each small grid map to obtain a recognition result of each small grid map, the recognition result includes: water area and non-water area;

[0009] Obtaining the water area of ​​the image to be identified according to the recognition results of each small grid map;

[0010] Cropping the image to be identified according to the water area to obtain a water area picture;

[0011] The water area image is input into a convolutional neural network to obtain the water area contour in the image to be identified.

[0012] Optionally, the convolutional neural network includes:

[0013] The input layer, data normalization layer, first convolutional layer, Bottleneck unit, feature pyramid network layer, feature fusion layer, second convolutional layer and contour output layer are connected in sequence.

[0014] Optionally, the Bottleneck unit includes five sequentially connected Bottleneck layers.

[0015] Optionally, dividing the image to be identified into a grid to obtain a plurality of small grid images specifically includes:

[0016] compressing the image to be recognized to obtain a compressed image;

[0017] Performing grid segmentation on the compressed image to obtain a plurality of grid images;

[0018] Each of the grid maps is compressed separately to obtain a plurality of small grid maps.

[0019] A water area contour recognition system, comprising:

[0020] An acquisition module, used for acquiring an image to be identified;

[0021] A grid division module, configured to divide the image to be identified into a grid to obtain a plurality of small grid images;

[0022] An SVM classifier processing module is used to process each small grid map using an SVM classifier to obtain a recognition result of each small grid map, wherein the recognition result includes: a water area and a non-water area;

[0023] A water area determination module, configured to obtain the water area of ​​the image to be identified based on the identification results of each small grid map;

[0024] A picture cropping module, configured to crop the image to be identified according to the water area to obtain a water area picture;

[0025] The water area contour recognition module is used to input the water area image into a convolutional neural network to obtain the water area contour in the image to be recognized.

[0026] Optionally, the convolutional neural network includes:

[0027] The input layer, data normalization layer, first convolutional layer, Bottleneck unit, feature pyramid network layer, feature fusion layer, second convolutional layer and contour output layer are connected in sequence.

[0028] Optionally, the Bottleneck unit includes five sequentially connected Bottleneck layers.

[0029] Optionally, the grid division module specifically includes:

[0030] A first compression unit, configured to compress the image to be recognized to obtain a compressed image;

[0031] a segmentation unit, configured to perform grid segmentation on the compressed image to obtain a plurality of grid images;

[0032] The second compression unit is used to compress each of the grid images to obtain multiple small grid images.

[0033] An electronic device, comprising:

[0034] A memory and a processor, wherein the memory is used to store a computer program, and the processor runs the computer program to enable the electronic device to perform the water contour recognition method described above.

[0035] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the water area contour recognition method as described above.

[0036] According to the specific embodiments provided by the present invention, the present invention discloses the following technical effects: the present invention processes the edge area of ​​the water area in a deep learning manner to realize real-time identification of the edge position of the water area. Since the deep neural network itself has a stronger anti-interference ability than OpenCV, the contour recognition result obtained by the present invention is more accurate. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0038] Figure 1 A flow chart of a water area contour recognition method provided by an embodiment of the present invention;

[0039] Figure 2 An image to be recognized obtained in an embodiment of the present invention;

[0040] Figure 3 For the embodiment of the present invention Figure 2 A compressed image generated after compression;

[0041] Figure 4 For the embodiment of the present invention Figure 3 The image after meshing;

[0042] Figure 5 For the embodiment of the present invention Figure 4 The image obtained by compression;

[0043] Figure 6 For the embodiment of the present invention Figure 5 Images obtained by performing image batch recognition;

[0044] Figure 7 According to the embodiment of the present invention Figure 6 The image obtained by synthesizing the water area frame with the recognition results;

[0045] Figure 8 For the embodiment of the present invention Figure 7The image obtained by magnification;

[0046] Figure 9 For the embodiment of the present invention Figure 8 The image obtained by cropping the water area image in ;

[0047] Figure 10 A more specific structural diagram of a convolutional neural network provided by an embodiment of the present invention;

[0048] Figure 11 A structural diagram of a convolutional neural network provided by an embodiment of the present invention;

[0049] Figure 12 To use convolutional neural network Figure 9 The result of contour recognition on the image. DETAILED DESCRIPTION

[0050] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0051] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0052] An embodiment of the present invention provides a water area contour recognition method, comprising:

[0053] Get the image to be recognized.

[0054] The image to be identified is divided into grids to obtain a plurality of small grid images.

[0055] The SVM classifier is used to process each small grid map to obtain the recognition results of each small grid map, and the recognition results include: water area and non-water area. Specifically, the SVM classifier is used to identify each small grid map, and the classification threshold is set to 0.5. If it is greater than 0.5, it can be determined as a water block. If the SVM classification result threshold is not greater than 0.5, it means that there is no water area in this image. If the number of thresholds greater than 0.5 meets certain conditions, the coordinates of the SVM classification results are mapped and combined to finally form a water frame to obtain the water area.

[0056] The water area of ​​the image to be identified is obtained according to the identification results of each small grid image.

[0057] The image to be identified is cropped according to the water area to obtain a water area picture.

[0058] The water area image is input into a convolutional neural network to obtain the water area contour in the image to be identified.

[0059] In practical applications, before inputting the water area picture into a convolutional neural network to obtain the water area contour in the image to be identified, the method further includes: compressing the water area picture to obtain a compressed water area picture.

[0060] In practical applications, such as Figure 11 As shown, the convolutional neural network includes:

[0061] The input layer, data normalization layer, first convolution layer, Bottleneck unit, feature pyramid network layer, feature fusion layer, second convolution layer and contour output layer are connected in sequence. The Bottleneck unit includes 5 Bottleneck layers connected in sequence. The feature fusion layer is used to fuse the features of the last Bottleneck layer in the Bottleneck unit with the features of the pyramid network layer. The output result of the contour output layer is a matrix mask. The precise contour based on the mask water area is obtained through the OpenCV contour extraction method. The contour generated by the mask is converted into the contour coordinates of the original image through the image linear relationship.

[0062] In practical applications, when training convolutional neural networks, data can be enhanced by adding interference, such as salt and pepper noise, image rotation, distortion, network depth, etc., to increase the network's anti-interference ability.

[0063] In practical applications, the step of dividing the image to be identified into a grid to obtain a plurality of small grid images specifically includes:

[0064] The image to be recognized is compressed to obtain a compressed image.

[0065] The compressed image is grid-segmented to obtain a plurality of grid images.

[0066] Each of the grid maps is compressed separately to obtain a plurality of small grid maps.

[0067] An embodiment of the present invention further provides a water area contour recognition system corresponding to the above method, including:

[0068] The acquisition module is used to acquire the image to be recognized.

[0069] The grid division module is used to divide the image to be identified into grids to obtain multiple small grid images.

[0070] The SVM classifier processing module is used to process each small grid map using an SVM classifier to obtain a recognition result of each small grid map, wherein the recognition result includes: a water area and a non-water area.

[0071] The water area determination module is used to obtain the water area of ​​the image to be identified based on the recognition results of each small grid map.

[0072] The image cropping module is used to crop the image to be identified according to the water area to obtain a water area image.

[0073] The water area contour recognition module is used to input the water area image into a convolutional neural network to obtain the water area contour in the image to be recognized.

[0074] In practical applications, the convolutional neural network includes:

[0075] The input layer, data normalization layer, first convolutional layer, Bottleneck unit, feature pyramid network layer, feature fusion layer, second convolutional layer and contour output layer are connected in sequence.

[0076] In practical applications, the Bottleneck unit includes five sequentially connected Bottleneck layers.

[0077] In practical applications, the grid division module specifically includes:

[0078] The first compression unit is configured to compress the image to be recognized to obtain a compressed image.

[0079] The segmentation unit is used to perform grid segmentation on the compressed image to obtain multiple grid images.

[0080] The second compression unit is used to compress each of the grid images to obtain multiple small grid images.

[0081] An embodiment of the present invention further provides an electronic device, including:

[0082] A memory and a processor, wherein the memory is used to store a computer program, and the processor runs the computer program to enable the electronic device to perform the water contour recognition method described above.

[0083] An embodiment of the present invention further provides a computer-readable storage medium storing a computer program, which implements the water area contour recognition method as described above when the computer program is executed by a processor.

[0084] The present invention further illustrates the water area contour recognition method of the present invention with a more specific embodiment below:

[0085] like Figure 1As shown, the water area contour recognition method is divided into two parts: water area frame recognition and water area contour recognition.

[0086] Water frame identification:

[0087] Due to problems with OpenCV's recognition of water areas, it is impossible to use OpenCV to accurately obtain a box containing the entire water area on the image. Therefore, the location of the water area can be quickly obtained by segmenting the image and determining whether it is a water area.

[0088] Get the image to be identified, that is, RGB three-dimensional image data, such as Figure 2 shown.

[0089] The RGB three-dimensional image data is compressed (generating a 640x480 or 320x240 or 160x120 image, but the water area needs to be clearly presented in the generated image). The 640x480 image generated by the embodiment of the present invention is as follows: Figure 3 shown.

[0090] Since the water area is continuous and irregular, direct border recognition requires a lot of data calculation, so the idea of ​​​​averaging is used to divide the water area into two parts on the compressed map. Figure 3 The image is segmented, 100 small images are generated on the compressed image, and 10x10 small grid images C1 to C100 are generated by cropping. The results are as follows Figure 4 As shown, (for example, a 640x480 compressed image generates 10x10 small grids, so each small grid is 64x48 pixels).

[0091] The small images are compressed into a uniform size of 12x12. The results are as follows Figure 5 As shown, the small grid images are generated into a batch data in the order of 1 to 100 for image batch recognition, and then put into the SVM classifier to obtain the binary classification results of each small grid image to identify the small grid images containing water areas, and then the results are combined in order to present the water area location. The result is as follows Figure 6 As shown in the figure, the confidence level output by SVM (for a two-classification problem, the confidence level corresponding to each small picture can be expressed as a one-dimensional array [0.6, 0.4], 0.6 represents non-water area, 0.4 represents water area, 0.6 greater than 0.4 represents non-water area, [0.1, 0.9] represents water area picture) is used to judge whether it is a water area. The recognition result is calculated based on the small picture Cn in Figure 2 The relative position in (0-1 represents the image position in percentage form, such as n is 0, which means in Figure 2 On the left side, n is 1, which means Figure 2 The right area of ​​the original image is combined and mapped to form the water area frame.

[0092] According to the recognition results of SVM classification, the top, leftmost, rightmost and bottom four small grids are obtained. The outermost recognition grid is used to obtain the circumscribed rectangular frame. According to the circumscribed rectangular frame, a small grid image is expanded in four directions to obtain the rectangular position that completely contains the water area, namely the water area frame. The result is as follows Figure 7 As shown, if the water area frame position is not obtained, it proves that there is no water area in the current image, and the undetected result is directly output. If the water area frame position information is available, the water area contour recognition is entered.

[0093] The obtained water area frame is divided into two parts by a proportional relationship (according to Figure 4 and Figure 2 The length and width corresponding ratio relationship) is linearly mapped to the original Figure 2 The actual location of the water area frame is obtained, and the result is as follows Figure 8 shown.

[0094] Water contour recognition:

[0095] Based on the water area frame identified in the above steps, the water area image is cropped from the original image and linearly compressed to 224x224 pixels. The result is as follows Figure 9 As shown, when predicting, Figure 9 The convolutional neural network is sent to perform normalization and contour recognition. In this embodiment, the network parameters of the convolutional neural network are as follows: Figure 10 As shown in the figure, through the convolution operation of the neural network, the characteristics of the water area are extracted for model feature extraction and regression to generate a mask. The mask is mapped to the original image size to generate the coordinates of the water area outline. The contour area of ​​the water area is identified by synthesizing features at different levels. This recognition is pixel-level recognition. The optimized mobilenetV2 is used as the backbone to extract image features. The FPN single-layer algorithm is added to generate the feature map. The convolution Conv2d operation generates a 28x28 contour mask. By linearly enlarging it to the size of the water area box on the original image, the water area outline is finally generated. The result is shown in the figure. Figure 12 shown.

[0096] After synthesizing the water frame, the scope of the water area is determined. Especially for small water areas in the distance, if the water area is directly identified without model feature extraction, the water area outline can basically not be extracted. Therefore, by generating the water frame, the distant water area is effectively brought closer, achieving almost the same recognition effect for both near and far areas.

[0097] like Figure 10 As shown in the figure, the Augmentation layer, i.e. the data normalization layer, is mainly used to enhance the data and increase the network's anti-interference ability.

[0098] Conv2d and Bottleneck are common convolution operations in MobileNet that can effectively extract multi-dimensional features. t represents the "dilation" factor, c represents the number of output channels, n represents the number of repetitions, and s represents the stride. The composition of Bottleneck is shown in Table 1:

[0099] Table 1 Bottleneck composition

[0100]

[0101] dwise stands for Depthwise. Convolution is a general convolution operation, where one convolution kernel is responsible for one channel, and a channel is convolved with only one convolution kernel. linear represents linearity, and ReLU6 is a general activation function.

[0102] Mask is the contour mask, and the output is a 28x28 matrix. The internal values ​​of the matrix are 0 and 1, 1 represents the water area, and the switching between 1 and 0 represents the contour position.

[0103] In L7, only one layer of FPN operation is used, and the nonlinearity of the network is increased through upsampling and 1x1 conv2d operations.

[0104] L8 is a concatenation operation. By obtaining the output of the L7 layer and fusing it with the features of the L6 layer, the layer and channel depth are deepened, and the feature expression is deeper, more accurate, and more resistant to interference.

[0105] The depth of the network is more nonlinear and can fit more angles, with more outstanding performance in multi-angle aspects. However, the depth of the network itself is positively correlated with the amount of computation. When the params is within 3.4M, the running speed on the CPU can reach 80ms.

[0106] The present invention also provides a specific embodiment of the above method:

[0107] Operating platform: Mainly Windows and Linux, but algorithm packages can be customized for different platforms. This algorithm is easy to deploy. Different algorithm packages can be developed for different system environments, but the deployment steps are the same as follows:

[0108] Step 1: Build the algorithm environment. The algorithm environment mainly includes basic neural network dependency libraries and algorithm development kits, mainly on Windows and Linux platforms.

[0109] Step 2: Build the backend server

[0110] Step 3: The server system accesses the camera data that needs to be identified in real time.

[0111] Step 4: Based on the above algorithm technology, encode the so or jar algorithm package.

[0112] Step 5: Connect the algorithm package to the server.

[0113] Step 6: Start the algorithm, call the camera to pull the camera image data for recognition, output the recognition algorithm, and connect to subsequent operations.

[0114] The present invention has the following technical effects:

[0115] 1. More accurate identification of water contours. Since deep neural networks have stronger anti-interference capabilities than OpenCV, interference enhancement of data during training, such as adding salt and pepper noise, image rotation, distortion, and network depth, can further enhance the network's anti-interference capabilities and achieve better pixel-level recognition and judgment. Traditional OpenCV requires manual parameter adjustment, making it difficult to achieve adaptive detection, resulting in lower accuracy.

[0116] 2. Neural networks are nonlinear. In terms of data processing, screening out effective data from multiple angles for training can achieve multi-angle recognition, so that one model can recognize multiple angles.

[0117] 3. It can identify at a long distance. The long distance can be judged by the initial water area, and then the distant water area can be magnified and the close water area can be reduced for identification.

[0118] 4. Wider application: the algorithm can be used not only for water areas but also for other contour recognition, but the training data needs to be modified.

[0119] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. Reference can be made to the common and similar parts between the various embodiments. For the systems disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple, and the relevant parts can be referred to the method description.

[0120] This document uses specific examples to illustrate the principles and implementation methods of the present invention. The above examples are only intended to help understand the method and core concept of the present invention. At the same time, those skilled in the art will find that the specific implementation methods and application scopes may vary based on the concept of the present invention. In summary, the contents of this specification should not be construed as limiting the present invention.

Claims

1. A method for identifying water area contours, characterized in that: include: Obtain the image to be recognized; Dividing the image to be identified into a grid to obtain a plurality of small grid images; Using an SVM classifier to process each small grid map to obtain a recognition result of each small grid map, the recognition result includes: water area and non-water area; The water area of ​​the image to be identified is obtained based on the recognition results of each small grid image; specifically, the top, leftmost, rightmost, and bottom four small grid images of the water area are obtained as the recognition results, a circumscribed rectangular frame is obtained based on the top, leftmost, rightmost, and bottom four small grid images, and a small grid image is expanded in four directions based on the circumscribed rectangular frame to obtain the water area of ​​the image to be identified; Cropping the image to be identified according to the water area to obtain a water area picture; Inputting the water area image into a convolutional neural network to obtain the water area contour in the image to be identified; The convolutional neural network includes: The input layer, data normalization layer, first convolutional layer, Bottleneck unit, feature pyramid network layer, feature fusion layer, second convolutional layer and contour output layer are connected in sequence.

2. A water area contour recognition method according to claim 1, characterized in that: The Bottleneck unit includes five Bottleneck layers connected in sequence.

3. A water area contour recognition method according to claim 1, characterized in that: The step of dividing the image to be identified into a grid to obtain a plurality of small grid images specifically includes: compressing the image to be recognized to obtain a compressed image; Performing grid segmentation on the compressed image to obtain a plurality of grid images; Each of the grid maps is compressed separately to obtain a plurality of small grid maps.

4. A water area contour recognition system, characterized in that: include: An acquisition module, used to acquire an image to be identified; A grid division module, configured to divide the image to be identified into a grid to obtain a plurality of small grid images; An SVM classifier processing module is used to process each small grid map using an SVM classifier to obtain a recognition result of each small grid map, wherein the recognition result includes: a water area and a non-water area; a water area determination module, configured to obtain the water area of ​​the image to be identified based on the recognition results of each small grid image; specifically, obtaining the top, leftmost, rightmost, and bottom four small grid images whose recognition results are the water area, obtaining a circumscribed rectangular frame based on the top, leftmost, rightmost, and bottom four small grid images, and expanding a small grid image in four directions based on the circumscribed rectangular frame to obtain the water area of ​​the image to be identified; A picture cropping module, configured to crop the image to be identified according to the water area to obtain a water area picture; A water area contour recognition module is used to input the water area image into a convolutional neural network to obtain the water area contour in the image to be recognized; The convolutional neural network includes: The input layer, data normalization layer, first convolutional layer, Bottleneck unit, feature pyramid network layer, feature fusion layer, second convolutional layer and contour output layer are connected in sequence.

5. A water area contour recognition system according to claim 4, characterized in that: The Bottleneck unit includes five Bottleneck layers connected in sequence.

6. A water area contour recognition system according to claim 4, characterized in that: The grid division module specifically includes: A first compression unit, configured to compress the image to be recognized to obtain a compressed image; a segmentation unit, configured to perform grid segmentation on the compressed image to obtain a plurality of grid images; The second compression unit is used to compress each of the grid images to obtain multiple small grid images.

7. An electronic device, characterized in that: include: A memory and a processor, wherein the memory is used to store a computer program, and the processor runs the computer program to enable the electronic device to perform the water area contour recognition method according to any one of claims 1 to 3.

8. A computer-readable storage medium, characterized in that The device stores a computer program, which, when executed by a processor, implements the water area contour recognition method according to any one of claims 1 to 3.

Images