Intelligent water level detection method based on deep learning

Abstract

The application relates to the technical field of water level detection, and specifically discloses an intelligent water level detection method based on deep learning, which comprises the following steps: firstly, a multi-level feature fusion method for strengthening small-scale features is proposed to improve the deep learning YOLOv5 algorithm, so that the capturing capacity of the algorithm for small targets such as water gauge scales is strengthened; then, a compression activation network (RankSENet module) and a Bottleneck Transformer module are fused to further improve the perception capacity for water gauge scales; finally, a brand-new water level elevation solution is proposed, accurate water level elevation information can be obtained only by using part of the water gauge scale anchor frame information, and the robustness of the detection method is greatly improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of water level detection, and particularly relates to a water level intelligent detection method based on deep learning. BACKGROUND

[0002] At present, global climate change is severe, with the frequent occurrence of extreme rainfall climate and the increasingly obvious urban heat island effect, urban waterlogging disasters in China are more and more frequent. It can be seen that hydrological data is closely related to human social life and ecological environment, and water level detection plays an important role in water conservancy projects, flood warning and other fields, and it is of great significance to realize intelligent water management and control and flood disaster warning.

[0003] However, the existing water level monitoring methods mainly use manual measurement or install water level meters to measure the water level, and these monitoring methods have many shortcomings: manual reading cannot guarantee real-time and accuracy, and the labor cost is high; the accuracy and stability of the water level meter device are poor, and long-time contact with the water surface will cause external rust and damage, thereby increasing the maintenance cost, and it is difficult to meet the intelligent and refined needs of modern water management and control.

[0004] With the continuous advancement of informationization process, image processing technology has developed rapidly, and the method of water level detection by image means has attracted widespread attention. However, the water gauge scale is a "small target" object in the detection image, although the deep learning target detection technology has achieved excellent results, but the existing target detection technology still has serious challenges in detecting small targets. Therefore, there is an urgent need for a water level detection method with high accuracy and strong generalization ability. SUMMARY

[0005] The present application provides a water level intelligent detection method based on deep learning to overcome the technical defects of low precision, poor stability and high cost of existing water level detection methods.

[0006] The present application is realized by adopting the following technical solutions:

[0007] A water level intelligent detection method based on deep learning includes the following steps:

[0008] Step 1, collect water level images, collect various types of water level images by field interval collection, and make a data set.

[0009] Step 2, select markers and label, wherein various types of labels have different functions:

[0010] E type markers are divided into positive E0 and negative E1, and according to the elevation information of the configuration E, the rough ratio K' of each pixel to the actual elevation in the vertical direction is obtained.

[0011] A digital class (N0) mark, according to the scale information of the complete number corresponding to E at the top of the water gauge, the pixel coordinates are connected with the world coordinates; specifically, the digital class includes numbers 0-9, which are respectively denoted as N0-N9.

[0012] A water level class (W) mark, the water level line and the image coordinates of the water level line in the vertical direction are obtained, and then the actual elevation is converted; specifically, the center of the water level mark box is attached to the water level line, and the mark boxes are kept continuous, which is denoted as W.

[0013] A water gauge class (R) mark, whether the E class and the digital class object are legal (that is, whether the target is in a legal area) is judged; specifically, the water gauge mark box horizontally contains the water gauge and vertically includes the water gauge, which is denoted as R.

[0014] And the labeled image is divided into training set I and test set II according to the proportion.

[0015] Step 3, input the training set I after labeling into the target detection neural network for training, and extract the training model; the performance of the training algorithm is tested by using the test set II.

[0016] The target detection neural network selects YOLOv5 network structure as the deep learning backbone network, and fuses RankSENet module and Bottleneck Transformer module.

[0017] Specifically, the YOLOv5 network structure is divided into four parts of input layer, benchmark network layer, neck feature fusion network layer and detection head.

[0018] Among them: the input layer is responsible for image preprocessing, including image adaptive scaling, Mosaic data enhancement, and adaptive anchor box calculation operation.

[0019] The benchmark network layer adopts a CSPDarknet53 structure as a benchmark, and a Focus structure is introduced to slice the image, then spliced in the channel direction, and a RankSENet module is introduced after the CSP module at the end of the benchmark network layer, and a Bottleneck Transformer module is introduced after the SPP module, after the data passes through the CSP module at the end of the benchmark network layer, in turn passes through the RankSENet module, the SPP module and the Bottleneck Transformer module, and then is transmitted into the neck feature fusion network layer. The RankSENet module includes an SENet network structure, which performs compression, activation and feature re-labeling operations on a feature map with C input channels; wherein, during the compression operation, the rank operation is combined with the global average pooling operation of the SENet network: the input feature map matrix is compressed into a real number A by the global average pooling operation; at the same time, the rank of the background feature map matrix is calculated, and the rank of the feature map matrix with scale information or water line is calculated, and the difference between the two is obtained to obtain the variable B of the rank of the feature map matrix, and the product AB is obtained to reflect the change degree of the feature map after adding the scale information or water line, that is, the real number AB reflects the potential importance of the channel feature, and the output dimension after compression is consistent with the input feature channel C.

[0020] The two transfer nodes of the mesoscale feature mapping and the large-scale feature mapping of the neck feature fusion network layer are merged into the small-scale feature mapping channel for close and sufficient fusion of the small-scale feature mapping.

[0021] The detection head main part is three detectors, each of which uses the prior box set in the grid to detect the target on the feature map of different scales, and finally selects the prior box with the highest probability score in the three detectors as the final prediction box.

[0022] Step 4, input the data set collected in the actual detection task into the target detection neural network trained in step 3, and extract the rectangular anchor box information output by the network; so as to obtain various parameters before actual water level elevation conversion: take any one non-top E class mark in the water gauge as x1, record the actual height of x1 center as H top , the longitudinal coordinate of x1 as y x1 , the number of E class marks between the top E and x1 as a; take the center horizontal coordinates of the adjacent two legal positive E and negative E targets as x water , and record the longitudinal coordinate of the water line as y water ; actually measure the actual height h e of each E class mark in the water gauge and the actual scale h top of the E class mark of the complete digital at the top of the water gauge.

[0023] Step 5, input the data set collected in the actual detection task into the target detection neural network trained in step 3, obtain the minimum value R xmin and the maximum value R xmax of the horizontal abscissa of the water gauge R detected by the network, and judge whether other detection objects except the water level class W are legal, that is, whether the horizontal abscissa of the detection target exists between (R xmin , R xmax ). If it exists, it is a legal object, otherwise it is an illegal object, and the illegal object is not calculated; if no R target is detected, the system inherits the last R object coordinate information.

[0024] Step 6, input the data set collected in the actual detection task into the target detection neural network trained in step 3, obtain the water level line prediction box detected by the network and sample the center coordinates, and then perform least squares fitting, that is, obtain the expression formula of the water level line; then detect whether the digital class (N class) target exists, and if it exists, proceed to the next step. If no N class target or W class target is detected, the system inherits the last water level actual elevation value, and the operation is completed.

[0025] Wherein, the expression formula of the water level line is y=ax+b, wherein a is the slope of the fitted straight line, and b is the intercept of the fitted straight line.

[0026] Least squares fitting, the fitting formula is Wherein, x i , y i is the center coordinate of the water level line prediction box, n is the number of water level line prediction boxes, is the average of the center coordinates of the water level line prediction box.

[0027] Step 7, input the data set collected in the actual detection task into the target detection neural network trained in step 3, obtain the information of the E class mark detected by the network, and obtain the mapping relationship between the actual height and the pixel height by using the single E actual height and the E pixel height, that is, the rough ratio K' If no E class target is detected, the system calculates by using the single N class target actual height and its pixel height.

[0028] Step 8, input the data set collected in the actual detection task into the target detection neural network trained in step 3, obtain the information of the N class target detected by the network, and judge the number of N class objects. If the number is greater than 1, calculate the ratio of the actual length and the pixel length between the top N class object and the bottom N class object, that is, the accurate ratio K; if the number is only 1, replace the accurate ratio K with the rough ratio K'.

[0029] Step 9, obtain the reference calculation coordinates of the fitting water level line, and the center horizontal coordinate x between adjacent two legal positive E and negative E targets water Substitute the expression formula y = ax + b of the water level line to obtain the current water level line longitudinal coordinate y water .

[0030] Step 10, calculate the number a of E class marks between the top E of the water gauge and x1, and utilize the actual scale h of the top E top And the actual height h of the E class mark e , calculate the x1 center actual height H top , and the calculation formula is H top = h top -h e ×a, and further calculate the water level actual height H;

[0031] The water level height value H calculation formula is: H = H top -(y x1 -y water )×K, wherein y x1 is the x1 corresponding longitudinal coordinate; K is the accurate ratio, and H top is the x1 center actual height.

[0032] Compared with the prior art, the technical scheme provided by the present application has the following advantages:

[0033] Compared with the method of measuring the water level by artificial measurement and installing a water level gauge, the water level intelligent detection method based on deep learning provided by the present application only needs to arrange a camera and related accessories at an observation point, does not affect the topography, does not interfere with the water ecology, does not pollute the water source, has lower cost, has higher intelligent degree, is more clean and environmentally friendly; in addition, the method does not need to detect all water level line regions, and through the sampling and fitting mode, the water level line close to the real water level is obtained, the problem that the water level line prediction in some regions is invalid due to the existence of foreign matter shielding on the water surface is solved to a certain extent, and the robustness is stronger; the YOLOv5 feature extraction capability of the fusion of the RankSENet and the BottleneckTransformer module is stronger, the target recognition accuracy is higher, the water gauge scale feature expression is more significant, and the irrelevant information such as background is greatly weakened.

[0034] The present application has reasonable design, good practical application and popularization value. DETAILED DESCRIPTION

[0035] The accompanying drawings, which are incorporated into and form part of the specification, illustrate embodiments consistent with the present application and, together with the specification, serve to explain the principles of the present application.

[0036] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed to be used in the embodiments or prior art description will be briefly introduced as follows. Obviously, for those skilled in the art, other drawings can also be obtained based on these drawings without any creative effort.

[0037] Figure 1 A flow chart of the water level intelligent detection method based on deep learning of the present application is shown.

[0038] Figure 2 A water gauge image labeling schematic diagram in step 2 of the embodiment of the present application is shown.

[0039] Figure 3 An existing YOLOv5 network structure diagram in step 3 of the embodiment of the present application is shown.

[0040] Figure 4 An existing SENet network structure diagram in step 3 of the embodiment of the present application is shown.

[0041] Figure 5 A principle schematic diagram of the importance of feature channel in step 3 of the embodiment of the present application is shown.

[0042] Figure 6 An improved RankSENet network structure diagram in step 3 of the embodiment of the present application is shown.

[0043] Figure 7 A comparison diagram of residual network and BoTNet structure in step 3 of the embodiment of the present application is shown.

[0044] Figure 8 An existing MHSA structure schematic diagram in step 3 of the embodiment of the present application is shown.

[0045] Figure 9 An improved YOLOv5 network structure (i.e. target detection neural network) schematic diagram in step 3 of the embodiment of the present application is shown.

[0046] Figure 10 A parameter configuration schematic diagram of the water level intelligent detection method based on deep learning of the present application is shown. DETAILED DESCRIPTION

[0047] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed to be used in the embodiments or prior art description will be briefly introduced as follows. Obviously, for those skilled in the art, other drawings can also be obtained based on these drawings without any creative effort.

[0048] Many specific details are set forth in the following description in order to provide a thorough understanding of the application. However, the application can be practiced according to other embodiments that can not be described in detail herein; and, obviously, the description is not intended to limit the application to the embodiments described, but to provide an adequately enabling description for the practice of the application.

[0049] The specific embodiments of the application will be described in detail below with reference to the accompanying drawings.

[0050] To realize intelligent flood disaster warning, it is necessary to automatically and accurately perceive the change of water level information. However, the existing water level recognition technology has poor sensing ability for water gauge scale. Therefore, the application proposes a water level intelligent detection method based on deep learning technology. First, a multi-level feature fusion method for strengthening small scale features is proposed to improve the deep learning YOLOv5 algorithm, so as to strengthen the capture ability of the algorithm for small targets such as water gauge scale; then, the improved compression activation network (RankSENet module) and the existing Bottleneck Transformer module are fused to further improve the perception ability for water gauge scale; finally, a new water level elevation solution is proposed, which can obtain accurate water level elevation information only by using part of the water gauge scale anchor box information, thereby greatly improving the robustness of the detection method. The specific process of the detection method is shown in Figure 1 The specific implementation scheme is as follows:

[0051] Step 1, collect water level images, collect various types of water level images by field interval collection, and make a data set.

[0052] Specifically, various types of water level images are collected in two scenes (scene A and scene B) by field interval collection. Scene A is a certain suburban river, and the insufficient night light is compensated by external light irradiation; scene B is a certain reservoir, and the water gauge image is collected by infrared function at night. The picture clarity of scene A is relatively high, but there are many shadows in the image due to direct light, and there are foreign objects in the water gauge area, and the suburban scene is also relatively complex, which brings certain challenges to target detection; the water gauge part of scene B is complete, but the clarity of the night image is not high due to the use of infrared collection at night, and the factors such as infrared overexposure seriously affect the image. The characteristics of the two scenes are different, which meets the sample diversity of the data set.

[0053] Step 2, select markers and label, wherein various types of labeling have different functions: E type markers on the water gauge are divided into positive E0 and negative E1, according to the elevation information of the configured E, the rough ratio K' of each pixel in the vertical direction to the actual elevation is obtained. The digital type (N0) is configured according to the scale information of the complete number at the top of the water gauge, and the pixel coordinates and world coordinates are connected. The water level type (W) obtains the water level line and the image coordinates of the water level line in the vertical direction, and then converts the actual elevation. The water gauge type (R) judges whether the E type and digital type objects are legal (i.e. whether the target is in a legal area).

[0054] and the labeled images are proportionally divided into training set I and test set II. The water gauge image labeling is as shown in Figure 2 .

[0055] Specifically, the digital class contains digits 0-9, denoted as N0-N9; the water level labeling box is attached to the water level line, and the labeling boxes are kept continuous, denoted as W; the water gauge labeling box horizontally contains the water gauge and vertically includes the water gauge, denoted as R.

[0056] Step 3, input the labeled training set I into the target detection neural network for training, and extract the training model. Test the performance of the training algorithm using test set II. The related experimental results show that the water level detection accuracy of this algorithm is 98.5%, which is 8.4% higher than that of the original algorithm.

[0057] Specifically, the target detection neural network selects the existing YOLOv5 network structure as the deep learning backbone network in order to improve the small target detection performance; the improved compression activation network and the Bottleneck Transformer module are fused to further improve the perception ability of the water gauge scale.

[0058] 3.1, the existing YOLOv5 network structure is as shown in Figure 3 , mainly divided into input layer, benchmark network layer, neck feature fusion network layer, and detection head. Among them: the input layer is responsible for image preprocessing, including image adaptive scaling, Mosaic data enhancement, adaptive anchor box calculation and other operations; the benchmark network layer uses CSPDarknet53 structure as the benchmark, and introduces Focus structure to slice the image, then splices in the channel direction, reduces the loss of original information, and at the same time, reduces the network calculation amount; the neck feature fusion network layer uses SPP module, which solves the problem of losing part of the important information of the picture when the image is cropped to a fixed size; uses FPN plus PAN feature fusion module, through upsampling, the high-level feature information is fused with the bottom-level feature, and then through the bottom-up fusion method, the diversity and robustness of feature mapping are improved, finally the fused large, medium and small three different scale feature mappings are sent to the detection head for prediction; the detection head main part is three detectors, each detector uses the prior box set in the grid to detect the target in different scale feature mappings, and finally selects the prior box with the highest probability score in the three detectors as the final prediction box.

[0059] 3.2, the traditional convolutional network realizes the classification and positioning of the target by perceiving the output of each feature channel information, and the Squeeze-and-Excitation Networks (SENet) improves the representation ability of the feature channel containing target information by redistributing the weight of each feature channel, and the accuracy of the model can be effectively improved by integrating the SENet into the YOLOv5 network.

[0060] The SENet network structure is shown in Figure 4 The three operations of compression, activation and feature re-labeling are performed on the input feature map (channel number C), the importance of each feature channel is automatically obtained by learning the correlation between each channel, then the features with high effect on the current task are improved and the features with low effect are suppressed according to the importance, so as to improve the accuracy of the model. The compression operation means that the SENet compresses each two-dimensional feature channel into a real number through global average pooling operation, which reflects the potential importance of the channel feature, and the output dimension after compression is consistent with the input feature channel C. The compression operation formula is:

[0061]

[0062] In the formula, X(i,j) is the input feature map, h and w are the height and width of the feature map respectively, and f(X) is the compressed real number.

[0063] However, in the actual water gauge scale detection task, the water gauge scale target is relatively subtle, and its area is relatively small compared to the whole water level image, which leads to less feature information of the water gauge scale contained in the feature map extracted by the detection algorithm through convolution. After the SENet compression (global average pooling) operation, the water gauge scale information is easily eliminated, which makes the obtained real number unable to accurately reflect the potential importance of the channel.

[0064] In a local range, the color of the water gauge surface and the water gauge background (such as the wall surface) is consistent, and the "scale mark" is obviously different in visual attribute. Obviously, when the high-dimensional features of the target are extracted by the convolutional neural network, the background information of the feature map may have little difference, which means that the output feature map matrix is approximately a low-rank matrix. For convenience of explanation and understanding, it is assumed that the background color of the local image is consistent, then the output feature map information of the convolutional network is also consistent. When this local area contains the water gauge target, it is equivalent to adding irregular new information to the output feature map, which increases the complexity of the feature map matrix. For example, Figure 5As shown, the circle is the water gauge image information, and since the water gauge scale image features are less, the real number obtained by the global average pooling operation of the SENet cannot well reflect the change degree of the feature map after adding the water gauge scale. However, due to the addition of the water gauge scale information, the original low-rank matrix becomes a high-rank matrix, that is, the rank of the feature map is compared to the real number obtained by the global average pooling, and the change degree of each feature channel can be better reflected.

[0065] However, when the detection target is the water level line, since the water level line features run through the entire picture, the feature matrix obtained after the rank operation still maintains a low rank, and cannot well reflect the importance of the feature channel. However, in this case, the feature information of the water level line is often distributed more.

[0066] Therefore, the SENet network is improved as follows in the present application: Figure 6 As shown, the rank operation is combined with the global average pooling operation of the original SENet network: the input feature map matrix is compressed into a real number A by the global average pooling operation; at the same time, the rank of the background feature map matrix is calculated, and the rank of the feature map matrix with the scale information or the water level line is calculated, and the difference between the two is obtained. The variable B of the rank of the feature map matrix is obtained, and the product AB is used to reflect the change degree of the feature map after adding the scale information or the water level line. Instead of the original average pooling operation on the feature map, in the case of more or less feature distribution of the detection object, the change degree of the compressed real number can be effectively improved. The new SENet is named as RankSENet module in the present application. The present application only improves the compression operation in the SENet network, and the activation and feature rescaling operations remain unchanged.

[0067] 3.3, Bottleneck Transformer module

[0068] The traditional convolution operation extracts features by sliding on the input data, and its advantage lies in parameter sharing and efficient fusion of local information. However, this operation can only extract local features. In order to integrate global information, the traditional convolution network needs to stack a large number of convolution layers, resulting in an increase in the number of parameters. However, the Transformer mechanism can extract global features at one time due to its unique structure. As shown, Figure 7 As shown, the Bottleneck Transformer module combines convolution operation and multi-head attention mechanism (Multi-Head Self-Attention, MHSA) to ensure the locality and globality of the extracted features. The present application introduces the existing Bottleneck Transformer module into the end of the CSP structure of the benchmark network layer, increases the feature extraction capability of the model, and reduces the parameter quantity and computational quantity.

[0069] The core of the Bottleneck Transformer module is an MHSA structure, such as Figure 8 As shown, the MHSA structure takes a feature map of input size H×W×d (image length, width, and number of channels, respectively) and obtains W through a 1×1 convolution. q W k W v Matrix (q, k, v represent query, key, and weight, respectively); R h R w The relative positional codes for the image height and width are respectively added together and then multiplied by W. q Multiplication, fusing image content and location information; W q W k Multiplication completes the encoding and fusion of image information; the two matrix results are added together, and the resulting matrix is ​​then subjected to softmax normalization, with the output matrix format being H*W×H*W; finally, the output value is compared with the weight matrix W. v Multiplication completes the fusion of global and local features of the image.

[0070] 3.4 The target detection neural network described in this invention (i.e., the improved YOLOv5 network structure)

[0071] According to convolution theory, shallow feature maps contain more spatial and detailed information about the image, which is beneficial for target localization, while deep feature maps contain more semantic information about the image, which is beneficial for target classification. Based on different receptive fields, YOLOv5 detects small targets through small-scale mappings. Since water level scale objects are small targets, it is necessary to improve YOLOv5's ability to recognize and localize small-scale mappings.

[0072] The target detection neural network described in this step is the improved YOLOv5 network structure, such as... Figure 9 As shown, the YOLOv5 network structure consists of four parts: input layer, baseline network layer, neck feature fusion network layer, and detection head.

[0073] The input layer is responsible for image preprocessing, including adaptive image scaling, Mosaic data augmentation, and adaptive anchor box calculation.

[0074] The benchmark network layer adopts the CSPDarknet53 structure as a benchmark, and a Focus structure is introduced to slice the image, then spliced in the channel direction, and a RankSENet module is introduced after the CSP module at the end of the benchmark network layer, and a Bottleneck Transformer module is introduced after the SPP module, and after the data passes through the CSP module at the end of the benchmark network layer, it successively passes through the RankSENet module, the SPP module and the Bottleneck Transformer module, and then is transmitted into the neck feature fusion network layer.

[0075] The two transmission nodes of the mesoscale feature mapping and the large-scale feature mapping of the neck feature fusion network layer are merged into the small-scale feature mapping channel, so as to tightly and fully fuse the small-scale feature mapping, and improve the ability of the small-scale feature mapping to transmit spatial information, detailed information and semantic information.

[0076] The detection head main part is three detectors, each of which uses the prior box set in the grid to detect the target on the feature mapping of different scales, and finally selects the prior box with the highest probability score in the three detectors as the final prediction box.

[0077] The improved YOLOv5 network structure has more advantages in water gauge recognition, increases the merging node of the small-scale feature mapping, realizes more sufficient and closer fusion, enriches the small-scale mapping feature information, and improves the ability of the small-scale feature mapping to recognize the water gauge scale.

[0078] Using the deep learning target detection algorithm can only obtain a plurality of rectangular anchor boxes containing target coordinates and category information, and cannot obtain accurate water level elevation values. Moreover, due to the poor performance of the algorithm itself or the influence of the shooting scene, the algorithm cannot identify all the targets to be detected. Therefore, in order to convert the output anchor box information into accurate actual water level elevation information and meet the algorithm application performance, the present application further proposes a new water level elevation conversion scheme, and forms a water level detection overall method based on the same, which will be described in detail in steps 4-10.

[0079] Step 4, input the data set collected in the actual detection task into the target detection neural network trained in step 3, and extract the rectangular anchor box information output by the network. Obtain various parameters before actual water level elevation conversion: take any one E class mark (non-top E) in the water gauge as x1, record the actual height of the center of x1 as H top , the longitudinal coordinate of x1 is y x1 , the number of E class marks between the top E and x1 is a; take the center horizontal coordinates of the adjacent two legal positive E and negative E targets as x water , and record the longitudinal coordinate of the water level line as y water ; actually measure the actual height h of each E class mark in the water gaugee and the actual scale h of the E-class mark of the complete number at the top of the water gauge top , as shown in Figure 10 .

[0080] Step 5, input the data set collected in the actual detection task into the target detection neural network trained in step 3 to obtain the minimum value R xmin and the maximum value R xmax of the horizontal abscissa of the water gauge (R) detected by the network, and determine whether the other detection objects except the water level class (W) are legal, that is, whether the detection target horizontal abscissa exists between (R xmin , R xmax ). If it is within the region, it is a legal object, otherwise it is an illegal object, and the illegal object is not calculated. If no R target is detected, the system inherits the last R object coordinate information.

[0081] Step 6, input the data set collected in the actual detection task into the target detection neural network trained in step 3 to obtain the water level line prediction box detected by the network and sample the center coordinates, and then perform least squares fitting to obtain the expression formula of the water level line; then determine whether the digital class (N class) target exists, and if it exists, proceed to the next step. If no N class target or W class target is detected, the system inherits the last water level actual elevation value, and the operation is completed.

[0082] As can be easily understood, the water level line expression formula is y = ax + b, where a is the slope of the fitted straight line, and b is the intercept of the fitted straight line.

[0083] The linear fitting uses the least squares method, and the fitting formula is where x i , y i are the center coordinates of the water level line prediction box, n is the number of water level line prediction boxes, is the average of the center coordinates of the water level line prediction box.

[0084] Step 7, input the data set collected in the actual detection task into the target detection neural network trained in step 3 to obtain the information of the E-class mark detected by the network, and use the actual height of a single E and the pixel height of E to obtain the mapping relationship between the actual height and the pixel height, that is, the rough ratio If no E class target is detected, the system calculates using the actual height of a single N class target and its pixel height.

[0085] Step 8: Input the dataset collected in the actual detection task into the target detection neural network trained in Step 3, obtain information on the N types of targets detected by the network, and determine the number of N types of objects. If the number is greater than 1, calculate the ratio of the actual length to the pixel length between the top N types of objects and the bottom N types of objects, i.e., the precision ratio K. If the number is only 1, use the coarse ratio K' instead of the precision ratio K.

[0086] Step 9: Obtain the baseline calculation coordinates of the fitted water level line, and set the center x-coordinate between two adjacent valid positive and negative E targets. water Substituting into the formula for water level, y = ax + b, we can obtain the ordinate of the current water level. water .

[0087] Step 10: Calculate the number 'a' of Class E markers between the top E and x1 on the water gauge, using the actual graduation h of the top E. top And the actual height h of the Class E sign e Calculate the actual height H of the center of x1. top The calculation formula is H top =h top -h e ×a, and then calculate the actual water level elevation H.

[0088] The formula for calculating the water level elevation H is: H = H top -(y x1 -y water )×K, where y x1 x1 represents the ordinate; K is the exact ratio, H... top The actual elevation of the center of x1.

[0089] Step 11: Design relevant comparative experiments to verify the superior performance of the target detection neural network described in this invention.

[0090] Specifically, to verify the superior performance of the network model that integrates RankSE, Bottleneck Transformer, and the improved YOLOv5, comparative experiments were designed. The comparative models were YOLOv5, improved YOLOv5, SE-improved YOLOv5 (integrating SENet and improved YOLOv5), RankSE-improved YOLOv5 (integrating RankSENet and improved YOLOv5), and BOTNet-RankSE-improved YOLOv5 (integrating Bottleneck Transformer, RankSENet, and improved YOLOv5).

[0091] During the experiment, the super parameter values are consistent: the batch size is set to 32, the initial learning rate is 0.01, the cosine annealing algorithm is used to update the learning rate, and the maximum number of iterations is 500. The experimental hardware scene: Intel Xeon Gold 5218R CPU, 256G memory, Nvidia Quadro RTX6000 24GB GPU. Software scene: 64-bit Windows 10, Python 3.8+Pytorch1.7.1.

[0092] Step 12, evaluation index and result analysis

[0093] Generally, the target detection algorithm model mainly evaluates the performance of the target detection algorithm through average precision (AP), mean average precision (mAP), accuracy and recall rate. However, the water level (W) object is infinite in its extension direction and infinitely small in its normal direction, and its characteristics do not change with the image scale, which leads to the failure of the above evaluation indexes. And considering that the selected target detection algorithm may be limited by its own performance, resulting in low recognition accuracy, high miss rate and other problems, therefore, the present application proposes a new water level elevation solution, which can accurately obtain the current water level under the condition of low target detection algorithm detection accuracy.

[0094] The water level intelligent detection method described in the application outputs the actual water level elevation. In order to verify the feasibility of the proposed scheme, the present application takes the relative accuracy P a As an evaluation index: In the formula, H is the predicted water level elevation of the algorithm; H real is the true water level elevation; the true water level elevation is the label value extracted manually.

[0095] Taking P a as an evaluation index, is the mean value of the aforementioned water level detection accuracy evaluation index P a , wherein the results are shown in the following table. As can be seen from the table, the algorithm model proposed in the present application obtains a score of 98.5% in the P a evaluation index, which is 8.4% higher than the original YOLOv5, and its accuracy can meet the actual engineering application, indicating that this method can well complete the water level detection task. Moreover, the parameter scale of this method is reduced by 4.7% compared with the original algorithm, and the floating point operation number is reduced by 0.6%. This shows that this method algorithm is more lightweight, and is more conducive to edge device transplantation.

[0096]

[0097] Compared with the water level measurement method of manual measurement and installation of water level meter, the specific implementation of the measurement method of the present application only needs to arrange a camera and related accessories at the observation point, does not affect the topography, does not interfere with the water ecology, does not pollute the water source, has lower cost, higher intelligent degree, and is cleaner and more environmentally friendly. In addition, the method does not need to detect all water level line areas, to a certain extent, solves the problem that the water level line prediction in some areas is invalid due to the existence of foreign matter shielding on the water surface, and has stronger robustness.

[0098] The above is only a specific embodiment of the present application, enabling those skilled in the art to understand or implement the present application. Although the foregoing embodiments are described in detail, those skilled in the art should understand that the technical solutions recorded in the foregoing embodiments can be modified, or some or all of the technical features can be replaced by equivalents; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments, and they should be covered in the protection scope of the claims.

Claims

1. A deep learning-based water level intelligent detection method, characterized in that: The method comprises the following steps: Step 1, collecting water level images, collecting various types of water level images by field interval collection to make a data set; Step 2, selecting markers and labeling, wherein various types of labeling have different functions: Class E mark, divided into positive E0, negative E1, according to the configuration E of the height information, obtain the vertical height of each actual height and the rough ratio of pixel height ; Digital markers, according to the scale information of the E scale corresponding to the complete number at the top of the water gauge, the pixel coordinates are connected with the world coordinates; specifically, the digital class contains numbers 0~9, which are respectively denoted as N0~N9; Water level markers, obtain the water level line and the image coordinates of the water level line in the vertical direction, and then convert the actual elevation; specifically, the center of the water level label box is attached to the water level line, and the label boxes are kept continuous, denoted as W; Water gauge markers, judge whether the E class and digital class objects are legal; specifically, the water gauge label box horizontally contains the water gauge and vertically includes the water gauge, denoted as R; And divide the labeled images into training set I and test set II according to the proportion; Step 3, input the training set I with completed labeling into the target detection neural network for training to extract the training model; the performance of the training model is tested by using the test set II; The target detection neural network selects YOLOv5 network structure as the deep learning backbone network, and fuses RankSENet module and Bottleneck Transformer module; Specifically, the YOLOv5 network structure is divided into input layer, benchmark network layer, neck feature fusion network layer and detection head four parts; Among them: the input layer is responsible for image preprocessing, including image adaptive scaling, Mosaic data enhancement and adaptive anchor box calculation operation; The benchmark network layer adopts CSPDarknet53 structure as the benchmark, simultaneously introduces Focus structure to slice the image, then splices in the channel direction, and introduces RankSENet module after the CSP module at the end of the benchmark network layer, and introduces Bottleneck Transformer module after the SPP module, after the data passes through the CSP module at the end of the benchmark network layer, in turn passes through RankSENet module, SPP module and Bottleneck Transformer module, and then is transmitted to the neck feature fusion network layer; The two transmission nodes of mesoscale feature mapping and large-scale feature mapping in the neck feature fusion network layer are merged into a small-scale feature mapping channel, and the small-scale feature mapping is tightly and fully fused; The main part of the detection head is three detectors, each detector uses the prior box set in the grid to detect the target on the feature mapping of different scales, and finally selects the prior box with the highest probability score in the three detectors as the final prediction box; The RankSENet module includes an SENet network structure, which performs compression, activation and feature re-labeling operations on a feature map with C input channels; wherein, when the compression operation is performed, the rank operation is combined with the global average pooling operation of the SENet network. Step 4, input the data set collected in the actual detection task into the target detection neural network trained in step 3, extract the rectangular anchor box information output by the network; thereby obtaining various parameters before the actual water level elevation conversion: take any one non-top E class marker in the water gauge as x1, record the actual height of the x1 center as H top , the x1 longitudinal coordinate as , the number of E class markers between the top E and x1 as a; take the center horizontal coordinate between the adjacent two legal positive E0 and negative E1 targets as , and record the water level line longitudinal coordinate as ; actually measure the actual height h e of each E class marker in the water gauge and the actual scale h top of the E class marker of the top complete number combination of the water gauge. Step 5, input the data set collected in the actual detection task into the target detection neural network trained in step 3, obtain the minimum value R xmin and the maximum value R xmax of the horizontal abscissa of the water gauge R detected by the network, and determine whether other detection objects except the water level W are legal, that is, whether the horizontal abscissa of the detection target exists between (R xmin , R xmax ); if it exists, it is a legal object; otherwise, it is an illegal object, and the illegal object is not calculated; if no R target is detected, the system inherits the last R target coordinate information; Step 6, input the data set collected in the actual detection task into the target detection neural network trained in step 3, obtain the water level line prediction box detected by the network and sample the center coordinates, then perform least squares fitting to obtain the expression formula of the water level line; then detect whether the N-class target exists, if it exists, proceed to the next step, if the N-class target or the W-class target is not detected, the system inherits the last actual elevation value of the water level and the operation is completed; wherein the water line expression formula is wherein, is the slope of the fitted line, is the intercept of the fitted line; least square fitting, the fitting formula is , , , is the center coordinate of the water line prediction frame, is the number of water line prediction frames, , is the mean value of the center coordinates of the water line prediction frame; Step 7, input the data set collected in the actual detection task into the target detection neural network trained in step 3, obtain the information of the E-class mark detected by the network, and obtain the mapping relationship between the actual height and the pixel height by using the actual height of a single E and the pixel height of E, that is, the rough ratio If no E-class target is detected, the system calculates by using the actual height and the pixel height of a single N-class target; Step 8, input the data set collected in the actual detection task into the target detection neural network trained in step 3, obtain the information of N classes of targets detected by the network, and judge the number of N classes of objects, if the number is greater than 1, calculate the ratio of the actual length between the top N class object and the bottom N class object to the pixel length, that is, the accurate ratio ; if the number is only 1, replace the accurate ratio with the rough ratio ; Step 9, get the reference calculation coordinates of the fitting water level line, and the center horizontal coordinate between the adjacent two legal positive E0, negative E1 targets Substitute the expression formula of the water level line , to obtain the current water level line longitudinal coordinate ; Step 10, calculate the number of E class marks between the top of the water gauge E and x1, using the actual scale h of the top E top and the actual height h of the E class marks e , calculate the actual height H of the center of x1 top , the calculation formula is , and then calculate the actual elevation H of the water level; The water level elevation value H is calculated by the formula: wherein, is the corresponding ordinate for x1 ; is the exact ratio, H top is the actual elevation of the center of x1.

2. The water level intelligent detection method based on deep learning according to claim 1, characterized in that: In step 3, during the training process, the super parameter values are as follows: the batch size is set to 32, the initial learning rate is 0.01, the cosine annealing algorithm is used to update the learning rate, the maximum number of iterations is 500; hardware scene: Intel Xeon Gold 5218R CPU, 256G memory, Nvidia Quadro RTX6000 24GB GPU; software scene: 64-bit Windows10, Python3.8+Pytorch1.7.

1. 3.The water level intelligent detection method based on deep learning according to claim 1, characterized in that: In step 3, during the compression operation, the rank operation is combined with the global average pooling operation of the SENet network: the input feature map matrix is compressed into a real number A after the global average pooling operation; at the same time, the rank of the background feature map matrix is calculated, and the rank of the feature map matrix with scale information or water level line is calculated, the difference between the two is obtained to obtain the variable B of the feature map matrix rank, and the product AB is obtained. AB is used to reflect the change degree of the feature map after adding the scale information or the water level line, that is, the real number AB reflects the potential importance of the channel feature, and the output dimension after compression is consistent with the input feature channel C.

4. The water level intelligent detection method based on deep learning according to claim 1, characterized in that: In step 3, the bottleneck transformer module core is the MHSA structure; the MHSA structure is applied to the feature map with an input size of HxWxd, and a 1x1 convolution is respectively used to obtain W q , W k , W v matrix, wherein H, W and d respectively represent the image length, width and channel number, and q, k and v respectively represent the query, key value and weight; R h , R w respectively represent the relative position encoding of the image height and width, and the sum thereof is multiplied by W q to fuse the image content and position information; W q , W k are multiplied to complete the encoding and fusion of the image information; the sum of the above two matrix results is normalized by using the softmax, and the output value is in the matrix format of ; finally, the output value is multiplied by the weight matrix W v to complete the fusion of the global feature and the local feature of the image.

Images