Water level identification method, device, equipment and readable storage medium

Abstract

The present invention provides a water level recognition method, device, equipment and readable storage medium. The water level recognition method includes: obtaining a water level gauge image collected by a camera; preprocessing the water level gauge image to obtain a preprocessed first image; inputting the first image into a corner point information and semantic information extraction network model to obtain corner point information and a semantic feature map; generating a candidate frame based on the corner point information, evaluating the candidate frame using the semantic feature map, and obtaining a prediction frame in the first image and corresponding prediction frame information; cropping the first image based on the prediction frame information to obtain a water level gauge scale image; inputting the water level gauge scale image into a sequence text recognition model to obtain a prediction sequence text; and outputting a water level recognition result based on the prediction sequence text. The present invention is easy to install and reuse in different outdoor scenes, and improves the robustness and accuracy of water level recognition while reducing the cost of water level measurement.

Description

Technical Field

[0001] The present invention relates to the field of computer vision, and in particular to a water level recognition method, device, equipment and readable storage medium. Background Art

[0002] Water level is one of the most important hydrological indicators of rivers and lakes. It plays an important role in water transport and water conservancy, flood control and drought relief, and the prediction of other water level data. It is reported that from 1960 to 2014, floods accounted for 34% of natural disasters worldwide, causing economic losses of more than US$2.5 billion and 1,254 deaths each year. Timely acquisition of water level information helps to grasp the changes in water levels and prevent flood disasters. Water level information monitoring also plays an important role in promoting industrial production. In addition, with the rise of smart agriculture, the study of water level information helps to improve agricultural irrigation efficiency. Therefore, the automatic collection of water level information is of great significance to ensure safety and improve economic benefits.

[0003] At present, the main methods for automatic water level monitoring include float type, pressure type, ultrasonic type and radar type water level gauges. These methods face problems such as low degree of automation, high construction and operation costs, and susceptibility to environmental influences. In order to solve these problems, some recent studies have also applied image processing to water level recognition, proposing a method of preprocessing the input image by grayscale transformation, median filtering, edge detection, and applying clustering algorithms for hierarchical recognition and further processing. Some researchers have proposed a method based on spherical buoys to calculate the relative distance between the camera and the buoy to obtain the water level results. However, due to different construction principles, their actual deployment is restricted by factors such as measurement accuracy and deployment difficulty. Summary of the invention

[0004] The main purpose of the present invention is to provide a water level identification method, device, equipment and readable storage medium, aiming to solve the technical problem that the accuracy of water level identification cannot be effectively met in the existing method of automatically monitoring water level.

[0005] In a first aspect, the present invention provides a water level identification method, the water level identification method comprising the following steps:

[0006] Get the water level gauge image collected by the camera;

[0007] Preprocessing the water gauge image to obtain a preprocessed first image;

[0008] Inputting the first image into a corner point information and semantic information extraction network model to obtain corner point information and a semantic feature map;

[0009] Generate a candidate box based on the corner point information, and evaluate the candidate box using the semantic feature map to obtain a prediction box in the first image and corresponding prediction box information;

[0010] Cropping the first image based on the prediction frame information to obtain a water level gauge scale image;

[0011] Inputting the water gauge scale image into a sequence text recognition model to obtain a predicted sequence text;

[0012] Based on the predicted sequence text, a water level recognition result is output.

[0013] Optionally, the step of preprocessing the water gauge image to obtain a preprocessed first image includes:

[0014] Detecting whether the percentage of the number of black pixels in the water gauge image is greater than a preset pigment ratio threshold;

[0015] When it is detected that the percentage of the number of black pixels in the water gauge image is greater than a preset pigment ratio threshold, the water gauge image is proportionally mapped to the intensity channel through a multi-scale weighted average method to obtain a channel-enhanced image;

[0016] Performing a convolution operation on the channel enhanced image using Gaussian filtering as a center surround function to obtain an illuminated component;

[0017] Based on the illuminated component and the channel enhanced image, obtaining a logarithmic function of the reflected component;

[0018] The logarithmic function of the reflection component is linearly quantized to obtain a first image.

[0019] Optionally, the step of inputting the first image into a corner point information and semantic information extraction network model to obtain corner point information and a semantic feature map includes:

[0020] Input the first image into the corner point information extraction branch of the corner point information and semantic information extraction network model to obtain a plurality of corner points at the upper left, upper right, lower left, and lower right of the water level gauge part in the first image and corresponding corner point information, wherein the corner point information includes pixel coordinates of the corner point, confidence scores, and short side length information corresponding to the corner point;

[0021] The first image is input into the semantic information extraction branch of the corner point information and semantic information extraction network model, and binary classification semantic segmentation is performed to obtain a semantic feature map of the water level gauge text part in the first image.

[0022] Optionally, the step of generating a candidate box based on the corner point information corresponding to the plurality of corner points, evaluating the candidate box using the semantic feature map, and obtaining a predicted box in the first image and corresponding predicted box information includes:

[0023] Retain corner points whose confidence scores are greater than a preset score threshold;

[0024] Combine the upper and lower corner points on the same side in pairs to obtain several corner point combinations, calculate the average of the short side lengths of the two corner points included in each corner point combination, and obtain the candidate box corresponding to each corner point combination based on the average of the short side lengths and the long side lengths corresponding to each corner point combination, where the long side length corresponding to each corner point combination is the distance between the two corner points included in each corner point combination;

[0025] The candidate boxes whose area intersection-over-union ratio with the semantic feature map is greater than a preset intersection-over-union ratio threshold are retained, and non-maximum suppression is performed on the retained candidate boxes to obtain a prediction box corresponding to the first image and corresponding prediction box information, wherein the prediction box information includes coordinate information of the prediction box and a border length.

[0026] Optionally, the step of inputting the water gauge scale image into a sequence text recognition model to obtain a predicted sequence text comprises:

[0027] Inputting the water gauge scale image into a sequence text recognition model, wherein the sequence text recognition model includes a convolution layer, a circulation layer, and a transcription layer;

[0028] Perform feature extraction through the convolution layer to obtain a plurality of frame feature sequences corresponding to the water level gauge scale image;

[0029] Calculating the context information of the plurality of frame feature sequences through the recurrent layer to obtain probability prediction results of the plurality of frame feature sequences;

[0030] The probability prediction results of the plurality of frame feature sequences are greedily searched through the transcription layer to obtain the prediction sequence text of the water level gauge scale.

[0031] Optionally, the step of outputting a water level recognition result based on the predicted sequence text includes:

[0032] Determining whether the predicted sequence text is arranged according to a preset character arrangement rule of the water level gauge;

[0033] If the predicted sequence text is not arranged according to the preset character arrangement rule of the water level gauge, the output water level recognition result is empty;

[0034] If the predicted sequence text is arranged according to the preset character arrangement rule of the water level gauge, the numeric characters, water level gauge characters and Chinese characters in the predicted result of the sequence text are traversed to calculate the water level scale value, and the output water level recognition result is the water level scale value.

[0035] In a second aspect, the present invention further provides a water level identification device, the water level identification device comprising:

[0036] An acquisition module is used to acquire the water level gauge image collected by the camera;

[0037] A preprocessing module, used for preprocessing the water gauge image to obtain a preprocessed first image;

[0038] An extraction module, used for inputting the first image into a corner point information and semantic information extraction network model to obtain corner point information and a semantic feature map;

[0039] A prediction box generation module, used to generate a candidate box based on the corner point information, evaluate the candidate box using the semantic feature map, and obtain a prediction box in the first image and corresponding prediction box information;

[0040] A cropping module, used for cropping the first image based on the prediction frame information to obtain a water level gauge scale image;

[0041] A sequence text recognition module, used for inputting the water gauge scale image into a sequence text recognition model to obtain a predicted sequence text;

[0042] The output module is used to output the water level recognition result based on the prediction sequence text.

[0043] Optionally, the preprocessing module is further specifically used for:

[0044] Detecting whether the percentage of the number of black pixels in the water gauge image is greater than a preset pigment ratio threshold;

[0045] When it is detected that the percentage of the number of black pixels in the water gauge image is greater than a preset pigment ratio threshold, the water gauge image is proportionally mapped to the intensity channel through a multi-scale weighted average method to obtain a channel-enhanced image;

[0046] Performing a convolution operation on the channel enhanced image using Gaussian filtering as a center surround function to obtain an illuminated component;

[0047] Based on the illuminated component and the channel enhanced image, obtaining a logarithmic function of the reflected component;

[0048] The logarithmic function of the reflection component is linearly quantized to obtain a first image.

[0049] Optionally, the extraction module is further specifically used for:

[0050] Input the first image into the corner point information extraction branch of the corner point information and semantic information extraction network model to obtain a plurality of corner points at the upper left, upper right, lower left, and lower right of the water level gauge part in the first image and corresponding corner point information, wherein the corner point information includes pixel coordinates of the corner point, confidence scores, and short side length information corresponding to the corner point;

[0051] The first image is input into the semantic information extraction branch of the corner point information and semantic information extraction network model, and binary classification semantic segmentation is performed to obtain a semantic feature map of the water level gauge text part in the first image.

[0052] Optionally, the prediction box generation module is further specifically used for:

[0053] Retain corner points whose confidence scores are greater than a preset score threshold;

[0054] Combine the upper and lower corner points on the same side in pairs to obtain several corner point combinations, calculate the average of the short side lengths of the two corner points included in each corner point combination, and obtain the candidate box corresponding to each corner point combination based on the average of the short side lengths and the long side lengths corresponding to each corner point combination, where the long side length corresponding to each corner point combination is the distance between the two corner points included in each corner point combination;

[0055] The candidate boxes whose area intersection-over-union ratio with the semantic feature map is greater than a preset intersection-over-union ratio threshold are retained, and non-maximum suppression is performed on the retained candidate boxes to obtain a prediction box corresponding to the first image and corresponding prediction box information, wherein the prediction box information includes coordinate information of the prediction box and a border length.

[0056] Optionally, the sequence text recognition module is further specifically used for:

[0057] Inputting the water gauge scale image into a sequence text recognition model, wherein the sequence text recognition model includes a convolution layer, a circulation layer, and a transcription layer;

[0058] Perform feature extraction through the convolution layer to obtain a plurality of frame feature sequences corresponding to the water level gauge scale image;

[0059] Calculating the context information of the plurality of frame feature sequences through the recurrent layer to obtain probability prediction results of the plurality of frame feature sequences;

[0060] The probability prediction results of the plurality of frame feature sequences are greedily searched through the transcription layer to obtain the prediction sequence text of the water level gauge scale.

[0061] Optionally, the output module is further specifically used for:

[0062] Determining whether the predicted sequence text is arranged according to a preset character arrangement rule of the water level gauge;

[0063] If the predicted sequence text is not arranged according to the preset character arrangement rule of the water level gauge, the output water level recognition result is empty;

[0064] If the predicted sequence text is arranged according to the preset character arrangement rule of the water level gauge, the numeric characters, water level gauge characters and Chinese characters in the predicted result of the sequence text are traversed to calculate the water level scale value, and the output water level recognition result is the water level scale value.

[0065] In a third aspect, the present invention also provides a water level identification device, which includes a processor, a memory, and a water level identification program stored in the memory and executable by the processor, wherein when the water level identification program is executed by the processor, the steps of the water level identification method described above are implemented.

[0066] In a fourth aspect, the present invention further provides a readable storage medium, on which a water level identification program is stored, wherein when the water level identification program is executed by a processor, the steps of the water level identification method as described above are implemented.

[0067] The present invention provides a water level recognition method, device, equipment and readable storage medium. The water level recognition method includes: obtaining a water level gauge image collected by a camera; preprocessing the water level gauge image to obtain a preprocessed first image; inputting the first image into a corner point information and semantic information extraction network model to obtain corner point information and a semantic feature map; generating a candidate frame based on the corner point information, evaluating the candidate frame using the semantic feature map, and obtaining a prediction frame in the first image and corresponding prediction frame information; cropping the first image based on the prediction frame information to obtain a water level gauge scale image; inputting the water level gauge scale image into a sequence text recognition model to obtain a prediction sequence text; and outputting a water level recognition result based on the prediction sequence text. The present invention is easy to install and reuse in different outdoor scenes, and improves the robustness and accuracy of water level recognition while reducing the cost of water level measurement. BRIEF DESCRIPTION OF THE DRAWINGS

[0068] Figure 1 A schematic diagram of the hardware structure of a water level identification device involved in an embodiment of the present invention;

[0069] Figure 2 It is a flow chart of an embodiment of a water level identification method of the present invention;

[0070] Figure 3 Schematic diagram of functional modules of an embodiment of a water level identification device of the present invention.

[0071] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0072] It should be understood that the specific embodiments described herein are only used to explain the present invention, and are not used to limit the present invention.

[0073] In a first aspect, an embodiment of the present invention provides a water level identification device.

[0074] Reference Figure 1 , Figure 1 Schematic diagram of the hardware structure of the water level identification device involved in the embodiment of the present invention. In the embodiment of the present invention, the water level identification device may include a processor 1001 (such as a central processing unit, CPU), a communication bus 1002, a user interface 1003, a network interface 1004, and a memory 1005. Among them, the communication bus 1002 is used to realize the connection and communication between these components; the user interface 1003 may include a display screen (Display), an input unit such as a keyboard (Keyboard); the network interface 1004 may optionally include a standard wired interface, a wireless interface (such as wireless fidelity WIreless-FIdelity, WI-FI interface); the memory 1005 may be a high-speed random access memory (random access memory, RAM), or a stable memory (non-volatile memory), such as a disk storage, and the memory 1005 may optionally be a storage device independent of the aforementioned processor 1001. Those skilled in the art will understand that Figure 1 The hardware structure shown in the figure does not constitute a limitation of the present invention, and may include more or less components than shown in the figure, or combine certain components, or arrange the components differently.

[0075] Continue to refer to Figure 1 , Figure 1 The memory 1005 as a computer storage medium may include an operating system, a network communication module, a user interface module, and a water level recognition program. The processor 1001 may call the water level recognition program stored in the memory 1005 and execute the water level recognition method provided in the embodiment of the present invention.

[0076] In a second aspect, an embodiment of the present invention provides a water level identification method.

[0077] Reference Figure 2 , Figure 2 It is a flow chart of an embodiment of a water level identification method of the present invention.

[0078] In an embodiment of the water level identification method of the present invention, the water level identification method includes:

[0079] Step S10, obtaining a water level gauge image captured by a camera;

[0080] In this embodiment, the camera is deployed on the shore within the visible range of the water level gauge of the river or lake, and the water level gauge image is collected by the camera, and the water level recognition device obtains the water level gauge image collected by the camera. Among them, there is a distance between the location where the camera is deployed and the water level gauge, so the water level gauge image obtained includes the water level gauge and the background part where the water level gauge is located. Among them, the use of visual water level measurement methods can greatly reduce costs. It only needs to install the camera and transmit the image collected by the camera back to the edge image processing server, avoiding the cost of tens of thousands of radar and other methods.

[0081] Step S20, preprocessing the water gauge image to obtain a preprocessed first image;

[0082] In this embodiment, when the outdoor camera acquires the water level gauge image, it is affected by weather, light, etc., which is not conducive to the subsequent water level gauge recognition based on the acquired water level gauge image. Therefore, after acquiring the water level gauge image obtained by the camera, it is necessary to preprocess the acquired water level gauge image to obtain the preprocessed first image.

[0083] Furthermore, in one embodiment, step S20 includes:

[0084] Detecting whether the percentage of the number of black pixels in the water gauge image is greater than a preset pigment ratio threshold;

[0085] When it is detected that the percentage of the number of black pixels in the water gauge image is greater than a preset pigment ratio threshold, the water gauge image is proportionally mapped to the intensity channel through a multi-scale weighted average method to obtain a channel-enhanced image;

[0086] Performing a convolution operation on the channel enhanced image using Gaussian filtering as a center surround function to obtain an illuminated component;

[0087] Based on the illuminated component and the channel enhanced image, obtaining a logarithmic function of the reflected component;

[0088] The logarithmic function of the reflection component is linearly quantized to obtain a first image.

[0089] In this embodiment, after obtaining the water level gauge image captured by the outdoor camera, it is necessary to determine whether the captured image needs to be processed for illumination enhancement. Generally, the low illumination intensity of the image captured in the outdoor environment at night will be detrimental to the subsequent water level recognition. Therefore, after determining that the captured image needs to be processed for illumination enhancement, the input water level gauge image can be preprocessed based on the traditional image processing algorithm Retinex theory. After the preprocessing, a balance can be achieved in the three aspects of dynamic range compression, edge enhancement, and color constancy, so as to achieve adaptive illumination enhancement of the water level gauge image captured in the dark.

[0090] Specifically, after obtaining the water gauge image captured by the outdoor camera, it is detected whether the percentage of the number of black pixels in the water gauge image is greater than the preset pigment ratio threshold value to determine whether it is necessary to perform light enhancement processing on the captured water gauge image. When it is detected that the percentage of the number of black pixels in the water gauge image is greater than the preset pigment ratio threshold value, it is determined that it is necessary to perform light enhancement processing on the captured water gauge image at this time. The water gauge image is proportionally mapped to the intensity channel through a multi-scale weighted averaging method to obtain a channel-enhanced image while retaining the color distribution of the original image.

[0091] The standard deviation σ, pixel mean μ and pixel mean square error φ of the channel-enhanced image I in the attention window ω are calculated, and the Gaussian filter is used as the center surround function to perform a convolution operation on the channel-enhanced image I to obtain the illumination component L. The specific formula of the convolution operation is: L=F*I, where F represents the Gaussian filter in the convolution operation, and the specific calculation formula is: σ is the spatial scale constant around Gaussian, r 2 =x 2 +y 2 Based on the illumination component L and the channel enhanced image I, the logarithmic function of the reflection component R is obtained, and the specific formula is: Log[R(x,y)]=Log[I(x,y)]-Log[L(x,y)], where (x, y) is the corresponding pixel coordinate position in the two-dimensional image.

[0092] The logarithmic function Log[R(x,y)] of the reflection component R is linearly quantized to a pixel value between [0,255] as the final output to obtain a first image, wherein the specific quantization formula is: Among them, the collected water level gauge images are pre-processed for night image enhancement. Even in a dark environment that the human eye cannot recognize, the collected water level gauge images can be enhanced to a range that the human eye can recognize. This is especially suitable for situations where outdoor lighting is not strong and power supply capacity is insufficient. It has low requirements for shooting conditions, does not emphasize sufficient light, and has strong robustness.

[0093] Step S30, inputting the first image into a corner point information and semantic information extraction network model to obtain corner point information and a semantic feature map;

[0094] In this embodiment, the preprocessed first image is input into a trained corner point information and semantic information extraction network model, so as to obtain the corner point information and semantic feature map corresponding to the first image.

[0095] Among them, the corner point information and semantic information extraction network model used in this embodiment can be designed and implemented based on the improved feature extraction module and U-net upsampling module of VGG16. Specifically, the two fully connected layers of VGG16 are converted into two convolutional layers, and four additional convolutional layers are added to increase the receptive field of feature extraction. Some deconvolution modules are connected after the feature extraction module. These deconvolution layers will generate cascade feature maps to better predict corner points. The input preprocessed first image is deconvoluted by the improved VGG16 feature extraction network and U-net structure to generate a feature map with rich information. Two operations are performed on the feature map, one is to input it to the corner branch to obtain the pixel coordinates and confidence score information of the corner point, and the second is to input it to the semantic branch to obtain a semantic feature map.

[0096] In the training phase of the corner information and semantic information extraction network model, corner detection and semantic segmentation are performed simultaneously. The goals of model training optimization include corner information and semantic feature maps. The pre-collected training data images and annotations are sent to the model for training, and the semantic feature maps obtained by training and optimizing are minimized by Dice Loss. The DiceLoss loss function is defined as: The CrossEntropy loss function is used to train and optimize the corner confidence score in the obtained corner information, where the CrossEntropy loss function is defined as: L conf = CrossEntropy(y c ,p c ), the SmoothL1 loss function is used to train and optimize the pixel coordinates of the corner points in the obtained corner point information, where the SmoothL1 loss function is defined as: L loc =SmoothL1(y l ,p l ). When the predicted value predicted by the corner information and semantic information extraction network model and the pre-annotated label value obtain the value of the above loss function, the joint loss is calculated together and then back-propagated to the next corner information and semantic information extraction network model to be trained until the joint loss value converges. The training is terminated and the latest corner information and semantic information extraction network model is used as the trained corner information and semantic information extraction network model.

[0097] Furthermore, in one embodiment, step S30 includes:

[0098] Input the first image into the corner point information extraction branch of the corner point information and semantic information extraction network model to obtain a plurality of corner points at the upper left, upper right, lower left, and lower right of the water level gauge part in the first image and corresponding corner point information, wherein the corner point information includes pixel coordinates of the corner point, confidence scores, and short side length information corresponding to the corner point;

[0099] The first image is input into the semantic information extraction branch of the corner point information and semantic information extraction network model, and binary classification semantic segmentation is performed to obtain a semantic feature map of the water level gauge text part in the first image.

[0100] In this embodiment, after the corner point information and semantic information extraction network model is trained, after obtaining the preprocessed unlabeled first image, on the one hand, the first image is input into the corner point information extraction branch of the corner point information and semantic information extraction network model to obtain several corner points and corresponding corner point information of the upper left, upper right, lower left, and lower right of the water level gauge part in the first image, wherein the corner point information includes the pixel coordinates of the corner points, the confidence score, and the short side length information corresponding to the corner points. On the other hand, the first image is input into the semantic information extraction branch of the corner point information and semantic information extraction network model to perform binary classification semantic segmentation to obtain the semantic feature map of the water level gauge text part in the first image.

[0101] Step S40, generating a candidate box based on the corner point information, and evaluating the candidate box using the semantic feature map to obtain a predicted box in the first image and corresponding predicted box information;

[0102] In this embodiment, based on the corner point information and semantic feature map obtained from the preprocessed unlabeled first image, a prediction box of the water gauge portion in the first image is generated, and prediction box information corresponding to the prediction box is obtained. Specifically, a candidate box is first generated based on the corner point information, and then the semantic features are used to evaluate the candidate box generated based on the corner point information, and the prediction box corresponding to the water gauge scale portion in the first image is screened and obtained, and the prediction box information corresponding to the prediction box can be obtained. Among them, the method of corner point detection can adapt to the situation where the water gauge image is shot at any angle, and the detected prediction box has a high area intersection and union ratio with the water gauge portion in the first image, which is suitable for subsequent water gauge sequence text recognition.

[0103] Furthermore, in one embodiment, step S40 includes:

[0104] Retain corner points whose confidence scores are greater than a preset score threshold;

[0105] Combine the upper and lower corner points on the same side in pairs to obtain several corner point combinations, calculate the average of the short side lengths of the two corner points included in each corner point combination, and obtain the candidate box corresponding to each corner point combination based on the average of the short side lengths and the long side lengths corresponding to each corner point combination, where the long side length corresponding to each corner point combination is the distance between the two corner points included in each corner point combination;

[0106] The candidate boxes whose area intersection-over-union ratio with the semantic feature map is greater than a preset intersection-over-union ratio threshold are retained, and non-maximum suppression is performed on the retained candidate boxes to obtain a prediction box corresponding to the first image and corresponding prediction box information, wherein the prediction box information includes coordinate information of the prediction box and a border length.

[0107] In this embodiment, specifically, the steps of generating a candidate box based on corner point information, evaluating the candidate box using the semantic feature map, and obtaining a predicted box in the first image and corresponding predicted box information include:

[0108] First, some corner points are removed through the obtained corner point information, and only the corner points with confidence scores greater than the preset score threshold are retained. Secondly, based on the retained corner points, the upper and lower corner points on the same side are combined in pairs to obtain several corner point combinations, where the combination matching of the corner points adopts the k-nearest method for matching rather than brute force matching. Compared with brute force matching, it can effectively reduce time overhead without causing too much precision loss. Then the mean of the short side lengths corresponding to the two corner points contained in each corner point combination is calculated, and the candidate box corresponding to each corner point combination is obtained based on the mean of the short side lengths corresponding to each corner point combination and the long side length, where the long side length corresponding to each corner point combination is the distance between the two corner points contained in each corner point combination, and the candidate box whose ratio of the short side lengths corresponding to the two corner points exceeds the preset ratio is directly removed.

[0109] Then the candidate box is divided into four parts: upper left, upper right, lower left, and lower right; the semantic feature map is divided into four categories: upper left, upper right, lower left, and lower right. The intersection over union (IoU) of the areas of the four semantic feature maps and the candidate box is calculated using the following formula: Where S i is the area of ​​the intersection of the regions, S u is the area of ​​the region. The final average of these four parts is taken to obtain the final area intersection-union ratio. If the area intersection-union ratio with the semantic feature map is greater than the preset intersection-union ratio threshold, the candidate box whose area intersection-union ratio with the semantic feature map is greater than the preset intersection-union ratio threshold is retained. The retained candidate boxes are subjected to non-maximum suppression, and the smaller candidate boxes whose area intersection-union ratio exceeds the preset intersection-union ratio threshold are removed. The candidate boxes are deduplicated to obtain the prediction box corresponding to the first image and the corresponding prediction box information, wherein the prediction box information includes the coordinate information of the prediction box and the border length.

[0110] Step S50, cropping the first image based on the prediction frame information to obtain a water level gauge scale image;

[0111] In this embodiment, the first image is cropped based on the final prediction frame screened in step S40. Specifically, the coordinate information and the frame length of the prediction frame are obtained, a prediction frame is formed at the corresponding coordinate position on the first image, and the position inside the prediction frame on the first image is cropped, wherein the image inside the prediction frame is the obtained water level gauge scale image.

[0112] Step S60, inputting the water gauge scale image into a sequence text recognition model to obtain a predicted sequence text;

[0113] In this embodiment, the water gauge scale image is input into a trained sequence text recognition model to obtain a predicted sequence text, and the sequence text corresponds to the water gauge scale in the water gauge scale image.

[0114] Furthermore, in one embodiment, step S60 includes:

[0115] Inputting the water gauge scale image into a sequence text recognition model, wherein the sequence text recognition model includes a convolution layer, a circulation layer, and a transcription layer;

[0116] Perform feature extraction through the convolution layer to obtain a plurality of frame feature sequences corresponding to the water level gauge scale image;

[0117] Calculating the context information of the plurality of frame feature sequences through the recurrent layer to obtain probability prediction results of the plurality of frame feature sequences;

[0118] The probability prediction results of the plurality of frame feature sequences are greedily searched through the transcription layer to obtain the prediction sequence text of the water level gauge scale.

[0119] In this embodiment, specifically, the water level gauge scale image is input into a sequence text recognition model, and the step of obtaining a predicted sequence text includes: inputting the water level gauge scale image into a sequence text recognition model, and the sequence text recognition model includes a convolutional layer, a recurrent layer, and a transcription layer.

[0120] Taking the following model structure as an example, the convolution layer of the sequence text recognition model uses Resnet-50 as the image sequence feature extraction network; the recurrent layer uses a bidirectional two-layer LSTM network; the transcription layer, because the input image label is fixed, uses dynamic programming to optimize the CTC loss to train and optimize the network parameters of the convolution layer and the recurrent layer during the training phase. Among them, Resnet-50 uses a cross-layer connection structure, which can effectively deepen the depth of the Resnet-50 network sequence feature extraction and improve the robustness of the Resnet-50 network sequence feature extraction. The bidirectional two-layer LSTM network can effectively utilize context information and output the predicted probability measurement results of the frame sequence features extracted by the convolution layer according to the context semantics.

[0121] After the training optimization is performed according to CTC loss to obtain the trained sequence text recognition model, the water gauge scale image is input into the trained sequence text recognition model. First, feature extraction is performed through the convolution layer to obtain several frame feature sequences corresponding to the water gauge scale image. Then, the context information of the several frame feature sequences is calculated through the circulation layer to obtain the probability prediction results of the several frame feature sequences. Finally, the probability prediction results of the several frame feature sequences are greedily searched through the transcription layer to obtain the predicted sequence text of the water gauge scale.

[0122] Step S70: outputting a water level recognition result based on the predicted sequence text.

[0123] In this embodiment, the character arrangement of the predicted sequence text obtained based on the sequence text recognition model is identified, and the corresponding water level recognition result is output. Among them, because the predicted box detected by the corner point detection method has a high area intersection and union ratio with the water level gauge part in the first image, it is suitable for subsequent water level gauge sequence text recognition. Therefore, the final water level recognition accuracy is high, and the probability of the error not exceeding 3 mm is 95.37%, which fully meets the requirement of the Chinese water level error standard not exceeding 1 cm.

[0124] Furthermore, in one embodiment, step S70 includes:

[0125] Determining whether the predicted sequence text is arranged according to a preset character arrangement rule of the water level gauge;

[0126] If the predicted sequence text is not arranged according to the preset character arrangement rule of the water level gauge, the output water level recognition result is empty;

[0127] If the predicted sequence text is arranged according to the preset character arrangement rule of the water level gauge, the numeric characters, water level gauge characters and Chinese characters in the predicted result of the sequence text are traversed to calculate the water level scale value, and the output water level recognition result is the water level scale value.

[0128] In this embodiment, after obtaining the predicted sequence text based on the sequence text recognition model, it is determined whether the predicted sequence text is arranged according to the preset character arrangement rule of the water level gauge. That is, it is determined whether the numeric characters or the character E in the predicted sequence text are misaligned or not placed in order. The preset character arrangement rule is as follows: the numeric characters and the character E must be arranged at intervals, the numeric characters are arranged one by one in ascending order from left to right, the Chinese characters are at the end, and they are the Chinese characters in the preset table. If the predicted sequence text is not arranged according to the preset character arrangement rule of the water level gauge, such as 9E7E, 9EE8, which are not arranged according to the preset character arrangement rule, then the water level gauge recognition result is output as empty at this time.

[0129] If the predicted sequence text is arranged according to the preset character arrangement rule of the water level gauge, such as AE9E8 Lei, where A represents the hexadecimal numeric character 10, Lei represents the Chinese character corresponding to the remaining part of the water level gauge scale that is less than the water level gauge character E, and E is the water level gauge character. Then traverse the numeric characters, water level gauge characters, and Chinese characters in the predicted sequence text, calculate the water level scale value, and output the water level recognition result as the water level scale value. The specific calculation method is as follows: First, obtain the reference value through the last numeric character, and the reference value is equal to the last digit × 10. If there is an E character after this numeric character, subtract 5 cm from the reference value. Finally, subtract the mapping value of the remaining part corresponding to the Chinese character to obtain the final recognition result. Taking the obtained predicted sequence text AE9E8 Lei as an example, the character "8" is the last numeric character, so the reference value is 8 * 10 = 80 cm. There is no character "E" after 8, so 5.0 cm does not need to be subtracted. The Chinese character "Lei" corresponding to the remaining part is mapped to 4.6 cm by looking up the table. Therefore, the obtained water level scale value is 80.0 - 4.6 = 75.4 cm, and the finally output water level recognition result is 75.4 cm.

[0130] In this embodiment, the water level recognition method includes: acquiring the water level gauge image collected by the camera; preprocessing the water level gauge image to obtain the preprocessed first image; inputting the first image into the corner point information and semantic information extraction network model to obtain the corner point information and the semantic feature map; generating candidate boxes based on the corner point information, and using the semantic feature map to evaluate the candidate boxes to obtain the predicted boxes and the corresponding predicted box information in the first image; cropping the first image based on the predicted box information to obtain the water level gauge scale image; inputting the water level gauge scale image into the sequence text recognition model to obtain the predicted sequence text; and outputting the water level recognition result based on the predicted sequence text. The present invention is easy to install and reuse in different outdoor scenarios, reducing the water level measurement cost while improving the robustness and accuracy of water level recognition.

[0131] In a third aspect, an embodiment of the present invention further provides a water level identification device.

[0132] Reference Figure 3 , a schematic diagram of functional modules of an embodiment of a water level identification device.

[0133] In this embodiment, the water level identification device includes:

[0134] An acquisition module 10 is used to acquire a water level gauge image captured by a camera;

[0135] A preprocessing module 20, used for preprocessing the water gauge image to obtain a preprocessed first image;

[0136] An extraction module 30, used for inputting the first image into a corner point information and semantic information extraction network model to obtain corner point information and a semantic feature map;

[0137] A prediction box generation module 40 is used to generate a candidate box based on the corner point information, evaluate the candidate box using the semantic feature map, and obtain a prediction box in the first image and corresponding prediction box information;

[0138] A cropping module 50, configured to crop the first image based on the prediction frame information to obtain a water level gauge scale image;

[0139] A sequence text recognition module 60 is used to input the water gauge scale image into a sequence text recognition model to obtain a predicted sequence text;

[0140] The output module 70 is used to output the water level recognition result based on the prediction sequence text.

[0141] Furthermore, in one embodiment, the preprocessing module 20 is further specifically used for:

[0142] Detecting whether the percentage of the number of black pixels in the water gauge image is greater than a preset pigment ratio threshold;

[0143] When it is detected that the percentage of the number of black pixels in the water gauge image is greater than a preset pigment ratio threshold, the water gauge image is proportionally mapped to the intensity channel through a multi-scale weighted average method to obtain a channel-enhanced image;

[0144] Performing a convolution operation on the channel enhanced image using Gaussian filtering as a center surround function to obtain an illuminated component;

[0145] Based on the illuminated component and the channel enhanced image, obtaining a logarithmic function of the reflected component;

[0146] The logarithmic function of the reflection component is linearly quantized to obtain a first image.

[0147] Furthermore, in one embodiment, the extraction module 30 is further specifically configured to:

[0148] Input the first image into the corner point information extraction branch of the corner point information and semantic information extraction network model to obtain a plurality of corner points at the upper left, upper right, lower left, and lower right of the water level gauge part in the first image and corresponding corner point information, wherein the corner point information includes pixel coordinates of the corner point, confidence scores, and short side length information corresponding to the corner point;

[0149] The first image is input into the semantic information extraction branch of the corner point information and semantic information extraction network model, and binary classification semantic segmentation is performed to obtain a semantic feature map of the water level gauge text part in the first image.

[0150] Furthermore, in one embodiment, the prediction frame generation module 40 is further specifically used for:

[0151] Retain corner points whose confidence scores are greater than a preset score threshold;

[0152] Combine the upper and lower corner points on the same side in pairs to obtain several corner point combinations, calculate the average of the short side lengths of the two corner points included in each corner point combination, and obtain the candidate box corresponding to each corner point combination based on the average of the short side lengths and the long side lengths corresponding to each corner point combination, where the long side length corresponding to each corner point combination is the distance between the two corner points included in each corner point combination;

[0153] The candidate boxes whose area intersection-over-union ratio with the semantic feature map is greater than a preset intersection-over-union ratio threshold are retained, and non-maximum suppression is performed on the retained candidate boxes to obtain a prediction box corresponding to the first image and corresponding prediction box information, wherein the prediction box information includes coordinate information of the prediction box and a border length.

[0154] Furthermore, in one embodiment, the sequence text recognition module 60 is further specifically used for:

[0155] Inputting the water level gauge scale image into a sequence text recognition model, wherein the sequence text recognition model includes a convolution layer, a recurrent layer, and a transcription layer;

[0156] Performing feature extraction through the convolution layer to obtain a plurality of frame feature sequences corresponding to the water level gauge scale image;

[0157] Calculating the context information of the plurality of frame feature sequences through the recurrent layer to obtain probability prediction results of the plurality of frame feature sequences;

[0158] The probability prediction results of the plurality of frame feature sequences are greedily searched through the transcription layer to obtain the prediction sequence text of the water level gauge scale.

[0159] Furthermore, in one embodiment, the output module 70 is further specifically used for:

[0160] Determining whether the predicted sequence text is arranged according to a preset character arrangement rule of the water level gauge;

[0161] If the predicted sequence text is not arranged according to the preset character arrangement rule of the water level gauge, the output water level recognition result is empty;

[0162] If the predicted sequence text is arranged according to the preset character arrangement rule of the water level gauge, the numeric characters, water level gauge characters and Chinese characters in the predicted result of the sequence text are traversed to calculate the water level scale value, and the output water level recognition result is the water level scale value.

[0163] Among them, the functional implementation of each module in the above-mentioned water level identification device corresponds to the various steps in the above-mentioned water level identification method embodiment, and its functions and implementation processes will not be repeated here one by one.

[0164] In a fourth aspect, an embodiment of the present invention further provides a readable storage medium.

[0165] The readable storage medium of the present invention stores a water level recognition program, wherein when the water level recognition program is executed by a processor, the steps of the water level recognition method described above are implemented.

[0166] Among them, the method implemented when the water level identification program is executed can refer to the various embodiments of the water level identification method of the present invention, and will not be repeated here.

[0167] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or system including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or system. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or system including the element.

[0168] The serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.

[0169] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) as described above, and includes a number of instructions for a terminal device to execute the methods described in each embodiment of the present invention.

[0170] The above are only preferred embodiments of the present invention, and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A water level identification method, characterized in that: The water level identification method comprises: Get the water level gauge image collected by the camera; Preprocessing the water gauge image to obtain a preprocessed first image; Inputting the first image into a corner point information and semantic information extraction network model to obtain corner point information and a semantic feature map; Generate a candidate box based on the corner point information, and evaluate the candidate box using the semantic feature map to obtain a prediction box in the first image and corresponding prediction box information; Cropping the first image based on the prediction frame information to obtain a water level gauge scale image; Inputting the water gauge scale image into a sequence text recognition model to obtain a predicted sequence text; Based on the predicted sequence text, output a water level recognition result; The step of inputting the first image into a corner point information and semantic information extraction network model to obtain corner point information and a semantic feature map comprises: Input the first image into the corner point information extraction branch of the corner point information and semantic information extraction network model to obtain a plurality of corner points at the upper left, upper right, lower left, and lower right of the water level gauge part in the first image and corresponding corner point information, wherein the corner point information includes pixel coordinates of the corner point, confidence scores, and short side length information corresponding to the corner point; Inputting the first image into the semantic information extraction branch of the corner point information and semantic information extraction network model to perform binary classification semantic segmentation to obtain a semantic feature map of the water level gauge text part in the first image; The step of generating a candidate frame based on the corner point information corresponding to the plurality of corner points, evaluating the candidate frame using the semantic feature map, and obtaining a predicted frame in the first image and corresponding predicted frame information comprises: Retain corner points whose confidence scores are greater than a preset score threshold; Combine the upper and lower corner points on the same side in pairs to obtain several corner point combinations, calculate the average of the short side lengths of the two corner points included in each corner point combination, and obtain the candidate box corresponding to each corner point combination based on the average of the short side lengths and the long side lengths corresponding to each corner point combination, where the long side length corresponding to each corner point combination is the distance between the two corner points included in each corner point combination; The candidate boxes whose area intersection-over-union ratio with the semantic feature map is greater than a preset intersection-over-union ratio threshold are retained, and non-maximum suppression is performed on the retained candidate boxes to obtain a prediction box corresponding to the first image and corresponding prediction box information, wherein the prediction box information includes coordinate information of the prediction box and a border length.

2. The water level identification method according to claim 1, characterized in that: The step of preprocessing the water gauge image to obtain a preprocessed first image comprises: Detecting whether the percentage of the number of black pixels in the water gauge image is greater than a preset pigment ratio threshold; When it is detected that the percentage of the number of black pixels in the water gauge image is greater than a preset pigment ratio threshold, the water gauge image is proportionally mapped to the intensity channel through a multi-scale weighted average method to obtain a channel-enhanced image; Performing a convolution operation on the channel enhanced image using Gaussian filtering as a center surround function to obtain an illuminated component; Based on the illuminated component and the channel enhanced image, obtaining a logarithmic function of the reflected component; The logarithmic function of the reflection component is linearly quantized to obtain a first image.

3. The water level identification method according to claim 1, characterized in that: The step of inputting the water gauge scale image into the sequence text recognition model to obtain the predicted sequence text comprises: Inputting the water level gauge scale image into a sequence text recognition model, wherein the sequence text recognition model includes a convolution layer, a recurrent layer, and a transcription layer; Performing feature extraction through the convolution layer to obtain a plurality of frame feature sequences corresponding to the water level gauge scale image; Calculating the context information of the plurality of frame feature sequences through the recurrent layer to obtain probability prediction results of the plurality of frame feature sequences; The probability prediction results of the plurality of frame feature sequences are greedily searched through the transcription layer to obtain the prediction sequence text of the water level gauge scale.

4. The water level identification method according to claim 1, characterized in that: The step of outputting the water level recognition result based on the predicted sequence text comprises: Determining whether the predicted sequence text is arranged according to a preset character arrangement rule of the water level gauge; If the predicted sequence text is not arranged according to the preset character arrangement rule of the water level gauge, the output water level recognition result is empty; If the predicted sequence text is arranged according to the preset character arrangement rule of the water level gauge, the numeric characters, water level gauge characters and Chinese characters in the predicted result of the sequence text are traversed to calculate the water level scale value, and the output water level recognition result is the water level scale value.

5. A water level identification device using the water level identification method according to claim 1, characterized in that: The water level identification device comprises: An acquisition module is used to acquire the water level gauge image collected by the camera; A preprocessing module, used for preprocessing the water gauge image to obtain a preprocessed first image; An extraction module, used for inputting the first image into a corner point information and semantic information extraction network model to obtain corner point information and a semantic feature map; A prediction box generation module, used to generate a candidate box based on the corner point information, evaluate the candidate box using the semantic feature map, and obtain a prediction box in the first image and corresponding prediction box information; A cropping module, used for cropping the first image based on the prediction frame information to obtain a water level gauge scale image; A sequence text recognition module, used for inputting the water gauge scale image into a sequence text recognition model to obtain a predicted sequence text; The output module is used to output the water level recognition result based on the prediction sequence text.

6. The water level identification device according to claim 5, characterized in that: The preprocessing module is further specifically used for: Detecting whether the percentage of the number of black pixels in the water gauge image is greater than a preset pigment ratio threshold; When it is detected that the percentage of the number of black pixels in the water gauge image is greater than a preset pigment ratio threshold, the water gauge image is proportionally mapped to the intensity channel through a multi-scale weighted average method to obtain a channel-enhanced image; Performing a convolution operation on the channel enhanced image using Gaussian filtering as a center surround function to obtain an illuminated component; Based on the illuminated component and the channel enhanced image, obtaining a logarithmic function of the reflected component; The logarithmic function of the reflection component is linearly quantized to obtain a first image.

7. A water level identification device, characterized in that: The water level identification device includes a processor, a memory, and a water level identification program stored in the memory and executable by the processor, wherein when the water level identification program is executed by the processor, the steps of the water level identification method as described in any one of claims 1 to 4 are implemented.

8. A readable storage medium, characterized in that: The readable storage medium stores a water level identification program, wherein when the water level identification program is executed by a processor, the steps of the water level identification method according to any one of claims 1 to 4 are implemented.

Images