Liquid level meter liquid level identification method and device based on deep learning

Abstract

The invention discloses a liquid level meter liquid level identification method and device based on deep learning. The method comprises the steps that a liquid level meter image is acquired; identifying a liquid level meter area in the liquid level meter image according to a target identification network model, performing semantic segmentation processing on the liquid level meter area, and segmenting the liquid level meter area into an air area and a liquid level area; the liquid level height of the liquid level meter is determined according to the air area and the liquid level area, and the liquid level reading of the liquid level meter is calculated according to the sum of the liquid level height and the height of the air area and the height of the liquid level area. According to the method, the problem of low efficiency of traditional manual inspection is effectively solved by utilizing an image recognition technology, wrong reading caused by manual subjective influence is avoided, and the requirements of reading accuracy and reading efficiency of the liquid level meter are met.

Description

Technical Field

[0001] The present invention relates to the field of liquid level detection of a liquid level meter, and in particular to a liquid level recognition method and device for a liquid level meter based on deep learning. Background Art

[0002] In the field of power grid, there are many types of substation level meters and the targets are small. Business personnel need to observe the liquid level of the level meter to determine whether there is an abnormality. Due to the dangerous working environment on site, it is inconvenient for business personnel to directly observe on site, and the results of personnel observation are subjective, with large errors and low efficiency.

[0003] However, the shape of the level gauge is complex, and the reading method is also different. Some level gauges need to rely on height recognition, while others do not. Therefore, reading different types of level gauges also faces different technical challenges.

[0004] Therefore, there is an urgent need for a new method for accurate and efficient automatic reading of liquid level gauges. Summary of the invention

[0005] In response to the above technical problems, the present invention provides a liquid level recognition method and device for a liquid level gauge based on deep learning, which can automatically and efficiently read the liquid level gauge.

[0006] A first aspect of the present invention provides a method for identifying a liquid level of a liquid level meter based on deep learning, comprising:

[0007] Get the level gauge image;

[0008] Identify a liquid level gauge region in the liquid level gauge image according to the target recognition network model, perform semantic segmentation processing on the liquid level gauge region, and segment the liquid level gauge region into an air region and a liquid level region;

[0009] The liquid level height of the liquid level gauge is determined according to the air area and the liquid level area, and the liquid level reading of the liquid level gauge is calculated according to the liquid level height and the sum of the heights of the air area and the liquid level area.

[0010] In an optional implementation, semantic segmentation is performed on the liquid level gauge area to segment the liquid level gauge area into an air area and a liquid level area, including:

[0011] Based on the semantic segmentation processing result, a binary image of semantic segmentation is output, and the air area and the liquid level area are screened out according to the binary image using a blob analysis algorithm.

[0012] In an optional implementation, the using of a blob analysis algorithm to screen out air areas and liquid level areas according to the binary image includes:

[0013] The pixels in the liquid level area in the binary image are assigned a value of 255, and the pixels in the non-liquid level area are assigned a value of 0; the connected areas of the liquid level area are filled, and the liquid level area with the largest height is found through the blob analysis algorithm.

[0014] In an optional implementation, the method of using a blob analysis algorithm to filter out air areas and liquid level areas according to the binary image further includes:

[0015] The pixels in the air area in the binary image are assigned a value of 255, and the pixels in the non-air area are assigned a value of 0; the gradient change is calculated for the pixel sum of each row in the liquid level area and the pixel sum of the two rows above and below the row, and the row with the largest gradient change is taken as the liquid level line, and the area above the liquid level line is the air area.

[0016] In an optional implementation, the performing semantic segmentation processing on the liquid level gauge area to segment the liquid level gauge area into an air area and a liquid level area includes:

[0017] The Xception model is used to perform semantic segmentation on the liquid level gauge area, and the resolution of feature extraction is controlled based on the atrous convolutional network to output the semantic segmentation processing result.

[0018] In an optional embodiment, the identifying the liquid level gauge area in the liquid level gauge image according to the target recognition network model includes: using a multi-scale network model to extract features for liquid level gauge images of different sizes, and performing a convolution operation on the liquid level gauge image through a residual structure to obtain a feature tensor of the liquid level gauge image, and locating the liquid level gauge area in the liquid level gauge image according to the feature tensor.

[0019] In an optional implementation, when calculating the liquid level reading of the liquid level meter according to the height of the liquid level area and the height of the air area, the following formula is used:

[0020] liquid_level_value=liquid_h / (air_h+liquid_h);

[0021] Where liquid_h is the height of the liquid level area of ​​the level gauge, and air_h is the height of the air area of ​​the level gauge.

[0022] A second aspect of the present invention provides a liquid level recognition device for a liquid level meter based on deep learning, comprising:

[0023] An acquisition module, used for acquiring an image of a liquid level meter;

[0024] A recognition module, used for identifying a liquid level gauge region in the liquid level gauge image according to a target recognition network model, performing semantic segmentation processing on the liquid level gauge region, and segmenting the liquid level gauge region into an air region and a liquid level region;

[0025] The reading module is used to determine the liquid level height of the liquid level gauge according to the air area and the liquid level area, and calculate the liquid level reading of the liquid level gauge according to the liquid level height and the sum of the heights of the air area and the liquid level area.

[0026] According to a third aspect of the present invention, there is provided an electronic device, comprising:

[0027] At least one processor; and at least one memory communicatively connected to the processor, wherein: the memory stores program instructions executable by the processor, and the processor calls the program instructions to execute the method described in the first aspect of the embodiment of the present invention.

[0028] According to a fourth aspect of the present invention, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a computer, the method according to the first aspect of the embodiment of the present invention is executed.

[0029] The present invention utilizes image recognition technology to effectively solve the problem of low efficiency of traditional manual inspection, avoids erroneous readings caused by human subjective influence, and meets the requirements of liquid level meter reading accuracy and reading efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 Schematic diagram of a flow chart of a method for identifying liquid level of a liquid level gauge based on deep learning in an embodiment of the present invention.

[0031] Figure 2 Schematic diagram of a liquid level meter image captured in an embodiment of the present invention.

[0032] Figure 3 Schematic diagram of the liquid level meter area in an embodiment of the present invention.

[0033] Figure 4 Schematic diagram of a binary image of an air region and a liquid level region in an embodiment of the present invention.

[0034] Figure 5 Schematic diagram of a module of a liquid level recognition device of a liquid level gauge based on deep learning in an embodiment of the present invention. DETAILED DESCRIPTION

[0035] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0036] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0037] See also Figure 1 The present invention provides a liquid level recognition method for a liquid level gauge based on deep learning, comprising the following steps.

[0038] S10: Acquire the liquid level meter image.

[0039] The level gauge image can be taken by a preset camera point, or by a drone cruise. Figure 2 As shown, the captured image includes a liquid level gauge and a background image around the liquid level gauge.

[0040] In the field of power grids, since there are many types of liquid level meters, the angles and sizes of the images of the liquid level meters taken are inconsistent, which will affect image recognition. The present invention can solve the angle and size problems based on image recognition technology.

[0041] S20: Identify the liquid level gauge area in the liquid level gauge image according to the target recognition network model, perform semantic segmentation processing on the liquid level gauge area, and segment the liquid level gauge area into an air area and a liquid level area.

[0042] For example, the target detection algorithm predicts the location area and classification of the calibrated object of the candidate box. The present invention uses a convolutional neural network as the main body and completes the positioning prediction of the liquid level meter based on the feature extraction of the liquid level meter and the multi-scale strategy.

[0043] The present invention adopts a multi-scale strategy to better adapt to the level gauge images collected by the camera at different distances, wherein the multi-scale strategy predicts image inputs (image pyramids) of multiple scales and then fuses the results. Multi-scale model integration can improve the performance of classification task models, and the model uses the average value fusion of prediction results of multiple scales to improve performance in tasks such as ImageNet.

[0044] After semantic segmentation processing is performed on the liquid level gauge area, two mask areas can be obtained, corresponding to the air area and the liquid level area respectively. It should be understood that the air area refers to an area without oil.

[0045] S30: Determine the liquid level height of the liquid level gauge according to the air area and the liquid level area, and calculate the liquid level reading of the liquid level gauge according to the liquid level height and the sum of the heights of the air area and the liquid level area.

[0046] Through the semantic segmentation of the liquid level gauge area, the height of the liquid level area and the height of the air area can be segmented. Figure 3 As shown, the meter reading area of ​​the liquid level gauge area is processed into a binary image to distinguish the liquid level area from the air area.

[0047] Then the ratio of the height of the liquid level area to the total height of the level gauge is calculated to get the liquid level reading of the level gauge. The total height of the level gauge is the sum of the heights of the liquid level area and the air area.

[0048] In some embodiments, when calculating the liquid level reading of the liquid level meter according to the height of the liquid level area and the height of the air area, the following formula is used for calculation.

[0049] liquid_level_value=liquid_h / (air_h+liquid_h);

[0050] Where liquid_h is the height of the liquid level area of ​​the level gauge, and air_h is the height of the air area of ​​the level gauge.

[0051] The present invention uses image recognition technology to realize the full-automatic real-time reading of the liquid level meter, ensures production safety and high efficiency, meets the requirements of liquid level meter reading accuracy and reading efficiency, reduces labor costs, and avoids erroneous readings caused by human subjective influence.

[0052] Furthermore, given that the background of the level gauge image is usually complex and changeable, the types and shapes of the level gauges are very different, and the shapes of the level gauges in the image are also very different at different angles and distances. The remaining image information in the image except the level gauge will cause great inconvenience to the level gauge reading in this experiment. This situation will affect the efficiency, accuracy, and fast calculation performance of the level gauge reading. Therefore, it is necessary to select a robust and effective network model.

[0053] The present invention uses convolutional neural networks as the main body, and completes the positioning of the liquid level meter through liquid level meter feature extraction and multi-scale strategy. When the liquid level meter area in the liquid level meter image is identified according to the target recognition network model, a multi-scale network model is used to extract features for liquid level meter images of different sizes. The multi-scale network model can better adapt to the liquid level meter images collected by the camera at different distances, and the backbone network backbone with the fully connected layer removed is used as a feature extractor.

[0054] The multi-scale network model performs box marking and category prediction on the level gauge image from the output tensors of each scale. The output tensors of the deep network of the multi-scale network model are used to predict the large-scale level gauge; the output tensors of the shallow network of the multi-scale network model are used to predict the medium-scale and small-scale level gauges respectively, and have good prediction effects for level gauges of different sizes.

[0055] Then, the residual structure is used to perform a convolution operation on the level gauge image to obtain the feature tensor of the level gauge image, and the level gauge area in the level gauge image is located according to the feature tensor. The residual refers to the error between the predicted value and the actual value. The residual block structure is called a short-cut structure, which is translated into Chinese as a highway. It can effectively transmit error values ​​and facilitate neural network training and recognition. Figure 3 In the embodiment shown, the liquid level meter area is finally accurately extracted according to the outputted box coordinate information.

[0056] Then, the region of interest is extracted from the liquid level gauge area to obtain the liquid level gauge area divided into an air area and a liquid level area. Figure 3 As shown, for Figure 3 The image shown uses traditional image segmentation technology, which can obtain several areas with common attributes and do not intersect each other according to certain rules. However, there are many types of substation level meters, which is not friendly to the adjustment of traditional image segmentation technology. In view of this, the present invention adopts semantic segmentation technology based on deep learning.

[0057] In some embodiments, the present invention uses the Xception model to perform semantic segmentation on the liquid level gauge area. Semantic segmentation realizes pixel-level classification of images. It can divide the image into multiple blocks according to the similarities and differences of the categories of each element in the liquid level gauge. In the Xception model, the semantic segmentation (ASPP) and decoding modules use depthwise separable convolution to improve the running speed and robustness of the encoder-decoder network.

[0058] In order to fuse multi-scale information, the encoder-decoder architecture commonly used in semantic segmentation is introduced. In the encoder-decoder architecture, the resolution of feature extraction by the encoder is arbitrarily controlled. In some embodiments of the present invention, the resolution of feature extraction is controlled based on a dilated convolutional network. The dilated convolutional network has the following advantages: significantly expanding the receptive field, which is conducive to capturing a wider range of contextual information; improving model performance, which is conducive to fine segmentation and boundary positioning.

[0059] Finally, a binary image of semantic segmentation is output based on the semantic segmentation processing result, and the air area and the liquid level area are determined according to the binary image.

[0060] For example, the pixel values ​​corresponding to each element of the segmentation map are defined, air=60 in the air area and oil=30 in the liquid level area. Figure 4 As shown, the light-colored area in the upper layer of the figure is the air area, and the dark-colored area in the lower layer is the liquid level area.

[0061] Furthermore, in some embodiments, the air area and the liquid level area in the binary image of the liquid level meter are traversed. Considering that semantic segmentation is a statistical pixel classification algorithm, in order to ensure the reading accuracy of the liquid level meter, in the binary image after semantic segmentation, the present invention introduces traditional image processing technology to enhance the effect of semantic segmentation, and screens out the air area and the liquid level meter area through blob analysis.

[0062] Specifically, the air area and the liquid level area are screened out according to the binary image using a blob analysis algorithm.

[0063] For example, the pixels in the liquid level area in the binary image are assigned a value of 255, and the pixels in the non-liquid level area are assigned a value of 0; the connected areas of the liquid level area are filled, and the liquid level area with the maximum height is found through the blob analysis algorithm.

[0064] Then, the pixels in the air area in the binary image are assigned a value of 255, and the pixels in the non-air area are assigned a value of 0; the gradient change is calculated for the pixel sum of each row in the liquid level area and the pixel sum of the two rows above and below the row, and the row with the largest gradient change is taken as the liquid level line, and the area above the liquid level line is the air area, as shown in the following example. Figure 4 shown.

[0065] Compared with the prior art, the present invention effectively solves the problem of slow efficiency of traditional manual inspections and reduces labor costs; it avoids erroneous readings caused by subjective human influence, and at the same time, realizes fully automatic real-time readings of the level meter, ensuring production safety and efficiency, and meeting the accuracy and efficiency requirements of the level meter readings. The present invention uses a multi-scale strategy to process different types of level meter images, and combines the residual structure network to accurately extract the level meter area. The semantic segmentation is to use the dilated convolution to increase the accuracy of the extracted features, and the blob analysis algorithm is used to increase the semantic segmentation progress, ensuring the accuracy of the level meter readings.

[0066] Corresponding to the above method, such as Figure 5 As shown, the present invention also provides a liquid level recognition device for a liquid level meter based on deep learning, comprising:

[0067] The acquisition module 51 is used to acquire the liquid level meter image.

[0068] The recognition module 52 is used to recognize the liquid level gauge area in the liquid level gauge image according to the target recognition network model, perform semantic segmentation processing on the liquid level gauge area, and divide the liquid level gauge area into an air area and a liquid level area.

[0069] For example, a binary image of semantic segmentation is output based on the result of semantic segmentation processing, and the air area and the liquid level area are screened out according to the binary image using a blob analysis algorithm.

[0070] In some embodiments, the pixels in the liquid level area in the binary image are assigned a value of 255, and the pixels in the non-liquid level area are assigned a value of 0; the connected areas of the liquid level area are filled, and the liquid level area with the largest height is found through a blob analysis algorithm.

[0071] The pixels in the air area in the binary image are assigned a value of 255, and the pixels in the non-air area are assigned a value of 0; the gradient change is calculated for the pixel sum of each row in the liquid level area and the pixel sum of the two rows above and below the row, and the row with the largest gradient change is taken as the liquid level line, and the area above the liquid level line is the air area.

[0072] In some embodiments, the Xception model is used to perform semantic segmentation processing on the liquid level meter area, and the resolution of feature extraction is controlled based on a hole convolutional network to output the semantic segmentation processing result.

[0073] Furthermore, a multi-scale network model is used to extract features of level gauge images of different sizes, and a convolution operation is performed on the level gauge image through a residual structure to obtain a feature tensor of the level gauge image, and the level gauge area in the level gauge image is located according to the feature tensor.

[0074] The reading module 53 is used to calculate the height of the liquid level area and the height of the air area, and calculate the liquid level reading of the liquid level meter according to the height of the liquid level area and the height of the air area.

[0075] The calculation is done using the following formula:

[0076] liquid_level_value=liquid_h / (air_h+liquid_h); liquid_h is the height of the liquid level area of ​​the level gauge, and air_h is the height of the air area of ​​the level gauge.

[0077] The functions or methods performed by the above modules can be referred to the description of the above-mentioned liquid level recognition method of the liquid level gauge based on deep learning, and will not be repeated here.

[0078] The present invention also provides an electronic device, comprising:

[0079] At least one processor; and at least one memory communicatively connected to the processor, wherein: the memory stores program instructions executable by the processor, and the processor calls the program instructions to execute the above-mentioned liquid level recognition method of the liquid level gauge based on deep learning.

[0080] The present invention also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the above-mentioned liquid level recognition method of a liquid level meter based on deep learning is implemented.

[0081] It can be understood that computer-readable storage media may include: any entity or device capable of carrying a computer program, recording media, USB flash drives, mobile hard disks, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), and software distribution media, etc. A computer program includes computer program code. The computer program code may be in source code form, object code form, executable file, or some intermediate form, etc. A computer-readable storage medium may include: any entity or device capable of carrying a computer program code, recording media, USB flash drives, mobile hard disks, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), and software distribution media, etc.

[0082] In certain embodiments of the present invention, the device may include a controller, which is a single-chip microcomputer chip that integrates a processor, a memory, a communication module, etc. The processor may refer to the processor included in the controller. The processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc.

[0083] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, segment or portion of code that includes one or more executable instructions for implementing the steps of a specific logical function or process, and the scope of the preferred embodiments of the present invention includes alternative implementations in which functions may not be performed in the order shown or discussed, including performing functions in a substantially simultaneous manner or in the reverse order depending on the functions involved, which should be understood by technicians in the technical field to which the embodiments of the present invention belong.

[0084] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in terms of function in the above description. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0085] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A liquid level recognition method for a liquid level gauge based on deep learning, characterized in that: include: Get the level gauge image; Identify a liquid level gauge region in the liquid level gauge image according to the target recognition network model, perform semantic segmentation processing on the liquid level gauge region, and segment the liquid level gauge region into an air region and a liquid level region; The liquid level height of the liquid level gauge is determined according to the air area and the liquid level area, and the liquid level reading of the liquid level gauge is calculated according to the liquid level height and the sum of the heights of the air area and the liquid level area.

2. The method for identifying liquid level of a liquid level gauge based on deep learning according to claim 1, characterized in that: Performing semantic segmentation processing on the liquid level gauge area to segment the liquid level gauge area into an air area and a liquid level area, including: Based on the semantic segmentation processing result, a binary image of semantic segmentation is output, and the air area and the liquid level area are screened out according to the binary image using a blob analysis algorithm.

3. The method for identifying liquid level of a liquid level gauge based on deep learning according to claim 2, characterized in that: The method of using the blob analysis algorithm to screen out the air area and the liquid level area according to the binary image includes: The pixels in the liquid level area in the binary image are assigned a value of 255, and the pixels in the non-liquid level area are assigned a value of 0; the connected areas of the liquid level area are filled, and the liquid level area with the largest height is found through the blob analysis algorithm.

4. The method for identifying liquid level of a liquid level gauge based on deep learning according to claim 3, characterized in that: The method of using the blob analysis algorithm to filter out the air area and the liquid level area according to the binary image also includes: The pixels in the air area in the binary image are assigned a value of 255, and the pixels in the non-air area are assigned a value of 0; the gradient change is calculated for the pixel sum of each row in the liquid level area and the pixel sum of the two rows above and below the row, and the row with the largest gradient change is taken as the liquid level line, and the area above the liquid level line is the air area.

5. The method for identifying liquid level of a liquid level gauge based on deep learning according to claim 1, characterized in that: The performing semantic segmentation processing on the liquid level gauge area to segment the liquid level gauge area into an air area and a liquid level area includes: The Xception model is used to perform semantic segmentation on the liquid level gauge area, and the resolution of feature extraction is controlled based on the atrous convolutional network to output the semantic segmentation processing result.

6. The method for identifying liquid level of a liquid level gauge based on deep learning according to claim 1, characterized in that: The method of identifying the liquid level gauge area in the liquid level gauge image according to the target recognition network model includes: using a multi-scale network model to extract features for liquid level gauge images of different sizes, and performing a convolution operation on the liquid level gauge image through a residual structure to obtain a feature tensor of the liquid level gauge image, and locating the liquid level gauge area in the liquid level gauge image according to the feature tensor.

7. The method for identifying liquid level of a liquid level gauge based on deep learning according to claim 1, characterized in that: When calculating the liquid level reading of the liquid level gauge according to the height of the liquid level area and the height of the air area, the following formula is used: liquid_level_value=liquid_h / (air_h+liquid_h); Where liquid_h is the height of the liquid level area of ​​the level gauge, and air_h is the height of the air area of ​​the level gauge.

8. A liquid level recognition device for a liquid level gauge based on deep learning, characterized in that: include: An acquisition module, used for acquiring an image of a liquid level meter; A recognition module, used for identifying a liquid level gauge region in the liquid level gauge image according to a target recognition network model, performing semantic segmentation processing on the liquid level gauge region, and segmenting the liquid level gauge region into an air region and a liquid level region; The reading module is used to determine the liquid level height of the liquid level gauge according to the air area and the liquid level area, and calculate the liquid level reading of the liquid level gauge according to the liquid level height and the sum of the heights of the air area and the liquid level area.

9. An electronic device, characterized in that: include: at least one processor; And at least one memory communicatively connected to the processor, wherein: the memory stores program instructions executable by the processor, and the processor calls the program instructions to execute the liquid level recognition method of the liquid level meter based on deep learning as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: A computer program is stored thereon, and when the computer program is executed by a computer, the method for identifying the liquid level of a liquid level gauge based on deep learning as claimed in any one of claims 1 to 7 is executed.

Images