Oil level identification method and device based on deep learning
Through deep learning technology, the oil level meter image is automatically identified and segmented, and the problems of low manual observation efficiency and large error of the oil level meter are solved, and the accurate and efficient automatic reading of the oil level meter is realized, ensuring production safety and efficiency.
Patent Information
- Application Number
- CN202411022681.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-29
- Publication Date
- 2024-12-13
AI Technical Summary
In the field of power grid, the visual background of the oil level meter is complex and the goals are small, resulting in subjective errors in manual observation of the oil level meter liquid level, low efficiency, and inconvenient on-site environmental hazards for direct observation, requiring an accurate and efficient automatic reading method.
Using the deep learning-based oil level recognition method, the oil level meter image is obtained, the oil level meter area is positioned using the target recognition network model, and the semantic segmentation process is performed, the area is divided into air and oil level areas, and the area is calculated to calculate the oil level readings. , using multi-scale networks, residual structures and blob analysis algorithms to improve accuracy.
It realizes fully automatic real-time reading of the oil level gauge, reduces labor costs, avoids manual subjective errors, ensures production safety and efficiency, and meets the accuracy and efficiency requirements of the oil level gauge reading.
Smart Images
Figure CN119131451A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of oil level detection of an oil level gauge, and in particular to an oil level recognition method and device based on deep learning. Background Art
[0002] In the field of power grid, there are many types of oil level gauges in substations, and the targets are small. The visual background of the oil level gauge is complex, and business personnel need to observe the liquid level of the oil level gauge to determine whether there is an abnormality. Due to the dangerous working environment on site, it is inconvenient for business personnel to directly observe on site, and the results of personnel observation are subjective, with large errors and low efficiency.
[0003] Therefore, there is an urgent need for a new method for accurate and efficient automatic reading of the oil level gauge. Summary of the invention
[0004] In response to the above technical problems, the present invention provides an oil level identification method and device based on deep learning, which can automatically perform real-time readings to ensure production safety and efficiency.
[0005] A first aspect of the present invention provides an oil level recognition method based on deep learning, comprising:
[0006] Acquire the captured image of the oil level gauge;
[0007] Using the target recognition network model to locate the oil level gauge area in the oil level gauge image, performing semantic segmentation processing on the oil level gauge area, and segmenting the oil level gauge area into an air area and an oil level area;
[0008] The area of the oil level region and the area of the air region are calculated, and the oil level reading of the oil level gauge is calculated according to the area of the oil level region and the area of the air region.
[0009] In an optional embodiment, the method of using a target recognition network model to locate the oil level gauge area in the oil level gauge image includes: using a multi-scale network model to perform feature extraction on oil level gauge images of different sizes, and performing a convolution operation on the oil level gauge image through a residual structure to obtain a feature tensor of the oil level gauge image, and locating the oil level gauge area in the oil level gauge image according to the feature tensor.
[0010] In an optional implementation manner, the performing semantic segmentation processing on the oil level gauge area to segment the oil level gauge area into an air area and an oil level area includes:
[0011] The Xception model is used to perform semantic segmentation on the oil level gauge area, and the resolution of feature extraction is controlled based on a dilated convolutional network;
[0012] A semantically segmented binary image is output based on the semantic segmentation processing result, and the air area and the oil level area are determined according to the binary image.
[0013] In an optional implementation manner, the outputting of a semantically segmented binary image based on the semantic segmentation processing result, and determining the air area and the oil level area according to the binary image, includes:
[0014] The air area and the oil level area are screened out according to the binary image using a blob analysis algorithm.
[0015] In an optional implementation manner, the method of screening out the air area and the oil level area according to the binary image using a blob analysis algorithm includes:
[0016] The pixels in the oil level area in the binary image are assigned a value of 255, and the pixels in the non-oil level area are assigned a value of 0; the connected areas of the oil level area are filled, and the largest oil level area is found through the blob analysis algorithm.
[0017] In an optional implementation, the pixels in the air area in the binary image are assigned a value of 255, and the pixels in the non-air area are assigned a value of 0; the connected areas of the air area are filled, and the air area with the largest area is found through a blob analysis algorithm.
[0018] In an optional implementation manner, when calculating the oil level reading of the oil level gauge according to the area of the oil level region and the area of the air region, the following formula is used:
[0019] oil_level_value=oil_area / (air_area+oil_area+1e -5 );
[0020] Where oil_area is the area of the oil level area of the oil level gauge, and air_area is the area of the air area of the oil level gauge.
[0021] A second aspect of the present invention provides an oil level recognition device based on deep learning, comprising:
[0022] An acquisition module, used for acquiring the photographed oil level gauge image;
[0023] a recognition module, used for locating the oil level gauge region in the oil level gauge image by using the target recognition network model, performing semantic segmentation processing on the oil level gauge region, and segmenting the oil level gauge region into an air region and an oil level region;
[0024] The reading module is used to calculate the area of the oil level area and the area of the air area, and calculate the oil level reading of the oil level gauge according to the area of the oil level area and the area of the air area.
[0025] According to a third aspect of the present invention, there is provided an electronic device, comprising:
[0026] At least one processor; and at least one memory communicatively connected to the processor, wherein: the memory stores program instructions executable by the processor, and the processor calls the program instructions to execute the method described in the first aspect of the embodiment of the present invention.
[0027] According to a fourth aspect of the present invention, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a computer, the method according to the first aspect of the embodiment of the present invention is executed.
[0028] The present invention utilizes image recognition technology to effectively solve the problem of slow efficiency of traditional manual inspections, reducing labor costs; avoiding erroneous readings caused by human subjective influences, ensuring production safety and efficiency, and meeting the accuracy and efficiency requirements of oil level gauge readings. BRIEF DESCRIPTION OF THE DRAWINGS
[0029] Figure 1 Schematic diagram of a flow chart of an oil level recognition method based on deep learning in an embodiment of the present invention.
[0030] Figure 2 Schematic diagram of an oil level gauge image in an embodiment of the present invention.
[0031] Figure 3 Schematic diagram of the air area and the oil level area of the oil level gauge in an embodiment of the present invention.
[0032] Figure 4 Schematic diagram of the air area and the oil level area of another oil level gauge in an embodiment of the present invention.
[0033] Figure 5 Schematic diagram of the air area and the oil level area of the oil level gauge in the binary diagram in an embodiment of the present invention.
[0034] Figure 6 It is a schematic diagram of the air area and the oil level area of the oil level gauge in another binary image in an embodiment of the present invention.
[0035] Figure 7 This is a module schematic diagram of an oil level recognition device based on deep learning in an embodiment of the present invention. DETAILED DESCRIPTION
[0036] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.
[0037] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.
[0038] See also Figure 1 The present invention provides an oil level identification method based on deep learning, which includes the following steps.
[0039] S10: Acquire the captured oil level gauge image.
[0040] The oil level gauge image can be taken by a preset camera point, or by a drone cruise.
[0041] In the field of power grids, since there are many types of oil level gauges, the angles and sizes of the images of the oil level gauges taken are inconsistent, which will affect image recognition. The present invention adopts a multi-scale strategy to better adapt to the oil level gauge images collected by the camera at different distances, which will be described in detail later.
[0042] S20: Utilize the target recognition network model to locate the oil level gauge area in the oil level gauge image, perform semantic segmentation processing on the oil level gauge area, and segment the oil level gauge area into an air area and an oil level area.
[0043] For example, the target detection algorithm predicts the location area and classification of the calibrated object of the candidate box. The present invention uses a convolutional neural network as the main body and completes the positioning prediction of the oil level gauge by extracting the features of the oil level gauge and using a multi-scale strategy.
[0044] The multi-scale strategy predicts image inputs at multiple scales (image pyramids) and then fuses the results. Multi-scale model integration can improve the performance of classification task models. The model uses the average value of prediction results at multiple scales to improve performance in tasks such as ImageNet.
[0045] After semantic segmentation processing is performed on the oil level gauge area, two mask areas can be obtained, corresponding to the air area and the oil level area respectively.
[0046] S30: Calculating the area of the oil level region and the area of the air region, and calculating the oil level reading of the oil level gauge according to the area of the oil level region and the area of the air region.
[0047] By counting the pixels of the air region and the oil level region, the area of the oil level region and the area of the air region may be calculated.
[0048] Then the ratio of the area of the oil level area to the total area of the oil level gauge is calculated to get the oil level reading of the oil level gauge. The total area of the oil level gauge is the sum of the areas of the oil level area and the air area.
[0049] In some embodiments, when calculating the oil level reading of the oil level gauge according to the area of the oil level area and the area of the air area, the following formula is used for calculation.
[0050] oil_level_value(value)=oil_area / (air_area+oil_area+1e -5 );Oil_area
[0051] is the area of the oil level area of the oil level gauge, and air_area is the area of the air area of the oil level gauge. -5 It refers to the calculation accuracy, which is 1 times 10 to the negative fifth power.
[0052] The present invention uses image recognition technology to realize the fully automatic real-time reading of the oil level gauge, ensures production safety and high efficiency, meets the accuracy and efficiency requirements of the oil level gauge reading, reduces labor costs, and avoids erroneous readings caused by human subjective influence.
[0053] Furthermore, given that the background of the oil level gauge image is usually complex and changeable, the types and shapes of the oil level gauges are very different, and the shapes of the oil level gauges in the image are also very different at different angles and distances. The remaining image information in the image except the oil level gauge will cause great inconvenience to the oil level gauge reading in this experiment. This situation will affect the efficiency, accuracy, and fast calculation performance of the oil level gauge reading. Therefore, it is necessary to select a robust and effective network model.
[0054] The present invention uses convolutional neural networks as the main body, and completes the positioning of the oil level gauge through oil level gauge feature extraction and multi-scale strategy. When the oil level gauge area in the oil level gauge image is located using the target recognition network model, the multi-scale network model is used to extract features for oil level gauge images of different sizes. The multi-scale network model can better adapt to the oil level gauge images collected by the camera at different distances, and the backbone network backbone with the fully connected layer removed is used as a feature extractor.
[0055] The multi-scale network model performs box marking and category prediction on the oil level gauge image from the output tensors of each scale. The output tensors of the deep network of the multi-scale network model are used to predict the large-scale oil level gauge; the output tensors of the shallow network of the multi-scale network model are used to predict the medium-scale and small-scale oil level gauges, respectively, and have good prediction effects for oil level gauges of different sizes.
[0056] Then, the oil level gauge image is convolved through the residual structure to obtain the feature tensor of the oil level gauge image, and the oil level gauge area in the oil level gauge image is located according to the feature tensor. The residual refers to the error between the predicted value and the actual value; the residual block structure is called a short-cut structure, which is translated into Chinese as a highway. It can effectively transmit the error value and facilitate neural network training and recognition. Figure 2 In the embodiment shown, the oil level gauge area is finally accurately extracted according to the outputted box coordinate information.
[0057] Then, the region of interest is extracted from the oil level gauge area to segment the oil level gauge area into an air area and an oil level area. Figure 3 and Figure 4 As shown in the figure, there are two images of oil level gauges. Figure 3 , Figure 4 The image shown uses traditional image segmentation technology, which can obtain several areas with common attributes and do not intersect each other according to certain rules. However, there are many types of oil level gauges in substations, which is not friendly to the adjustment of parameters of traditional image segmentation technology. In view of this, the present invention adopts semantic segmentation technology based on deep learning.
[0058] In some embodiments, the present invention uses the Xception model to perform semantic segmentation on the oil level gauge area. Semantic segmentation realizes pixel-level classification of images. It can divide the elements in the oil level gauge into multiple blocks according to the similarities and differences of categories. In the Xception model, the semantic segmentation (ASPP) and decoding modules use depthwise separable convolution to improve the running speed and robustness of the encoder-decoder network.
[0059] In order to fuse multi-scale information, the encoder-decoder architecture commonly used in semantic segmentation is introduced. In the encoder-decoder architecture, the resolution of feature extraction by the encoder is arbitrarily controlled. In some embodiments of the present invention, the resolution of feature extraction is controlled based on a dilated convolutional network. The dilated convolutional network has the following advantages: significantly expanding the receptive field, which is conducive to capturing a wider range of contextual information; improving model performance, which is conducive to fine segmentation and boundary positioning.
[0060] Finally, a semantically segmented binary image is output based on the semantic segmentation processing result, and the air area and the oil level area are determined according to the binary image.
[0061] For example, the pixel values corresponding to each element of the segmentation map are defined, air=100 in the air area and oil=50 in the oil level area. Figure 5 and Figure 6 As shown, the light-colored area in the upper layer of the figure is the air area, and the dark-colored area in the lower layer is the oil level area.
[0062] Furthermore, in some embodiments, the air area and the oil level area in the binary image of the oil level gauge are traversed. Considering that semantic segmentation is a statistical pixel classification algorithm, in order to ensure the accuracy of the oil level gauge readings, in the binary image after semantic segmentation, the present invention introduces traditional image processing technology to enhance the effect of semantic segmentation, and screens out the air area and the oil level gauge area through blob analysis.
[0063] Specifically, the air area and the oil level area are screened out according to the binary image using a blob analysis algorithm.
[0064] For example, the pixels in the oil level area in the binary image are assigned a value of 255, and the pixels in the non-oil level area are assigned a value of 0; the connected areas of the oil level area are filled, and the largest oil level area is found through the blob analysis algorithm.
[0065] The pixels in the air area in the binary image are assigned a value of 255, and the pixels in the non-air area are assigned a value of 0; the connected areas of the air area are filled, and the air area with the largest area is found through the blob analysis algorithm.
[0066] Compared with the prior art, the present invention effectively solves the problem of slow efficiency of traditional manual inspections and reduces labor costs; it avoids erroneous readings caused by subjective human influence, and at the same time, realizes fully automatic real-time reading of the oil level gauge, ensuring production safety and efficiency, and meeting the accuracy and efficiency requirements of the oil level gauge reading. The present invention uses a multi-scale strategy to process different types of oil level gauge images, and combines the residual structure network to accurately extract the oil level gauge area. The semantic segmentation is to use the dilated convolution to increase the accuracy of the extracted features, and the blob analysis algorithm is used to increase the semantic segmentation progress, ensuring the accuracy of the oil level gauge reading.
[0067] Corresponding to the above method, such as Figure 7 As shown, the present invention also provides an oil level identification device based on deep learning, comprising:
[0068] The acquisition module 71 is used to acquire the photographed oil level gauge image.
[0069] The recognition module 72 is used to locate the oil level gauge area in the oil level gauge image by using the target recognition network model, perform semantic segmentation processing on the oil level gauge area, and segment the oil level gauge area into an air area and an oil level area.
[0070] For example, a multi-scale network model is used to extract features of oil level gauge images of different sizes, and a convolution operation is performed on the oil level gauge image through a residual structure to obtain a feature tensor of the oil level gauge image, and the oil level gauge area in the oil level gauge image is located according to the feature tensor.
[0071] In some embodiments, the oil level gauge area is semantically segmented using an Xception model, and the resolution of feature extraction is controlled based on a dilated convolutional network; a binary image of semantic segmentation is output based on the result of the semantic segmentation processing, and the air area and the oil level area are determined based on the binary image.
[0072] For example, the air area and the oil level area are screened out according to the binary image using the blob analysis algorithm. The pixels of the oil level area in the binary image are assigned a value of 255, and the pixels of the non-oil level area are assigned a value of 0; the connected areas of the oil level area are filled, and the oil level area with the largest area is found through the blob analysis algorithm.
[0073] The pixels in the air area in the binary image are assigned a value of 255, and the pixels in the non-air area are assigned a value of 0; the connected areas of the air area are filled, and the air area with the largest area is found through the blob analysis algorithm.
[0074] The reading module 73 is used to calculate the area of the oil level region and the area of the air region, and calculate the oil level reading of the oil level gauge according to the area of the oil level region and the area of the air region.
[0075] The calculation is done using the following formula:
[0076] oil_level_value=oil_area / (air_area+oil_area+1e -5 );
[0077] Where oil_area is the area of the oil level area of the oil level gauge, and air_area is the area of the air area of the oil level gauge.
[0078] The functions or methods performed by the above modules can be referred to the description of the above-mentioned oil level recognition method based on deep learning, which will not be repeated here.
[0079] The present invention also provides an electronic device, comprising:
[0080] At least one processor; and at least one memory communicatively connected to the processor, wherein: the memory stores program instructions executable by the processor, and the processor calls the program instructions to execute the above-mentioned oil level identification method based on deep learning.
[0081] The present invention also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the above-mentioned oil level identification method based on deep learning is implemented.
[0082] It can be understood that computer-readable storage media may include: any entity or device capable of carrying a computer program, recording media, USB flash drives, mobile hard disks, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), and software distribution media, etc. A computer program includes computer program code. The computer program code may be in source code form, object code form, executable file, or some intermediate form, etc. A computer-readable storage medium may include: any entity or device capable of carrying a computer program code, recording media, USB flash drives, mobile hard disks, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), and software distribution media, etc.
[0083] In certain embodiments of the present invention, the device may include a controller, which is a single-chip microcomputer chip that integrates a processor, a memory, a communication module, etc. The processor may refer to the processor included in the controller. The processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc.
[0084] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, segment or portion of code that includes one or more executable instructions for implementing the steps of a specific logical function or process, and the scope of the preferred embodiments of the present invention includes alternative implementations in which functions may not be performed in the order shown or discussed, including performing functions in a substantially simultaneous manner or in the reverse order depending on the functions involved, which should be understood by technicians in the technical field to which the embodiments of the present invention belong.
[0085] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in terms of function in the above description. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.
[0086] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.
Claims
1. A method for identifying oil level based on deep learning, characterized in that: include: Acquire the captured oil level gauge image; Using the target recognition network model to locate the oil level gauge area in the oil level gauge image, performing semantic segmentation processing on the oil level gauge area, and segmenting the oil level gauge area into an air area and an oil level area; The area of the oil level region and the area of the air region are calculated, and the oil level reading of the oil level gauge is calculated according to the area of the oil level region and the area of the air region.
2. The oil level recognition method based on deep learning according to claim 1 is characterized in that: The method of using a target recognition network model to locate the oil level gauge area in the oil level gauge image includes: using a multi-scale network model to extract features for oil level gauge images of different sizes, and performing a convolution operation on the oil level gauge image through a residual structure to obtain a feature tensor of the oil level gauge image, and locating the oil level gauge area in the oil level gauge image according to the feature tensor.
3. The oil level recognition method based on deep learning according to claim 1 is characterized in that: The performing semantic segmentation processing on the oil level gauge area to segment the oil level gauge area into an air area and an oil level area includes: The Xception model is used to perform semantic segmentation on the oil level gauge area, and the resolution of feature extraction is controlled based on a dilated convolutional network; A semantically segmented binary image is output based on the semantic segmentation processing result, and the air area and the oil level area are determined according to the binary image.
4. The oil level recognition method based on deep learning according to claim 3 is characterized in that: The method of outputting a binary image of semantic segmentation based on the semantic segmentation processing result and determining an air area and an oil level area according to the binary image comprises: The air area and the oil level area are screened out according to the binary image using a blob analysis algorithm.
5. The oil level recognition method based on deep learning according to claim 4 is characterized in that: The method of using the blob analysis algorithm to screen out the air area and the oil level area according to the binary image includes: The pixels in the oil level area in the binary image are assigned a value of 255, and the pixels in the non-oil level area are assigned a value of 0; the connected areas of the oil level area are filled, and the largest oil level area is found through the blob analysis algorithm.
6. The oil level recognition method based on deep learning according to claim 4 or 5, characterized in that: The pixels in the air area in the binary image are assigned a value of 255, and the pixels in the non-air area are assigned a value of 0; the connected areas of the air area are filled, and the air area with the largest area is found through the blob analysis algorithm.
7. The oil level recognition method based on deep learning according to claim 1, characterized in that: When calculating the oil level reading of the oil level gauge according to the area of the oil level area and the area of the air area, the following formula is used: oil_level_value=oil_area / (air_area+oil_area+1e -5 ); Where oil_area is the area of the oil level area of the oil level gauge, and air_area is the area of the air area of the oil level gauge.
8. An oil level recognition device based on deep learning, characterized in that: include: An acquisition module, used for acquiring the photographed oil level gauge image; a recognition module, used for locating the oil level gauge region in the oil level gauge image by using the target recognition network model, performing semantic segmentation processing on the oil level gauge region, and segmenting the oil level gauge region into an air region and an oil level region; The reading module is used to calculate the area of the oil level area and the area of the air area, and calculate the oil level reading of the oil level gauge according to the area of the oil level area and the area of the air area.
9. An electronic device, characterized in that: include: at least one processor; And at least one memory communicatively connected to the processor, wherein: the memory stores program instructions executable by the processor, and the processor calls the program instructions to execute the oil level identification method based on deep learning as described in any one of claims 1 to 7.
10. A computer-readable storage medium, characterized in that: A computer program is stored thereon, and when the computer program is executed by a computer, the oil level identification method based on deep learning as described in any one of claims 1 to 7 is executed.